Help improve this page
To contribute to this user guide, choose the Edit this page on GitHub link that is located in the right pane of every page.
Create a storage class
A StorageClass in Amazon EKS Auto Mode defines how Amazon EBS volumes are automatically provisioned when applications request persistent storage. This page explains how to create and configure a StorageClass that works with the Amazon EKS Auto Mode to provision EBS volumes.
By configuring a StorageClass, you can specify default settings for your EBS volumes including volume type, encryption, IOPS, and other storage parameters. You can also configure the StorageClass to use AWS KMS keys for encryption management.
EKS Auto Mode does not create a StorageClass for you. You must create a StorageClass referencing ebs.csi.eks.amazonaws.com to use the storage capability of EKS Auto Mode.
First, create a file named storage-class.yaml:
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: auto-ebs-sc annotations: storageclass.kubernetes.io/is-default-class: "true" allowedTopologies: - matchLabelExpressions: - key: eks.amazonaws.com/compute-type values: - auto provisioner: ebs.csi.eks.amazonaws.com volumeBindingMode: WaitForFirstConsumer parameters: type: gp3 encrypted: "true"
Second, apply the storage class to your cluster.
kubectl apply -f storage-class.yaml
Key components:
-
provisioner: ebs---csi---eks.amazonaws.com.rproxy.goskope.com- Uses EKS Auto Mode -
allowedTopologies- SpecifyingmatchLabelExpressionsto match oneks.amazonaws.com/compute-type:autowill ensure that if your pods need a volume to be automatically provisioned using Auto Mode then the pods will not be scheduled on non-Auto nodes. -
volumeBindingMode: WaitForFirstConsumer- Delays volume creation until a pod needs it -
type: gp3- Specifies the EBS volume type -
encrypted: "true"- EBS will encrypt any volumes created using theStorageClass. EBS will use the defaultaws/ebskey alias. For more information, see How Amazon EBS encryption works in the Amazon EBS User Guide. This value is optional but suggested. -
storageclass.kubernetes.io/is-default-class: "true"- Kubernetes will use this storage class by default, unless you specify a different volume class on a persistent volume claim. This value is optional. Use caution when setting this value if you are migrating from a different storage controller.
Use self-managed KMS key to encrypt EBS volumes
To use a self-managed KMS key to encrypt EBS volumes automated by EKS Auto Mode, you need to:
-
Create a self-managed KMS key.
-
For more information, see Create a symmetric encryption KMS key or How Amazon Elastic Block Store (Amazon EBS) uses KMS in the KMS User Guide.
-
-
Create a new policy that permits access to the KMS key.
-
Use the sample IAM policy below to create the policy. Insert the ARN of the new self-managed KMS key. For more information, see Creating roles and attaching policies (console) in the AWS IAM User Guide.
-
-
Attach the policy to the EKS Cluster Role.
-
Use the AWS console to find the ARN of the EKS Cluster Role. The role information is visible in the Overview section. For more information, see Amazon EKS cluster IAM role.
-
-
Update the
StorageClassto reference the KMS Key ID at theparameters.kmsKeyIdfield.
Sample self-managed KMS IAM Policy
Update the following values in the policy below:
-
<account-id>– Your AWS account ID, such as111122223333 -
<aws-region>– The AWS region of your cluster, such asus-west-2
{ "Version":"2012-10-17", "Id": "key-auto-policy-3", "Statement": [ { "Sid": "Enable IAM User Permissions", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::123456789012:root" }, "Action": "kms:*", "Resource": "*" }, { "Sid": "Allow access through EBS for all principals in the account that are authorized to use EBS", "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": [ "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:CreateGrant", "kms:DescribeKey" ], "Resource": "*", "Condition": { "StringEquals": { "kms:CallerAccount": "123456789012", "kms:ViaService": "ec2.us-east-1.amazonaws.com" } } } ] }
Sample self-managed KMS StorageClass
parameters: type: gp3 encrypted: "true" kmsKeyId: <custom-key-arn>
StorageClass Parameters Reference
For general information on the Kubernetes StorageClass resources, see Storage Classes
The parameters section of the StorageClass resource is specific to AWS. Use the following table to review available options.
| Parameters | Values | Default | Description |
|---|---|---|---|
|
"csi.storage.k8s.io/fstype" |
xfs, ext2, ext3, ext4 |
ext4 |
File system type that will be formatted during volume creation. This parameter is case sensitive! |
|
"type" |
io1, io2, gp2, gp3, sc1, st1, standard, sbp1, sbg1 |
gp3 |
EBS volume type. |
|
"iopsPerGB" |
I/O operations per second per GiB. Can be specified for IO1, IO2, and GP3 volumes. |
||
|
"allowAutoIOPSPerGBIncrease" |
true, false |
false |
When |
|
"iops" |
I/O operations per second. Can be specified for IO1, IO2, and GP3 volumes. |
||
|
"throughput" |
125 |
Throughput in MiB/s. Only effective when gp3 volume type is specified. |
|
|
"encrypted" |
true, false |
false |
Whether the volume should be encrypted or not. Valid values are "true" or "false". |
|
"blockExpress" |
true, false |
false |
Enables the creation of io2 Block Express volumes. |
|
"kmsKeyId" |
The full ARN of the key to use when encrypting the volume. If not specified, AWS will use the default KMS key for the region the volume is in. This will be an auto-generated key called |
||
|
"blockSize" |
The block size to use when formatting the underlying filesystem. Only supported on linux nodes and with fstype |
||
|
"inodeSize" |
The inode size to use when formatting the underlying filesystem. Only supported on linux nodes and with fstype |
||
|
"bytesPerInode" |
The |
||
|
"numberOfInodes" |
The |
||
|
"ext4BigAlloc" |
true, false |
false |
Changes the |
|
"ext4ClusterSize" |
The cluster size to use when formatting an |
For more information, see the AWS EBS CSI Driver
Considerations
Note
You can only deploy workloads depending on EKS Auto Mode StorageClasses on EKS Auto Mode nodes. If you have a cluster with mixed types of nodes, you need to configure your workloads to run only on EKS Auto Mode nodes. For more information, see Control if a workload is deployed on EKS Auto Mode nodes.
The block storage capability of EKS Auto Mode is different from the EBS CSI Driver.
-
Static Provisioning
-
If you want to use externally-created EBS volumes with EKS Auto Mode, you need to manually add an AWS tag with the key
eks:eks-cluster-nameand the value of the cluster name.
-
-
Node Startup Taint
-
You cannot use the node startup taint feature to prevent pod scheduling before storage capability readiness
-
-
Custom Tags on Dynamically Provisioned Volumes
-
You cannot use the extra-tag CLI flag to configure custom tags on dynamically provisioned EBS volumes
-
You can use
StorageClasstagging to add custom tags. EKS Auto Mode will add tags to the associated AWS resources. You will need to update the Cluster IAM Role for custom tags. For more information, see Custom AWS tags for EKS Auto resources.
-
-
EBS Detailed Performance Metrics
-
You cannot access Prometheus metrics for EBS detailed performance
-
Annotation-based EBS volume modification is deprecated
The volume-modifier-for-k8sVolumeAttributesClasstype, iops, or throughput) after that date, use the Kubernetes VolumeAttributesClass API instead. For more information, see Volume Attributes Classes
Migrate from volume-modifier-for-k8s to VolumeAttributesClass
For EKS Auto Mode clusters running Kubernetes 1.34 or later, use a VolumeAttributesClass (VAC) to change an EBS volume’s type, IOPS, or throughput.
Existing PVCs can be migrated in place by creating a VAC and setting spec.volumeAttributesClassName on the existing PVC. The PVC and PV are not recreated and there are no interruptions to your workloads during this process.
-
Define the volume settings in a VAC
VMK annotations map to VAC parameters as follows:
Existing VMK annotation VAC parameter ebs.csi.eks.amazonaws.com/volumeTypetype
ebs.csi.eks.amazonaws.com/iopsiops
ebs.csi.eks.amazonaws.com/throughputthroughput
Use
type, notvolumeType, in the VAC. Values must be strings, and a VAC only needs the parameters relevant to the desired EBS configuration. Example:apiVersion: storage.k8s.io/v1 kind: VolumeAttributesClass metadata: name: ebs-auto-gp3-6000-250 driverName: ebs.csi.eks.amazonaws.com parameters: type: gp3 iops: "6000" throughput: "250"VACs are cluster-scoped, so the manifest has no namespace. Apply it normally:
kubectl apply -f vac.yaml -
Assign the VAC to the PVC
VAC_NAME=ebs-auto-gp3-6000-250 kubectl patch pvc <pvc-name> -n <namespace> --type=merge \ -p "{\"spec\":{\"volumeAttributesClassName\":\"${VAC_NAME}\"}}" -
Confirm that the change completed
kubectl get pvc <pvc-name> -n <namespace> \ -o custom-columns='NAME:.metadata.name,REQUESTED:.spec.volumeAttributesClassName,CURRENT:.status.currentVolumeAttributesClassName,STATUS:.status.modifyVolumeStatus.status'The migration is complete when
REQUESTEDandCURRENTcontain the same VAC name andSTATUSis empty.If it does not complete, inspect the PVC events:
kubectl describe pvc <pvc-name> -n <namespace>Pendingusually means the referenced VAC is unavailable.Infeasiblemeans the driver rejected the requested EBS parameters; create a valid VAC and assign that instead.
Optional: Remove the VMK annotations
Removing the old annotations is not required for VAC to work. Once a PVC or PV references a VAC, VMK refuses annotation-based modifications and VAC is the authoritative path.
The annotations may be removed later as configuration cleanup. If they are removed, also update any Helm, GitOps, or other automation that would add them back. VMK status annotations left on the PV are harmless and do not need to be removed.
Changing or Reverting the Settings
VAC parameters are immutable. To change the EBS configuration, create another VAC and assign its name to the PVC.
To cancel a change that has not completed, restore the previously assigned VAC. If the PVC did not previously use one, clear the field:
kubectl patch pvc <pvc-name> -n <namespace> --type=merge \ -p '{"spec":{"volumeAttributesClassName":null}}'
Clearing the field does not undo an EBS modification that already completed. To restore previous volume settings, create a VAC containing those settings and assign it to the PVC. Normal EBS parameter constraints and modification cooldowns apply.
Install CSI Snapshot Controller add-on
EKS Auto Mode is compatible with the CSI Snapshot Controller Amazon EKS add-on.
AWS suggests you configure this add-on to run on the built-in system node pool.
For more information, see:
To install snapshot controller in system node pool
-
Open your EKS cluster in the AWS console
-
From the Add-ons tab, select Get more add-ons
-
Select the CSI Snapshot Controller and then Next
-
On the Configure selected add-ons settings page, select Optional configuration settings to view the Add-on configuration schema
-
Insert the following yaml to associate the snapshot controller with the
systemnode pool. The snapshot controller includes a toleration for theCriticalAddonsOnlytaint.{ "nodeSelector": { "karpenter.sh/nodepool": "system" } } -
Select Next
-
-
Review the add-on configuration and then select Create