AWS Identity Federation

AWS - IAM Roles for Service Accounts (IRSA)

In 2014, AWS Identity and Access Management added support for federated identities using OpenID Connect (OIDC). This feature allows you to authenticate AWS API calls with supported identity providers and receive a valid OIDC JSON web token (JWT). You can pass this token to the AWS STS AssumeRoleWithWebIdentity API operation and receive IAM temporary role credentials. You can use these credentials to interact with any AWS service, including Amazon S3 and DynamoDB.

Enable IAM roles for service accounts by completing the following procedures:

1. Create an IAM OIDC provider for your cluster – You only complete this procedure once for each cluster.
2. Assign IAM roles to Kubernetes service accounts – Complete this procedure for each unique set of permissions that you want an application to have.
3. Configure DGX Cloud Lepton workloads to use a Kubernetes service account – Complete this procedure for each workload that needs access to AWS services.
4. Use IRSA with the AWS SDK – Confirm that the workload uses an AWS SDK of a supported version and that the workload uses the default credential chain.

This guide will take you through each of the above steps.

Create OIDC Provider (AWS Console)

1. Open the DGX Cloud Lepton console.
2. In the top menu bar, select Nodes, and then select the name of your NodeGroup on the Nodes page.
3. In the Details section, note the value of the OIDC provider URL.
4. Open the IAM console at https://console.aws.amazon.com/iam/.
5. In the left navigation pane, choose Identity Providers under Access management. If a provider is listed that matches the URL for your cluster, then you already have a provider for your cluster. If a provider isn't listed that matches the URL for your cluster, then you must create one.
6. To create a provider, choose Add provider.
7. For Provider type, select OpenID Connect.
8. For Provider URL, enter the OIDC provider URL for your cluster.
9. For Audience, enter https://kubernetes.default.svc.
10. (Optional) Add any tags, for example, a tag to identify which NodeGroup this provider is for.
11. Choose Add provider.

Assign IAM Roles to Kubernetes Service Accounts

The first step is to create a new S3 bucket if necessary in the AWS console (or CLI). Note that the region depends on where you provision the S3 bucket. Be sure to select the correct region in the console after logging in. In the following instructions, we will use a $BUCKET_NAME of my-s3-bucket-$ACCOUNT_ID.

Capture your AWS account ID to a variable:

Once the S3 bucket and OIDC identity provider are created, set up Bash environment variables specific to the AWS-related requirements. Note that the $S3_REGION will depend on where you provisioned your bucket.

DGX Cloud Lepton, by default, will prepend the namespace with ws-, so the workspace ID xyzpdq123 will become the Kubernetes namespace ws-xyzpdq123. Next, define some variables for the Kubernetes-related resources that will be used later. Specify a different $BUCKET_NAME if you used a custom name in the AWS console.

Now create a JSON file that defines an AWS Security Token Service (STS) role policy using the OIDC URL. Note the use of Bash variable manipulation to omit the scheme from the URL. Then create the role using the AWS CLI:

Now define and create the AWS policy for accessing the S3 bucket in another JSON file:

Now that you have created both a role and bucket access policy, attach the policy to that role:

Configure DGX Cloud Lepton Workloads to Use a Kubernetes Service Account

Kubernetes has the concept of service accounts as resources, which are essentially non-human identities that can interact with the API server and other resources. Every DGX Cloud Lepton workspace runs workloads as the default service account. Here we set up the workloads (container) with the three environment variables needed for the AWS SDK to utilize the token and above configuration. Place the following lines in your container startup (replacing the ACCOUNT_ID, ROLE_NAME, and S3_REGION appropriately):

Use IRSA with the AWS SDK

Install the AWS CLI in your workload and run the following commands to confirm (replace BUCKET_NAME with the appropriate bucket name):

See also https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-minimum-sdk.html for full details on various AWS SDKs.