Pulling Images from Amazon Elastic Container Registry

In order to pull container images from ECR, the container engine needs to authenticate the request. What makes ECR tricky is the credentials expire after 12 hours. So, how do we continuously ensure credentials are ready to go?

To solve this, we are going to do the following:

1. Configure an AWS IAM role that can obtain ECR credentials
2. Create a CronJob that gets new credentials every 8 hours and stores them as a Kubernetes secret
3. Configure our application to use the secrets to pull the images

Obtaining ECR Credentials

To easily configure an IAM role that can be used to obtain ECR credentials, we suggest using the Platform Role Connector described in Assuming an AWS IAM Role. By using the following Terraform, we are going to define an IAM role named ecr-credential-fetcher that has read-only access to ECR.

# Create the role and the Identity Provider needed to allow assume the role from the platform
module "plaform-role-connector" {
  source                   = "git@code.vt.edu:it-common-platform/support/terraform/aws-platform-role-connector.git"
  role_name                = "ecr-credential-fetcher"
  role_description         = "A role for the Common Platform to fetch ECR credentials"
  k8s_namespace            = "<your-tenant-identifier>"
  k8s_service_account_name = "ecr-cred-updater"

  role_tags = {
    ResponsibleParty = "<your-pid>"
  }
}

# Attach the ECR Read-Only policy to the role created by the Role Connector
resource "aws_iam_role_policy_attachment" "ecr_read_only" {
  role       = module.platform-role-connector.role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
}

Create a CronJob to Get and Store Credentials

To get you going, we have created a Helm chart that will define everything you need! You're welcome to read the README for all configuration options, but we'll keep it simple here.

To deploy your Helm chart, define the following manifest in your manifest repo. Note that you will need to change the value of iamRoleArn at the end of the file to by the full ARN created during the Terraform setup.

apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: HelmRepository
metadata:
  name: ecr-pull-credential-fetcher
spec:
  url: https://code.vt.edu/api/v4/projects/14546/packages/helm/stable
  interval: 1h
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: ecr-pull-credential-fetcher
spec:
  interval: 1h
  chart:
    spec:
      chart: ecr-pull-credential-fetcher
      version: "0.1.0"
      sourceRef:
        kind: HelmRepository
        name: ecr-pull-credential-fetcher
      interval: 1h
  values:
    aws:
      iamRoleArn: <your-iam-role-arn>

Once you commit the manifest to your manifest repo, you should see the Pods startup fairly soon.

Use the ECR Credentials to Pull your Images

Now that we have a secret defined, we can use it to pull our container images! By simply adding imagePullSecrets, you're good to go! The default name of the secret created by the Helm chart we used is aws-registry. If you ever need to change it, check out the documentation for the Helm chart.

apiVersion: v1
kind: Pod
spec:
  imagePullSecrets:
    - name: aws-registry
  containers:
    - name: app
      image: 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:latest

And that's it! If you have any questions or feedback, don't hesitate to let the Platform team know!