• Overview
• Solution
• Target Architecture
• Key Features
• Limitations
• Pre Requisite
• How to Deploy
• Verification
• Run Image pipeline
• Parameter Details
• Clean Up
• Conclusion

Golden Amazon Machine Image (AMI) pipeline enables creation, distribution, verification, launch-compliance of the AMI and creates a continuous and repeatable process for the consumers to generate the golden AMI. Currently when a user wants to build, test and distribute a Golden AMI across multiple accounts/regions, user will have to create multiple resources including image recipes, build and test steps, infrastructure creation and distribution step and connect them together in image pipeline using EC2 Image Builder service. User needs to define each stage and resources required for each stage of the image builder pipeline which is a lengthy process

In this solution, we will be using EC2 Image Builder service for the heavy lifting work of building, testing and distributing the Golden AMI. The code repository contains all the configuration files provided the user. These configuration files will define how the AMI will be built, tested and distributed across multiple accounts/regions. AWS Cloud Development Kit (AWS CDK) application will read configuration file ( the details of the configuration file provided by user is described in Configuration File section) and deploy the necessary resources to create AMI Pipeline.

Amazon Machine Image (AMI) provides the information required to launch an Elastic Compute Cloud (Amazon EC2) instance, which is a virtual server in the AWS Cloud. A golden AMI is an AMI that contains the latest security patches, software, configuration, and software agents that you need to install for logging, security maintenance, and performance monitoring. In this solution, we will provide users a way to deploy Golden AMI Pipeline using AWS CDK as Infrastructure as Code that will be driven by user configuration. User will be able to provide all configuration information including build, test, distribution that can be used by the AWS CDK application to create and customize the Golden AMI Pipeline. The solution will ensure that the Security best practice is also integrated for AMI Image encryption and distribution. On a high level, the image builder pipeline consists of the following -

• Recipe
  • What Base Image to Use
  • What are the Build steps
  • What are the Test and validate steps
• Infrastructure
  • What type of EC2 instance to launch, build, test
  • Which VPC, Subnet and Security Group to use while launching the instance.
• Distribution
  • Where to Distribute the AMI after creation - which account/region
  • What tag will be added in the AMI in the target account

• As part of the security best practice, there will be one Customer Managed Key (CMK) created per pipeline and the underlying Amazon Elastic Block Store (Amazon EBS) volume of AMI will be encrypted with the same. This can be turned on/off with parameters which is described later.

• Base image can refer to AWS managed public Parameter Store, a capability of AWS Systems Manager (for example - /aws/service/eks/optimized-ami/1.14/amazon-linux-2/recommended) that holds the latest Amazon Linux 2 AMI or latest Amazon Elastic Kubernetes Service (Amazon EKS) optimized AMI or it can refer any Base AMI ID (ami-0123456789) that is available in the region where the service is being deployed

• User can bring their own Build and Test steps ( in yaml file) or AWS managed pre-build Systems Manager documentation can also be used.

• Image Pipeline will send Amazon SNS (Amazon Simple Notification) notification for success or failure.

• AMI Pipeline creation is configuration driven. AWS CDK application will read the user provided configuration and provision the pipeline.

• An EC2 Image Builder recipe defines the base image to use as your starting point to create a new image, along with the set of components that you add to customize your image and verify that everything is working as expected. A maximum of 20 components, which include build and test, can be applied to a recipe. After you create an image recipe you cannot modify or replace the recipe. To update components after a recipe is created, you must create a new recipe or recipe version. Pease check the parameter details section here on how to provide the recipe version. Any changes to base image, components ( ad/delete/update/re-order) requires a new recipe or recipe version. More information on recipe can be found here
• Updating to any existing component requires a new version to be created. Pease check the parameter details section here on how to provide the component version

• Ensure you have a Git client installed following these instruction
• Set up AWS Credentials in your environment using these instruction
• Ensure you have Node installed
• An active AWS account
• A web browser that is supported for use with the AWS Management Console. (See the list of supported browsers)
• AWS CDK (version2 ) CLI. Refer link on how to install

To configure cross-account distribution permissions in AWS Identity and Access Management (IAM), follow these steps:

1. To use Image Builder AMIs that are distributed across accounts, the destination account owner must create a new IAM role in their account called EC2ImageBuilderDistributionCrossAccountRole.

2. They must attach the Ec2ImageBuilderCrossAccountDistributionAccess policy to the role to enable cross-account distribution.

3. Verify that the source account ID is added to the trust policy attached to the IAM role of the destination account. Example -

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::<SOURCE_AWS_ACCOUNT_ID>:root"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

4. If the AMI you distribute is encrypted, the destination account owner must add the following inline policy to the EC2ImageBuilderDistributionCrossAccountRole in their account so that they can use your KMS keys. The Principal section contains their account number. This enables Image Builder to act on their behalf when it uses AWS KMS to encrypt and decrypt the AMI with the appropriate keys for each Region.

📌 set iamEncryption parameter in bin/config.json file to enable/disable encryption. More information can be found here

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "kms:CreateGrant",
                "kms:Decrypt",
                "kms:DescribeKey",
                "kms:Encrypt",
                "kms:Generate*",
                "kms:ListGrants",
                "kms:ReEncrypt*"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": "ec2:CreateTags",
            "Resource": "arn:aws:ec2:*::snapshot/*",
            "Effect": "Allow"
        }
    ]
}

For more information on setting up cross-account AMI distribution, visit Page

1. Clone the Repo and navigate to the folder

   git clone https://github.com/aws-samples/aws-cdk-golden-ami-pipeline.git

   cd cdk-golden-ami-pipeline

2. Provide the props ami_config (which is type of MainConfig) inside the class name createImageBuilder located in the file bin/cdk.ts For sample props, refer to example_props folder and the parameter details are described here.

3. set the region where the stack will be deployed. For example

   export CDK_DEPLOY_REGION=us-west-2

4. Install required packages

   npm install

5. Deploy the CDK application

   cdk deploy

Once the CDK application is deployed successfully , navigate to Image Builder Service to verify and check all the following resources created

Recipe Components Infrastructure Distribution Image Pipelines

After you deploy the solution, all you have to do is to navigate to Image Builder Service Console, select the Image Pipeline and start the pipeline by clicking ‘Run Pipeline’ button in the upper right corner and your golden AMI is ready for distribution.

Once the status of the Pipeline execution status is available, click version link to get all the AMI ids ( along with the distributed AMI is different region/account)

In less than few minutes, you've created and deployed a new golden image pipeline that you can continue to iterate on in the future and leverage for additional image pipelines as your business grows

If you setup distribution(Settings in the bin/config.json file), the image should be available in the target account/region. Please check EC2 AMI section from AWS Console.

MainConfig interface contains the following parameters ("?" represents optional parameters)-

baseImage: ec2.IMachineImage
amiComponentBucketName?: s3.IBucket;
amiComponentBucketCreate?: boolean;
amiComponentBucketVersion?: boolean;
imagePipelineName?: string;
instanceProfileName?: string;
instanceProfileRoleName?: string;
iamEncryption?: boolean;
componentsPrefix: string;
keyAlias?: string;
imageRecipe: Recipe;
snsTopic?: sns.ITopic
attribute?: string;
amitag?: Tags;
tag?: Tags;
schedule?: object;
infrastructure?: Infrastructure;
componentConfig: ComponentConfig;
distributionConfig?: Distribution[];
distributionName?: string;
distributionDescription?: string;
resourceRemovalPolicy?: cdk.RemovalPolicy;
defaultComponentConfig?: ComponentConfig

{
  region: string;
  accounts: string[];
}

[
    {
        "region": "us-east-1",
        "accounts": [
            "111122223333",
            "444455556666"
        ]
    }
]

{
        name?: string;
        instanceType?: ec2.InstanceType[];
        subnetId?: ec2.ISubnet;
        securityGroups?: ec2.ISecurityGroup[];
}

{
	  "name": "golden-ami-infra-blog-demo",
	  "instanceType": [
		ec2.InstanceType.of(ec2.InstanceClass.T2, ec2.InstanceSize.LARGE),
		ec2.InstanceType.of(ec2.InstanceClass.T2, ec2.InstanceSize.SMALL)
	  ],
	  "subnetId": ec2.Subnet.fromSubnetId(this,'subnet-name', "<EXISTING_SUBNET_ID>"),
	  "securityGroups": [
		ec2.SecurityGroup.fromSecurityGroupId(this,'sg-name', "<EXISTING_SECURITY_GROUP_ID>")
	  ]
}

{
    Build?: {
      name?: string;
      file?: string;
      version?: string;
      arn?: string;
      parameter?: { name: string; value: string[]
        }[];
    }[];
    Test?: {
      name?: string;
      file?: string;
      version?: string;
      arn?: string;
      parameter?: { name: string; value: string[]
        }[];
    }[];
}

{
    "Build": [
        {
            "name": "build1",
            "file": "components/build1.yaml",
            "version": "1.0.1",
            "parameter": [
                {
                    "name": "testparam",
                    "value": [
                        "samplevalue"
                    ]
                }
            ]
        },
        {
            "arn": "arn:aws:imagebuilder:us-east-1:aws:component/update-linux/1.0.2/1"
        },
        {
            "name": "build2",
            "file": "components/build2.yaml",
            "version": "1.0.1"
        }
    ],
    "Test": [
        {
            "name": "test2",
            "file": "components/test1.yaml",
            "version": "1.0.1"
        },
        {
            "arn": "arn:aws:imagebuilder:us-east-1:aws:component/reboot-test-linux/1.0.0/1"
        }
    ]
}

⚠️ componentConfig contains one or more Build/Test components. Each Build or Test may contain some parameter as given below - Any changes in Component content , requires a new version to be created. All the Components immutable with a specific version. If you change the content of any component , update the version as well. Otherwise, component creation will fail.

	"defaultComponentConfig" : {
		"Build": [
		  {
			"arn": "arn:aws:imagebuilder:us-east-1:aws:component/update-linux/1.0.2/1"
		  }
		],
		"Test": [
		  {
			"arn": "arn:aws:imagebuilder:us-east-1:aws:component/reboot-test-linux/1.0.0/1"
		  }
		]
	  }

{
    imageRecipeVersion: string; 
    imageRecipeName?: string ; 
    volumeSize?: number;
    volumeType?: string;
    deleteOnTermination?: boolean
}

⚠️ Image Recipe is immutable with a specific version. Recipe contains all the components with specific version in a specific order. If the component version changes, or new components added, or components order has been modified, please make sure to update the recipe version.

{
	"imageRecipeVersion": "3.1.6",
	"imageRecipeName": "golden-ami-recipe-demo",
	"volumeSize": 3072,
	"deleteOnTermination": false,
	"volumeType": "gp2"
}

{
	"env": "dev",
	"Name": "golden-ami-dev-{{imagebuilder:buildDate}}",
	"Date_Created": "{{imagebuilder:buildDate}}",
}

{
	"env": "dev",
	"Name": "golden-ami-demo-dev"
}

[
    {
        "name": "param1",
        "value": [
            "value1"
        ]
    },
    {
        "name": "param2",
        "value": [
            "value2"
        ]
    }
]

• set the region where the stack is deployed. For example

  export CDK_DEPLOY_REGION=us-west-2

• Destroy the CDK application

  cdk destroy

In today’s world, organizations need to be agile and be able to scale their applications based on the demand. Applications need operating systems to run and operating systems need to have tools and third party apps installed on them and properly patched on a regular basis. Maintaining and managing organization-wide operating systems at scale can be a challenging task for the operation teams. Manually setting up servers just doesn’t scale. With this solution, customers can build, test, and deploy custom-built, secure AMIs for their organization and distribute them in a multi-account environment.

• Sandip Gangapadhyay
• Siamak Heshmati
• Viyoma Sachdeva