The evolution of Infrastructure as Code with the AWS Cloud Development Kit (CDK)

17 min readJan 16, 2024

Introduction

As the AWS Cloud Development Kit (CDK) surpasses four years since launching as generally available, I reminisce of the excitement felt seeing the next iteration of Infrastructure as Code (IaC) that would change the way we define and build resources in the cloud. On the surface, the CDK provides a way to define cloud resources in AWS using a general programming language (GPL), which on its own is incredibly powerful. By using a general programming language, IaC authors can:

Leverage advanced control flows (ie, conditional statements, for loops, etc)
Inheritance
Package reusable patterns into libraries
Author unit tests

In addition, the CDK supports multiple languages which provides choice for users to pick a language that they are more comfortable with using, which is powered by the jsii library. This provides a more intuitive experience compared to the nuances often encountered when working with a DSL or configuration languages like YAML and JSON. On its own, the CDK provides an incredibly powerful experience for IaC authors, and a lot of folks hearing about the CDK may assume that the imperative authoring experience is the main differentiator between it and other IaC tools; however, this is just the tip of the iceberg. Whether you’re a seasoned IaC developer or just starting out, the capabilities of the CDK promise change in the way we approach IaC.

The CDK represents a significant paradigm shift in Infrastructure as Code as we are starting to see the application (runtime) code and infrastructure code blend together, treating the entire cloud stack as a cohesive application. This approach eliminates the traditional need for multiple, fragmented deployment pipelines for cloud resources and application code. Tools like the CDK integrate these components into a single workflow, in turn improving the DevOps lifecycle from development, to deployment.

In this blog we’re going to explore the transformative features of the CDK introduced over the past four years and show how it can help users achieve a true DevOps workflow. Let’s dive in and start with unpacking CDK constructs.

Constructs and Abstractions

The heart of the CDK is where you’ll discover its true superpower: abstractions through API’s, which are referred to as constructs. Traditionally, building cloud resources meant defining the lower level “boilerplate” resources and stitching everything together on your own, such as Identity and Access Management (IAM) and networking. Regardless of skill level, this process is not just tedious but can be error prone. Enter CDK constructs, which can reduce the complexity and intricate glue logic required when integrating AWS services with one another.

Constructs come in three flavors, each offering a distinct level of abstraction:

Level 1 (L1): These map directly to CloudFormation resources. Using these constructs require a deeper knowledge and understanding of the resources and how to connect dependencies. Developers will often use L1 constructs when there is lack of a higher level construct available or lack of support for a particular feature within a resource.
Level 2 (L2): A step up from L1, these constructs offer abstractions through a higher level, intend-based API. They come with good practices and sane defaults built in, along with helper methods to simplify working with dependencies across resource types.
Level 3 (L3): These offer the highest level of abstraction, which encapsulate patterns by combining multiple resources for specific use cases and scenarios, such as a “Load Balanced Fargate Web Service”.

Let’s take the concepts of the leveled constructs and apply them to a real world scenario. For this fictitious use case, we want to create a Lambda Function that runs our application in Typescript. In addition, we need a DynamoDB table in which our Lambda function can read and write to. Let’s start by looking at how we can do this via CloudFormation.

CloudFormation

With CloudFormation we declaratively define our resources using YAML. This is a great approach that offers a clean interface for defining cloud resources. Here is an example template written in CloudFormation using YAML.

What we see in the above YAML is that the author is required to define all of the cloud resources end to end, including the IAM policies and roles, and attaching them to the required resources. For this example we have about 85 lines of YAML, which is not too bad; however, readability can become more challenging as more resources are added, but this can happen with any project as it grows, regardless of language. Additionally, we need to create a process to build and deploy our lambda function code, as that is not something that we can do with CloudFormation. Now let’s take a look at doing this in the CDK, starting with L1 constructs.

Level 1

CDK Code in Typescript that defines resources using the AWS CDK L1 constructs

As mentioned above, L1 constructs represent a one to one mapping of CloudFormation resources. This approach will feel similar to defining CloudFormation via YAML or JSON, with the obvious difference being that users author IaC using a general programming language. With this code we can leverage all of the native functionality of a general programming language, such as control loops, running unit tests, as well as building our own constructs for reusability. For example, if we wanted to make DynamoDB table encryption a requirement for all environments deployed named “prod”, we could easily build this logic into our IaC.

Let’s move on to L2 constructs and see the difference in user experience.

Level 2

AWS CDK App built leveraging L2 constructs

There is a noticeable difference between the L1 code vs the L2 with ~85 lines of code authoring using L1 vs ~30 lines when authoring using L2’s. Another key difference is the authoring of IAM policies. For the L1 code, we had to define our policies, roles, and attach them to the proper resources. With our L2 code, we didn’t author any IAM policies or roles and leveraged the helper methods available via the L2 constructs. To grant access to the DynamoDB table, we used the .grantRead() method which does the heavy lifting of IAM policy authoring following least privilege access. In addition, the Lambda Function construct automatically created an IAM role for the Lambda function to assume, and then automatically attached the proper policy to the role. The below diagram helps illustrate how this works:

Diagram representing the authz workflow with CDK L2 constructs

While we decided to leverage the abstractions for authoring IAM policies and roles, users can author IAM policies on their own and attach them to the resources; however, this functionality gives authors greater flexibility and lessens the burden of stitching resources together. This is just one of the many benefits that user users gain when using L2 constructs. These constructs provide sane defaults as well as a plethora of convenient helpers that obfuscate the need to define every single component of a cloud resource.

There’s another key difference here that we need to mention, which is how we package our function code and deploy it. Instead of directing the construct as to where to look for my bundled code in S3 (or ECR), I am pointing to an asset on the filesystem. I can have my code and dependencies installed on the filesystem and the construct will package it up and manage the deployment of that code to the Lambda function. This is possible thanks to the Assets construct, which we’ll dive into later in the blog. Without the assets construct, the author of the IaC needs to determine how the code and dependencies get deployed prior to deploying the IaC, which requires managing two deployment processes to achieve a single outcome. There is another option if you don’t want to think of the build process, which is to leverage a container image (ie, Dockerfile) as your asset, and the CDK will build the docker image and push it to an ECR repository. Let’s take a quick look at what the lambda code would look like in this case of using a Dockerfile as our asset.

As we can see, the L2 constructs provide a rich experience that simplifies the heavy lifting often associated when authoring IaC. L2’s are the most common constructs that are used by customers as they offer the most flexibility when building in AWS. That said, there are times when customers want to take advantage of more opinion to further alleviate the heavy lift and this is when users can look to L3 constructs.

Level 3

Let’s continue down the path of our previous example and find an L3 construct for authoring a NodeJS lambda function using the CDK. In our examples above, we were able to leverage the L2 constructs to simplify the IaC authoring experience, but still had to manage the configuration and bundling of our application code and it’s respective processes outside of our CDK app. Given that we were developing our lambda function using NodeJS, we can leverage the L3 NodeJsFunction pattern. By leveraging this L3 construct, the user benefits from:

Building/Transpiling of the application code using esbuild or Docker
Manages dependencies in a single package.json for both the CDK and application code (optional)
Organization of application code in a single location for easier management

. 
├── lib 
│ ├── cdk-demo-blog-stack.api.js # Lambda handler for API 
│ └── cdk-demo-blog-stack.ts # CDK construct with Lambda function and DDB table
├── package-lock.json # single lock file 
├── package.json # CDK and runtime dependencies defined in a single package.json 
└── tsconfig.json

Here’s what the same use case looks like with an L3 construct:

When users use L3 constructs, we can see that there is opinion baked in that is scoped to specific use cases. We commonly see L3 constructs leveraged when the use case fits within the scope and has little need for deviation outside of the defined pattern. When the use case fits into the use case of the L3 pattern, this can save users time, effort, and allows them to focus less on IaC and more on the application code. With this particular L3 construct, it will determine the location of the application code based on the id of the construct, which in this case is called api. In the example pasted above, we’re leveraging all of the defaults for our NodeJS Lambda function; however, we can also go beyond the defaults and configure the lambda function as well as build options that are available via the construct, so i’m not limited in my development as the use case grows (for more information, see the API docs). For users that prefer to manage their applications within a monorepo, this construct is designed to work there as well.

.
├── packages 
│ ├── data-processor
│ │ ├── lib 
│ │ │ ├── data-processor.api.ts 
│ │ │ ├── data-processor.backend.ts 
│ │ │ └── data-processor.ts 
│ │ ├── package.json # CDK and runtime dependencies for data-processor 
│ │ └── tsconfig.json 
│ └──frontend
│ ├── lib 
│ │ ├── frontend.api.ts
│ │ ├── frontend.web.ts
│ │ └── frontend.ts 
│ ├── package.json # CDK and runtime dependencies for frontend
│ └── tsconfig.json 
├── package-lock.json # single lock file 
├── package.json # root dependencies 
└── tsconfig.json

With the three levels of constructs, users have choice and power when building their cloud resources depending on the use case and availability of higher level constructs. This means that users aren’t relegated to a single approach and can mix and match based on their needs. Lastly, CDK authors have options on how they leverage constructs, whether it’s choosing one of the over 200 constructs available within the aws-cdk-lib library, choosing from one of the 1600 community constructs available in the Construct Hub, or crafting bespoke constructs that encompass your organizations required practices.

Now let’s dive into the artifact process and how the CDK can help alleviate the manual build/deploy scripts for applications.

Artifacts and packaging

Modern application development has seen a paradigm shift, with customers increasingly adopting containers and serverless functions to run their applications. In this landscape, container images have become the standard blueprint for defining application dependencies and requirements for containerized workloads, while AWS Lambda functions can be packaged using zip files or container images for added flexibility. A common challenge when working with application code and infrastructure code is figuring out how to deploy independently while ensuring that the cloud resource dependencies and outputs are available at application deployment time. With the CDK, users can simplify their application and IaC deployment workflows by coupling the infrastructure and runtime code closely together in the CDK application. This ensures consistency and reliability in the provisioning of the resources and application as users don’t have to manage multiple processes and workflows to deploy the various components of the application and infrastructure.

Let’s consider how teams have to approach building and deploying cloud resources alongside their business applications, we’ll use a simple diagram to illustrate the process.

Parallel build/deployment workflows for application and IaC

First, the process around how customers define and deploy their cloud resources needs to be determined. Following good practices, the infrastructure code will live in its own repository, with some form of automation to deploy it (via a pipeline for example). The same goes for the application code, which has it’s own build, test, and deployment workflow. As mentioned earlier in the blog, having to configure the build and deployment processes for both use cases is not only extra work, but it requires context switching between the two codebases when troubleshooting or reviewing inter-related components. By leveraging the internal build processes for application artifacts in the cdk, you can have your infrastructure code closely coupled with your application code, enabling faster integrations and less overhead of managing multiple deployment processes.

To leverage this functionality in your CDK app, users can simply point to the asset constructs that can build the docker image or bundle and push application artifacts to S3. In the screenshot below, we are creating a new Docker image asset by instructing the construct to look in the directory containing the Dockerfile.

CDK App demonstrating the creation of a Docker image asset

During the deployment process, the CDK will build the Docker image, push it to the assets repository in ECR (which is created during the bootstrap process when first instantiating the cdk into an AWS region/account), and setup the proper permissions via IAM to allow for the compute service to pull the image and run it. Let’s look at it visually to better understand the end to end workflow.

Diagram showcasing the deployment process of assets

We’ve touched on constructs and packaging of our application artifacts, the next step is to touch on how the CDK can help with authoring CI/CD pipelines, so let’s continue and dive into CI/CD with the AWS CDK.

Continuous Integration and Continuous Delivery

Defining the resources related to the application architecture is just one step of the delivery lifecycle. It’s recommended to follow DevOps practices by leveraging automation via CI/CD pipelines as the mechanism for deploying IaC. Of course this is the recommended practice, but it does add more work onto the plate of the IaC author as they need yet another thing to build and manage. If we take what we’ve discussed above, we can see that the CDK has absorbed a lot of the heavy lifting by helping stitch cloud resources together with the higher level constructs as well as packaging and pushing our application artifacts to their respective artifact storage services. This gives us everything we need to deploy from our local workstations, but as we mentioned above it’s good practice to leverage automation of infrastructure and application deployments. Once again, the CDK has a higher level construct library called CDK Pipelines, that alleviates the pain of having to manage and build pipelines on your own and consolidates the pipeline automation work into the CDK application.

When creating a pipeline via the CDK Pipelines construct, the CDK application will do the following:

Create a self mutating pipeline using AWS CodePipeline
Connect the pipeline to a git repository with a deployment trigger on commits to the specified branch
Create steps to build and publish the artifacts
Create stages to deploy to environments in order, parallel, or in waves

While this construct is heavily opinionated by design, users aren’t relegated to the defaults and have some flexibility in configuring the pipelines to suit their needs. The constructs provide various helper methods to enable users to add additional steps to stages, such as adding a custom shell step to run your own custom testing scripts post deployment, or add a manual approval step before the production wave kicks off. It’s also common for teams to have multiple AWS accounts, representing the environment for the application (test/staging/production). In addition to constructing the pipeline, this construct also helps simplify the cross account deployments, starting with bootstrapping the new account (with the cdk bootstrap command) and then passing in the account and region information as the env parameter when defining a stage. For more information on CDK pipelines and how to get started, check out the documentation.

Ok, so we’ve reached a point where we have determined that the CDK can deploy our cloud resources, package our application artifacts, and wrap all of that into a CI/CD pipeline. The next step to achieve an end to end DevOps workflow is to add security and compliance into our code, ensuring that we stay within the guardrails of the organization.

Security and Compliance of the IaC

When authoring IaC and managing cloud resources, users have to consider securing the code and the resources that get deployed via that code. A common compliance practice is to monitor environments post deployment, or at runtime. While this is a good and needed practice, it also introduces a friction point in the development lifecycle as users have to wait until resources are deployed to determine if their resources are compliant. Policy as code was introduced to help alleviate this by shifting the compliance further left, which results in faster feedback loops. As policy as code has increased in popularity and demand, users are still unsure as to how to best leverage it. To better integrate with policy as code tools, the CDK released functionality enabling users to natively integrate policy validation during the synthesis phase. Users can either leverage one of the plugins that exist today, such as the Cloudformation Guard Plugin, third party provider plugins such as CDK Validator for Chekov and Kics CDK Validator, or build their own. By leveraging the plugin at synthesis time, the CDK will run the policy validation using the policies defined against the synthesized cloud assembly (CloudFormation), and in turn fail if any resources don’t meet the compliance requirements. Let’s look at a quick example of what this would look like.

CDK App leveraging the policy validation at synthesis

The only change I made to my cdk app was that I injected the policyValidationBeta1 parameter with the policy validation library that I want to leverage, which in this case is the CfnGuardValidator library. Now of course I can add more inputs to the validator, such as where to look for rules locally on the filesystem, which rules to disable, as well as the ability to disable the included Control Tower rules. Next. when I run a cdk synth, the CfnGuardPolicyValidator will run policy validation against the synthesized CloudFormation template. In the example above, we are creating a DynamoDB table, but are not enabling point in time recovery for the table, which is one of the policy rules that ran against this template. Here’s what the output looks like:

Failed output based on policy as code validations from a cdk synth command

One of the more challenging aspects that comes with shifting security controls closer to the developer is ensuring that there is actionable feedback, so the developer can quickly take action and continue with their development. In this case, we were not only notified of what the problem was, but the validation library also provided a fix with documentation to review for further investigation if needed.

Following the direction of the policy validator, I added point in time recovery to my DynamoDB table and my code has passed the policy checks.

At this point we’ve walked through how the CDK is not just an IaC tool that leverages a general programming language, but it provides an end to end toolkit for IaC authors to define, build, deploy, and secure their IaC in one place. The last thing I want to touch on before we wrap up is the CDK community, and all of the really great contributions and support that has been built over the past few years.

Community

One of the most significant factors that have catalyzed the progress of the CDK is the vibrant community. Events like CDK Day serve as focal points for CDK enthusiasts to gather, share knowledge, and collaborate on pushing the technology new limits. Furthermore, the community has a great ecosystem of authors that contribute their own constructs to an extensive library known as the Construct Hub, boasting over 1,600 reusable cloud components. These constructs significantly reduce the time and complexity of setting up new cloud resources, based on common patterns that aren’t natively built into the cdk library.

The AWS CDK has catapulted in popularity which has propelled the creation of other CDK projects. This has led to the birth of tools like CDK for Terraform and CDK for Kubernetes (CDK8s), allowing developers to use the familiar programming languages to define cloud infrastructure in other ecosystems, such as Terraform and Kubernetes. By wrapping complex configurations in easy-to-use constructs, these tools have made it easier to adopt best practices and accelerate cloud modernization efforts. Whether you’re working purely in AWS or integrating multiple cloud services, the CDK and its offshoots are reshaping what’s possible in the world of Infrastructure as Code, all thanks to community contributions and collaborative events.

Wrapping up

In this comprehensive blog, we dove deep into the transformative capabilities of the AWS Cloud Development Kit and it’s role in the next generation of Infrastructure as Code. We started by understanding the core concepts of constructs, which come in three sizes — L1, L2, and L3, each offering a varying level of abstraction and control. We also looked at how the CDK simplifies the packaging and deployment of application artifacts, which in turn streamlines the DevOps workflow and consolidates IaC and application code into a single deployable unit. The CDK’s ability to integrate CI/CD pipelines and enforce security and compliance through policy as code further solidifies it’s position as a robust tool for end-to-end cloud deployments.

As you look to elevate your DevOps and IaC practices, here are some action items to get started:

Subscribe to our YouTube channel, CDK Live!: Our channel offers a dynamic blend of tutorials, deep dives, and interviews with industry experts. Whether you’re tuning in for our live sessions or jumping into one of our pre-recorded tutorials, we’ll keep you ahead of the curve with the latest in CDK capabilities and good practices.
Explore CDK Constructs: Familiarize yourself with the various levels of constructs to better understand how you can leverage each level.
Experiment with Packaging Assets: Try out the CDK’s assets and see how you can integrate your application build and deployment processes into a CDK Application.
Build a CI/CD pipeline: Setup a simple CI/CD pipeline using the CDK pipelines construct. Start small to get familiar and then build out a real world use case.
Start integrating policy as code into your IaC: Experiment with policy validation during synthesis to ensure your cloud resources meet requirements. Start with the included rules and build from there.
Stay up to date: The CDK is constantly evolving, so keep an eye out for new announcements and community contributions to the Construct hub.