Using strong typing practices to declare a large number of resources with AWS CDK

AWS Cloud Development Kit (AWS CDK) is an open source software development framework that is used to declare Infrastructure as Code (IaC). It allows users to declare infrastructure in a general-purpose programming language and is an abstraction built on top of AWS CloudFormation. Resources declared in AWS CDK compile down to CloudFormation stacks that can be deployed using the CLI, console, or through deployment pipelines. AWS CDK’s high-level constructs make it easier to declare complex resources, while still allowing the generated CloudFormation to be inspected and manually tuned.

Using general-purpose languages allows the use of logical constructs, such as for-loops, objects, strong types, and other programming techniques, to declare infrastructure in a concise and error-free manner. This approach also makes it possible to use the IDE and related tooling to help manage the complexity around declaring a large number of resources. In this article, we’ll show how you can dynamically declare a large number of resources and still get compile time checks and auto-completion in the IDE, thereby reducing errors and improving the developer experience in large AWS CDK projects.

The examples shown here are specific to TypeScript, which is the most commonly used language for declaring AWS CDK. I used VSCode in my examples; however, this technique will work with any editor that uses the TypeScript compiler for Intellisense.

Prerequisites

This article assumes that you know how to bootstrap and create an AWS CDK project. You can follow the steps on cdkworkshop.com if you have not done this before.

This technique uses features that are present in ECMAScript 2019 and above. In most AWS CDK projects, this should not be problem if there is no application-specific requirement. To configure tsc, you need to edit the tsconfig.json file, which is generated when you run cdk init. You can use any value between ES2019, ES2020 and ESNext for target and lib values. Here’s the relevant section of my tsconfig.json file.

    "target": "ESNext",
    "module": "commonjs",
    "lib": [
      "ESNext"
    ],

Problem

In a large infrastructure deployment, often a large number of resources is consumed by other resources, which usually means passing references of the producers to the consumers. Even if they are in the same stacks, this usually means creating and naming variables for all of them. This process is repetitive at best and error prone at worst, as we make typos in the names or make changes and forget to make other dependent changes. For example, we might want to do the following:

Declare many Amazon SageMaker endpoints and have to pass the URLs to AWS Lambda functions environment variables to Elastic Compute Cloud (Amazon EC2) instances user data script.
Declare many AWS Lambda functions and want to use them as part of an AWS Step Functions state machine.
Declare many events in Amazon EventBridge and want to refer to them at various consumers.

There are many use cases, and this is a non-exhaustive list. In this post, we will use types to tell the compiler to provide autocomplete and perform checks.

You can also refer to the GitHub repo that shows a working example of this technique. The example creates multiple Lambda functions and that are used in a state machine.

Ok then, let’s solve it.

Using types

Let’s start with a problem that will illustrate what this technique solves. We want to declare two Lambda functions, which will be used in a state machine. Note that this only an example and is meant to be extended to a much larger number of resources.

Let’s start by declaring the name of the functions and three associated types:

    // names of functions. adding as const makes the list readonly and allows us to make new types from it
    const names = ["SayHello", "SayGoodbye"] as const;

    // new types create from the names of the lambda functions
    type lambdaNamesType = typeof lambdaFnNames[number];

    // create a dictionary that maps each new function type to a lambda function object
    type lambdaFnsType = { [key in lambdaNamesType]: lambda.Function };

    // create a dictionary that maps each new function type to configurations unique to that function
    type lambdaFnConfigType = { [key in lambdaFnTypes]: any };

By using these types when creating resources, we allow the IDE Intellisense to help us with suggestions and autocomplete. The lambdaFnConfigType is the most vague of all these types, and fixing it is an extension of this technique but beyond the scope of this article. However, you can look at the example code to see how to manipulate it.

Declaring resources

Now we will use the above types to declare resources. First, let’s use lambdaFnConfig; it is meant to contain configuration unique to a particular Lambda function. For example, the path of the Amazon Simple Storage Service (Amazon S3) bucket path of a function can be unique to each function.

    const lambdaFnConfig: lambdaFnConfigType = {
      SayHello: {
        path: "s3://typed-example-bucket/logic/hello.py.zip",
      },
      SayGoodbye: {
        path: "s3://typed-example-bucket/logic/goodbye.py.zip",
      },
    }

You’ll notice that the IDE shows an error if you’ve not declared SayGoodbye. By typecasting lambdaFnConfig to lambdaFnConfigType, the compiler ensures that we specify properties for all functions.

Next, let’s create some properties common to all functions.

    *const* runtime = lambda.Runtime.PYTHON_3_7;
    *const* timeout = cdk.Duration.seconds(20)

Finally, we can declare all the Lambda functions inside a loop and store them inside a dictionary/object. This is where everything comes together.

    // convert an array of key value pair to an object
    const functions = Object.fromEntries(
      // map names in each function to a different return value
      names.map((fnName: lambdaNamesTypes) => {
        // values unique to each function
        let code = lambda.S3Code.fromAsset(fnConfig[fnName].path);
        let handler = `${fnName}.lambda_handler`;
  
          // create the function
        let fn = new lambda.Function(this, fnName, {
          functionName: fnName,
          handler,
          code,
          runtime,
          timeout,
        });
  
        // return function name and function object, this will be converted to a key value pair
        return [fnName, fn];
      })
    ) as lambdaFnsType;

Let’s break this down. From bottom to top, we do the following:

Typecast the functions as lambdaFnsType, which means each of its keys are function names and its values are lambda.Function objects.
Map each value of fnName to a key value pair made of the function name and the function object.
Declare the function using properties we’ve created.
Initialize function-specific properties.
Map each function name to a return value.
Use Object.fromEntries method to convert the array of key value pairs to an object. This is also the method that is available to ES version 2019 and above.

Result

This setup allows us to utilize the powerful Intellisense to help us. Although this approach appears complex and may seem like overkill for creating two functions, it makes a difference in large projects with tens or even hundreds of functions. It is also better than the alternative dictionary-style access, such as functions[“SayGoodbye”]. Any spelling mistake in this style will lead to misconfigured stacks or stack deployment errors.

Adding a new function to this stack is also easy. Adding a new name to lambdaFnNames, like so:

    const names = ["SayHello", "AskAboutWeather", "SayGoodbye"] as const;

will automatically highlight all the places you need to make changes to get this function to work. In this example, fnConfig is highlighted because we need to mention the Amazon S3 path specific to this function.

This approach also makes it much easier to reuse resources, possibly in different stacks. Take a look at the example code and see how it creates a state machine with these Lambda functions.

Lessons learned

We’ve learned how we can group our resources and use const to create new types from their names. And, we learned how to keep their common and unique configurations separate, which makes it easy to use array maps to dynamically create objects from their configuration. We chose this setup in order to utilize the Intellisense and autocomplete features of our IDE and help reduce errors when defining infrastructure resources.