Introducing AWS Blueprints for Crossplane

Kubernetes is gaining popularity as a control plane application programming interface (API), and coupling it with Crossplane further extends its usability. Kubernetes not only orchestrates and schedules containers, but also manages resources by extending the declarative APIs and adding a reconciliation process. The combination is appealing to both DevOps teams and application development teams because they can standardize their practices around similar patterns, particularly in combination with modern delivery technologies such as GitOps.

When GitOps and Kubernetes are coupled, Git repositories are treated as the source of truth, and Kubernetes controllers ensure the resource specifications described in Git match the actual state of deployed resources. While in a traditional Kubernetes ecosystem, resources come in the form of native Kubernetes objects, such as Pods. In a setting with support for the universal cloud infrastructure APIs, they can be extended to entities such as Amazon Elastic Compute Cloud (Amazon EC2).

AWS Controllers for Kubernetes (ACK) and Crossplane are examples of such enablers, extending the Kubernetes APIs to also understand and act on cloud infrastructure constructs. Crossplane is an open source Cloud Native Computing Foundation (CNCF) project released under the Apache 2.0 license which extends existing Kubernetes API and provides the ability to provision and manage cloud infrastructure. In addition to provisioning individual cloud resources, Crossplane offers a higher abstraction layer called Compositions. Compositions allow users to build opinionated templates for deploying cloud resources. For example, organizations may require certain tags to be present to all AWS resources or add specific encryption keys for all Amazon Simple Storage (S3) buckets. Platform teams can define these self-service API abstractions within Compositions and ensure that all the resources created through these Compositions meet the organization’s requirements.

Writing your first Composition and debugging can be intimidating. To help with this, we have open sourced AWS Blueprints for Crossplane. This new project aims to simplify and accelerate your journey to managing AWS resources with Crossplane example Compositions. Crossplane currently offers two providers to manage AWS resources. AWS provider and Terrajet AWS provider. We aim to create the example Compositions for each provider so that users can choose based on their requirements.

About the repository

To help you get started, we have made two methods available for you to create a new Amazon Elastic Kubernetes Service (Amazon EKS) cluster with Crossplane installed. These methods are available under the bootstrap directory within the repository. One uses eksctl, and the other uses the EKS Blueprints for Terraform. Getting started page provides step-by-step instructions to help you bootstrap Amazon EKS Cluster and configure Crossplane with both AWS providers.

The repository contains a directory called compositions. This is the main directory for this repository, and it contains example Compositions for the AWS provider and the new Terrajet AWS provider. You can leverage these Compositions to learn how to define application infrastructure declaratively.

Usage

The below example code represents a sample blueprint. Let’s look at deploying Amazon DynamoDB resource with example Compositions. Compositions allow you to define your own API which defines one or more resources specific to your use case. Crossplane provides CustomResourceDefinitions (CRDs) to achieve this.

$ kubectl get crd | grep apiextensions.crossplane.io 

compositeresourcedefinitions.apiextensions.crossplane.io 
compositionrevisions.apiextensions.crossplane.io 
compositions.apiextensions.crossplane.io

To define Compositions, you first define the interface using compositeresourcedefinitions.apiextensions.crossplane.io . Once a CompositeResourceDefinition (XRD) is defined, Compositions are created to provide concrete implementations for the interface. Let’s look at an example.

In the compositions/aws-provider/dynamodb directory, there is a file called definition.yaml . This XRD file defines the interface for DynamoDB table. It defines names for this Composite Resource, some useful information about this resource such as region and table name, and the schema. The schema defines types of each field, constraints, and table properties.

# cat compositions/aws-provider/dynamodb/definition.yaml
apiVersion: apiextensions.crossplane.io/v1 
kind: CompositeResourceDefinition 
metadata: 
  name: xdynamodbtables.awsblueprints.io 
spec: 
  group: awsblueprints.io 
  names: 
    kind: XDynamoDBTable 
    plural: xdynamodbtables 
  claimNames: 
    kind: DynamoDBTable 
    plural: dynamodbtables 
  connectionSecretKeys: # exposed as Kubernetes secrets 
    - region 
    - tableName 
    - tableArn 
  versions: 
    - name: v1alpha1 
      schema: ...

The other files in the directory are Composition files. They provide concrete implementation for the interface defined above. Note the spec.compositeTypeRef field. This field tells Crossplane controller which interface this Composition satisfies. In this particular Composition, we define a DynamoDB table with on-demand capacity and partition key.

apiVersion: apiextensions.crossplane.io/v1 
kind: Composition 
metadata: 
  name: dynamo-on-demand-partition.dynamodb.awsblueprints.io 
  labels: 
    awsblueprints.io/provider: aws 
    awsblueprints.io/environment: dev 
    dynamodb.awsblueprints.io/capacity: on-demand 
    dynamodb.awsblueprints.io/pkType: partition 
  spec: 
    writeConnectionSecretsToNamespace: crossplane-system 
    compositeTypeRef: 
      apiVersion: awsblueprints.io/v1alpha1 
      kind: XDynamoDBTable 
      ...

Step 1: Deploy Compositions

To use the DynamoDB table Composition, apply the directory. It contains several implementation for the interface.

$ kubectl apply -f compositions/aws-provider/dynamodb 

compositeresourcedefinition.apiextensions.crossplane.io/xdynamodbtables.awsblueprints.io           created 
composition.apiextensions.crossplane.io/dynamo-on-demand-composite.dynamodb.awsblueprints.io       created 
composition.apiextensions.crossplane.io/dynamo-on-demand-partition.dynamodb.awsblueprints.io       created 
composition.apiextensions.crossplane.io/dynamo-provisioned-composite-gsi.dynamodb.awsblueprints.io created 
composition.apiextensions.crossplane.io/dynamo-provisioned-composite.dynamodb.awsblueprints.io     created 
composition.apiextensions.crossplane.io/dynamo-provisioned-composite-lsi.dynamodb.awsblueprints.io created

Check and make sure the XRD is ready to be used. Note the established field in the output below. The field is set to True when it is ready to be used.

$ kubectl get xrd xdynamodbtables.awsblueprints.io 

NAME                             ESTABLISHED OFFERED AGE 
xdynamodbtables.awsblueprints.io True        True    67s

Step2: Deploy a DynamoDB example

To provision a resource using this Composition, take a look at the examples directory. It contains various examples of using Compositions available in this repository. Let’s create a table with on-demand capacity and partition key.

$ kubectl apply -f examples/aws-provider/composite-resources/dynamodb/dynamodb-on-demand-partition.yaml

dynamodbtable.awsblueprints.io/test-table-on-demand-partition-key created

While you are waiting for the table to be created, take a look at the yaml file.

# cat examples/aws-provider/composite-resources/dynamodb/dynamodb-on-demand-partition.yaml 
apiVersion: awsblueprints.io/v1alpha1 
kind: DynamoDBTable
metadata: 
  name: test-table-on-demand-partition-key 
  namespace: default
spec: 
  compositionSelector: 
    matchLabels:
     awsblueprints.io/provider: aws 
     awsblueprints.io/environment: dev 
     dynamodb.awsblueprints.io/capacity: on-demand 
     dynamodb.awsblueprints.io/pkType: partition
    
  writeConnectionSecretToRef: 
    name: test-table-on-demand-partition-key
  resourceConfig: 
    providerConfigName: default 
    region: us-west-2
    tags: 
      - key: env 
        value: test 
      - key: anotherKey 
        value: anotherValue
  tableIndex: 
    hashKeyName: hashKey 
    hashKeyType: S

kind and apiVersion were set according to the spec.apiGroup, spec.claimNames.kind fields in the definition.yaml file.

Notice the spec.compositionSelector.matchLabels field. This field tells Crossplane which Composition you want to use. The DynamoDB Composition directory contains multiple implementations for the interface defined in the definition.yaml file. An implementation may be for a table with composite primary key, while another implementation maybe for a table with on-demand capacity. By specifying the compositionSelector field, we are asking Crossplane to create a DynamoDB table with a specific configuration.

Check the status of table. It should become available after a few minutes.

$ kubectl get dynamodbtable.awsblueprints.io/test-table-on-demand-partition-key 

NAME                               READY CONNECTION-SECRET                  AGE 
test-table-on-demand-partition-key False test-table-on-demand-partition-key 56s

$ kubectl describe dynamodbtable.awsblueprints.io/test-table-on-demand-partition-key

... 
Events: 
  Type    Reason                     Age                  From                                                              Message 
  ----    ------                     ----                 ----                                                              ------- 
  Warning ConfigureCompositeResource 112s (x2 over 113s)  offered/compositeresourcedefinition.apiextensions.crossplane.io   cannot apply composite resource: cannot patch object: Operation cannot be fulfilled on xdynamodbtables.awsblueprints.io "test-table-on-demand-partition-key-q6nng": the object has been modified; please apply your changes to the latest version and try again 
  Normal BindCompositeResource       111s (x4 over 113s)  offered/compositeresourcedefinition.apiextensions.crossplane.io   Composite resource is not yet ready 
  Normal PropagateConnectionSecret   11s                  offered/compositeresourcedefinition.apiextensions.crossplane.io   Successfully propagated connection details from composite resource 
  Normal ConfigureCompositeResource  10s (x6 over 113s)   offered/compositeresourcedefinition.apiextensions.crossplane.io   Successfully applied composite resource 
  Normal BindCompositeResource       10s (x2 over 11s)    offered/compositeresourcedefinition.apiextensions.crossplane.io   Successfully bound composite resource


$ aws dynamodb list-tables 

  { "TableNames": [ "test-table-on-demand-partition-key-q6nng-nwzqb" ] }

Clean up

When finished with this exercise, please clean up using the commands below. These commands will remove the table resources, Compositions, XRDs, and the EKS cluster from your account.

# Remove claim: 
$ kubectl delete -f examples/aws-provider/composite-resources/dynamodb/dynamodb-on-demand-partition.yaml 

# Remove compositions and XRDs: 
$ kubectl delete -f compositions/aws-provider/dynamodb 

# Remove cluster (eksctl) 
$ eksctl delete cluster -f eksctl.yaml 

# Remove cluster (terraform) 
$ terraform destroy --auto-approve

Conclusion

This tour of the repository is just the beginning. There are more complex examples, such as nested Compositions, available for you to explore. If you would like to become more familiar with Crossplane, check out the Crossplane documentation. To learn more about the inner workings of Crossplane, check out these blog posts. The Crossplane AWS Blueprints project is evolving and we are adding more blueprints. We also want to grow our base of contributors, so if you would like more blueprints for other AWS services or if you have ideas to share, please reach out on the Crossplane AWS Blueprints GitHub. Crossplane and this repository are open source projects, and our community is growing. Join us on Slack and GitHub if you would like to contribute.