See also: my blog post.
I wish that AWS Lambda functions could be configured to use secrets stored in AWS Parameter Store and AWS Secrets Manager in the same way that AWS ECS task definitions can be. Specifically, I wish I could do this:
Transform:
- AWS::LanguageExtensions
- cloudenv
- AWS::Serverless-2016-10-31
Resources:
Example:
Type: AWS::Serverless::Function
Properties:
Architectures: [arm64]
Runtime: python3.9
Handler: index.handler
Environment:
Variables:
HELLO: WORLD
Secrets:
MY_SECRET: !Sub arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/my/parameter/name
MY_SECOND_SECRET: !Sub arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/my/parameter/name
THIRD_SECRET: !Sub arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/my/parameter/third
REAL_SECRET: !Sub arn:aws:secretsmanager:${AWS::Region}:${AWS::AccountId}:secret:mysecret-rlBksU
InlineCode: |
import os
def handler(event, context):
return {
'HELLO' : os.environ['HELLO'],
'MY_SECRET' : os.environ['MY_SECRET'],
'MY_SECOND_SECRET': os.environ['MY_SECOND_SECRET'],
'THIRD_SECRET' : os.environ['THIRD_SECRET'],
'REAL_SECRET' : os.environ['REAL_SECRET'],
}
The code in this repo achieves the above. That's all you need to know if you want to use it.
- dotnet6
- dotnetcore2.1
- dotnetcore3.1
- java11
- java8.al2
- nodejs10.x
- nodejs12.x
- nodejs14.x
- nodejs16.x
- python3.8
- python3.9
- ruby2.7
- provided (see below)
- provided.al2 (see below)
If you want to know how it was built, read on. It is made up of three parts:
An executable in cloudenv/cloudenv.go
. This application
is bundled into a Lambda layer that does the following:
- Is invoked like so:
cloudenv /var/lang/bin/python3.9 /var/runtime/bootstrap.py
- Looks for environment variables that look like either
MY_PASSWORD={aws-ssm}arn:aws:ssm:...
orOTHER_VAL={aws-sm}arn:aws:secretsmanager:...
. - Fetches the values for those ARNs and substitutes them into the current
environment, using
ssm:GetParameters
andsecretsmanager:GetSecretValue
. - Calls
exec()
to pass control to/var/lang/bin/python3.9 /var/runtime/bootstrap.py
- The Python code for the user's Lambda function can access the values at
os.environ.MY_PASSWORD
oros.environ.OTHER_VAL
, with no AWS SDKs required.
This means that secrets are fetched during Lambda init time, which is free on most runtimes. It also runs as parallel as possible for best performance.
The second part is a CloudFormation macro, seen on the third line of the example
YAML above. When included in a CloudFormation template, the cloudenv
macro
looks for any AWS::Serverless::Function
that has an Environment.Secrets
property (like the example function does) and:
- Moves these to the function's
Environment.Variables
section with the appropriate{aws-ssm}
or{aws-sm}
prefix expected by the Lambda layer. - Adds the (correct per CPU architecture) Lambda layer to the function's list
of
Layers
. - Adds an
AWS_LAMBDA_EXEC_WRAPPER
environment variable to intercept function cold starts with the executable in the Lambda layer. - Adds IAM policies that grant access to the specific values in Parameter Store and Secrets Manager.
The third part is a second two-line Lambda layer. It is an implementation detail
required by the fact that provided
and provided.al2
Lambda runtimes don't
support wrapper scripts. So the function's Handler
needs to
be a command that executes the actual Lambda function handler in those cases.