You can automate the OIDC and IAM roles creation when you provision EKS cluster by Terraform or CloudFormatio but what if you want to automate the creation of OIDC and IAM roles when creating the EKS cluster by Rancher!
What if you want to use a crossplane to create AWS resources from a new EKS cluster, how does crossplane get access to the cluster, now we are facing a chicken-egg problem.
In this article, I am going to show you how to Automate the process once the EKS cluster is created.
We need OIDC and IAM roles to use service accounts in Kubernetes to get access to AWS resources like S3, SQS,SNS ..etc
AWS is recommended to use service accounts rather than using
static ACCESS KEY and SECRET KEY inside your pods.
In the architecture diagram when a new EKS cluster is created then it will send an event to the event bridge and then we added a rule to invoke lambda only when the event name is CreateNodegroup or DeleteCluster.
You will ask why the event name is not CreateCluster because we are interested in values after the cluster is created, the creation of the cluster takes around 8 minutes.
Steps:
1- Go to lambda service -> Create function -> Add "eks-iam" in the Function name -> Author from scratch -> choose Runtime Python 3.8 -> Create a new role with basic Lambda permissions -> Create function
2- Create IAM a policy eks-iam-policy and add the below policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"iam:UntagRole",
"iam:TagRole",
"iam:CreateRole",
"iam:DeleteRole",
"iam:AttachRolePolicy",
"iam:CreateOpenIDConnectProvider",
"iam:DetachRolePolicy",
"iam:ListAttachedRolePolicies",
"eks:DescribeCluster",
"iam:UntagOpenIDConnectProvider",
"eks:ListClusters",
"iam:DeleteOpenIDConnectProvider",
"iam:TagOpenIDConnectProvider"
],
"Resource": "*"
}
]
}
3- Attach the policy to the lambda Execution role, go to lambda eks-iam function -> Configuration -> Permissions -> Execution role -> click on Role name eks-iam-role-XXXX -> attach policy eks-iam-policy
4- Add the below code to eks-iam lambda function
import json
import boto3
# in which namespace that used by the service account
namespace = 'default'
# iam policies
policies = ['arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess']
iam = boto3.client('iam')
def create_role(roleName,policy):
response = iam.create_role(
RoleName=roleName,
AssumeRolePolicyDocument=json.dumps(policy),
Description='Lambda invoked by Rancher event to Create this Role',
MaxSessionDuration=3600,
Tags=[
{
'Key': 'Created-By',
'Value': 'Lambda'
},
]
)
return response
def lambda_handler(event, context):
if event['detail']['eventName'] == "CreateNodegroup":
clusterName = event['detail']['responseElements']['nodegroup']['clusterName']
eks = boto3.client('eks')
eksResponse = eks.describe_cluster(name=clusterName)
oidc = eksResponse['cluster']['identity']['oidc']['issuer']
# create oidc provider
oidcResponse = iam.create_open_id_connect_provider(Url=oidc,
ClientIDList=['sts.amazonaws.com',],
ThumbprintList=['9e99a48a9960b14926bb7f3b02e22da2b0ab7280',],
Tags=[{ 'Key': 'Created-By', 'Value': 'Lambda' }, ])
arn = oidcResponse['OpenIDConnectProviderArn']
policy = { "Version": "2012-10-17","Statement": [ {"Sid": "", "Effect": "Allow", "Principal": { "Federated": arn}, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { arn.split('oidc-provider/')[1] +":sub": "system:serviceaccount:" + namespace + ":" + clusterName }}}]}
# create iam role
createResponse = create_role (clusterName, policy)
print(createResponse)
# attach policies to the role
for i in range(len(policies)):
iam.attach_role_policy(
RoleName=clusterName,
PolicyArn= policies[i]
)
return oidcResponse
elif event['detail']['eventName'] == "DeleteCluster":
clusterName = event['detail']['responseElements']['cluster']['name']
oidc = event['detail']['responseElements']['cluster']['identity']['oidc']['issuer']
accountId = event['detail']['userIdentity']['accountId']
arn = "arn:aws:iam::" + accountId + ":oidc-provider/" + oidc.split('https://')[1]
# remove oidc provider
oidcResponse = iam.delete_open_id_connect_provider(
OpenIDConnectProviderArn=arn
)
# get all policies attached to the role
policyResponse = iam.list_attached_role_policies(
RoleName=clusterName
)
# detach all policies from the role
for i in range(len(policyResponse['AttachedPolicies'])):
iam.detach_role_policy(
RoleName=clusterName, PolicyArn=policyResponse['AttachedPolicies'][i]['PolicyArn']
)
# remove the iam role
roleResponse = iam.delete_role(
RoleName=clusterName
)
return oidcResponse
5- Go to EventBridge -> Rules -> Create Rule -> Rule with an event pattern -> In the Event pattern, choose Custom patterns (JSON editor) and add the following:
{
"source": ["aws.eks"],
"detail-type": ["AWS API Call via CloudTrail"],
"detail": {
"eventSource": ["eks.amazonaws.com"],
"eventName": ["CreateNodegroup", "DeleteCluster"]
}
}
Choose target -> AWS service -> Select a target -> Lambda function -> Function -> eks-iam
6- Create a service account in your EKS cluster, now we can know the arn of our IAM role because you only change the EKS-CLUSTER-NAME.
Hint: I assumed you have a unique name for EKS clusters
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: list-s3
namespace: default
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::AWS-ACCOUNT-NUMBER:role/EKS-CLUSTER-NAME
7- Create a deployment and define the service account name like the example below
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: aws-cli
namespace: default
spec:
selector:
matchLabels:
app: aws-cli
replicas: 1
template:
metadata:
labels:
app: aws-cli
spec:
serviceAccountName: list-s3
containers:
- name: aws-cli
image: amazon/aws-cli
command: [ "/bin/bash", "-c", "--" ]
args: [ "while true; do sleep 30; done;" ]
now you can access your pod and test it by:
aws s3 ls
In the end, you can have a default policy that is attached for each EKS cluster and then use tools like crossplane to create specific IAM policies and roles.
Sources:
https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html
Top comments (0)