Skip to content

aws-samples/amazon-eks-scaling-with-keda-and-karpenter

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

39 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Amazon EKS scaling with KEDA and Karpenter

EKS with KEDA HPA & Karpenter cluster autoscaler

This repository contains the necessary files and instructions to deploy and configure KEDA (Kubernetes-based Event Driven Autoscaling) and Karpenter (Kubernetes Node Autoscaler) on an Amazon Elastic Kubernetes Service (EKS) cluster.

KEDA enables autoscaling of Kubernetes pods based on the number of events in event sources such as Azure Service Bus, RabbitMQ, Kafka, and more. Karpenter is a Kubernetes node autoscaler that scales the number of nodes in your cluster based on resource usage.


Sample Usecase

Full Demo is available @Youtube

Watch the video

Prerequisites

Before you begin, ensure that you have the following prerequisites:

Installation

To install KEDA and Karpenter on your AWS EKS cluster, follow these steps:

  1. Clone this repository to your local machine or download it as a ZIP file.

    git clone https://github.com/aws-samples/amazon-eks-scaling-with-keda-and-karpenter.git
  2. Navigate to the repository's directory.

    cd amazon-eks-scaling-with-keda-and-karpenter
  3. Update environmentVariables.sh file in /deployment

Variable Name Description
AWS_REGION The AWS region.
ACCOUNT_ID The AWS account ID.
TEMPOUT Temporary output file. This used to temp. store CFN for karpenter
DYNAMODB_TABLE The name of the DynamoDB table.
CLUSTER_NAME The name of the EKS cluster.
KARPENTER_VERSION The version of Karpenter.
NAMESPACE The Kubernetes namespace for KEDA.
SERVICE_ACCOUNT The Kubernetes service account for KEDA.
IAM_KEDA_ROLE The IAM role for KEDA.
IAM_KEDA_SQS_POLICY The IAM policy for KEDA to access SQS.
IAM_KEDA_DYNAMO_POLICY The IAM policy for KEDA to access DynamoDB.
SQS_QUEUE_NAME The name of the SQS queue.
SQS_QUEUE_URL The URL of the SQS queue.
SQS_TARGET_DEPLOYMENT The target deployment for KEDA to scale based on SQS messages.
SQS_TARGET_NAMESPACE The target namespace for the deployment that KEDA scales based on SQS messages.
  1. To strat deployment run

    sh ./deployment/_main.sh
  2. You will be asked to verfiy the account in context

  1. Select your deployment option

🚨🚨 Experiencing Issues 🚨🚨

If you are experiencing error while deploying Karpenter, it can be due to Karpenter repository currently supports limited version and you may be havign old version in environmentVariables.sh. Please follow steps below:

  • Please check latest karpenter version from https://karpenter.sh -> Getting Started. environmentVariables should match with supported version

  • Check file createkarpenter.sh under deployment-->karpenter line 80 - 90 and compare with karpneter getting started helm https://karpenter.sh/

Mock Testing πŸ’»

I have used iTerm2 on Mac with K9s to connect with cluster and run scripts

Architecture

  1. Once Cluster, Karpenter & KEDA is deployed

  1. Open two more terminal and connect to cluster using K9s (check Prerequisites)

  2. In first terminal with K9s, navigate to keda-test namespace pod view

  1. In seconds terminal with k9s, navigate to Nodes

  1. Open three or more terminals, copy content from deployment/environmentVariables.sh and run it on all three terminal

  1. In the terminal run below command to test Karpenter status. If you see error first fix it before moving furthe
kubectl logs -f -n karpenter -l app.kubernetes.io/name=karpenter -c controller

  1. Run keda-mock-sqs-post.py script on all four terminals

    cd app/keda
    python3 -m venv env
    source env/bin/activate
    pip install boto3
    cd  {your path}/amazon-eks-scaling-with-keda-and-karpenter
    python3 ./app/keda/keda-mock-sqs-post.py

  1. Pod sclaing using KEDA

  1. Node scaling using Karpenter

🚨 Cleanup 🚨

  1. Navigate to the repository's root directory.

    cd amazon-eks-scaling-with-keda-and-karpenter
  2. To strat cleanup

    sh ./cleanup.sh

This will delelte all the AWS services and workloads created for the solution.

Configuration

The repository contains the necessary configuration files for deploying KEDA and Karpenter. You can modify these files to suit your specific requirements. Here are some important files to note:

Feel free to modify these files according to your needs.

Acknowledgements

  • KEDA - Kubernetes-based Event Driven Autoscaling
  • Karpenter - Kubernetes Node Autoscaler

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file..