AWS Cost optimization — Stop unused resources and start them when required using Jenkins, SNS and Lambda

Many times, AWS resources are under utilized and each team has it’s own way of managing this. Here we will see how we can stop an idle resource and start it back when we need it.

High level flow

To achieve this we will be using:

• Jenkins — Declarative pipeline style
• Jenkins plugin — Pipeline: AWS Steps
• AWS Simple notification service (SNS)
• AWS Lambda

AWS SNS

Create SNS topic. We need this for the next step while publishing message from Jenkins. Also this topic will be used when attaching triggers for lambda.

Jenkins pipeline

Install Jenkins plugin — Pipeline: AWS Steps.

Jenkins pipeline script

In the above script:

• aws_resource_status_toggle is the name of the AWS SNS topic
• mapEnvironmentToAWSis a groovy map datastructure, which helps to map to list of EC2 instance if required. It also has ec2InstanceIds as separate key, so that same script can be refactored to support start / stop of other AWS resource types. All this is passed as it is to the SNS and then to Lambda.
• getMessage function transforms groovy object to JSON, so that it can be processed at the other end easily
• withAWS(credentials: 'AWSCredentialsForSnsPublish') — For this, we need to add AWS Access Key and Secret to Jenkins credentials. This helps to not hard code the credentials here

AWS Lambda function

Create a lambda function from scratch ( without any template ). During it., it will automatically create and IAM role and add required access.

Add SNSas the trigger., Here select the topic that we created earlier.

• event contains the event object from SNS
• we use json.loads to consume json sent from Jenkins
• boto3 library is used here to stop and start the ec2 instance. As we see., Jenkins to SNS to Lambda is just a flow of data. The same flow can be used not only to stop & start EC2 but also to trigger any action on any AWS resource

AWS IAM role and permissions

While creating lambda, it will create an IAM role but this role may not have EC2 start, stop permissions.

Paste this JSON policy document into the policy editor.,

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:Start*",
        "ec2:Stop*"
      ],
      "Resource": "*"
    }
  ]
}

Start & stop EC2 from Jenkins is ready

Now we can test the complete flow from Jenkins till stopping or starting EC2.

Automate stop of Idle EC2 instance using AWS Alarm

AWS EC2 alarm to stop Idle EC2

• Usually CPU utilization will be used to determine idle servers. But when using high end servers., even when using the server, it may or may not be using more CPU.
• Network Packets Out is helpful, if the server is hosting a web application and not much request / response is going on. Based on the usage pattern, determine a count value to identify idle state.
• 4 consequtive period(s) of 15 minutes — By this we check for this threshold for 4 consequtive 15 minutes ( 1 hour ). This is better than doing 1 consequtive period of 1 hour because shorter the period, frequent the alarm counter increments & check happens frequently avoiding edge case conditions to get triggered too late.

References

• https://github.com/jenkinsci/pipeline-aws-plugin