Roham

Roham GitHub Repo: https://github.com/Haufe-Lexware/Roham

What is Roham?

Roham saves you cost on AWS by stopping/terminating/starting Instances on schedules defined by you. Roham gives you these benefits:

  • It helps you to stop/start/terminate EC2 Instances based on schedule tags
  • Schedule tags follow cron expressions in Linux which are simple to learn
  • It can be implemented across different AWS accounts to be manage centrally
  • Roham can create tags on your EC2 Instances automatically in case people forget to
  • It is a free open-source project and costs you almost nothing on AWS
  • It is simple to implement by just following the implementation guide

The word ‘Roham’ refers to a well-known hero in Persian legends. It also literally means ‘undefeatable’.

Architecture and Components

Roham consists of the following Lambda functions which are written in Python:

  • Roham Tagger: It automatically tags EC2 Instances to stop/terminate on a schedule in case they are not properly tagged.
  • Roham Terminator: It terminates EC2 Instances which are tagged to terminate.
  • Roham Stopper: It stops (shuts down) EC2 instances which are tagged to stop at a specified time.
  • Roham Starter: It starts EC2 instances which are tagged to start at a specified time.

These four Lambda functions above are independent of one another and could be independently enabled/disabled, even though it is best to enable all of them together to get the best cost-saving results.

Our recommendation is to create a central Shared Services AWS Account and create/import these Lambda functions there and then control tagging/termination/stop/start of your EC2 Instances in all your other AWS project Accounts. The diagram below shows the big picture:

The diagram and the steps below show how Roham (in this case Roham Stopper) works across AWS Accounts and takes actions on EC2 Instances:

  1. CloudWatch Event triggers the SNS Topic and passes the IAM Role ARN to it.
  2. SNS Topic publishes a message to Roham Stopper Lambda function (in the Shared Services AWS Account) and passes the IAM Role ARN as a message to it.
  3. Roham Stopper Lambda function assumes the given IAM Role (in the Project1 Account)
  4. Request for Role assumption is accepted and Rohan Stopper is allowed to take actions on EC2 Instances.
  5. Roham Stopper reads and interprets the tags on the Instances and takes actions accordingly.

The same concept in the steps above applies to the other three Lambda functions. This means every project AWS Account will have a separate CloudWatch Rule and a separate SNS Topic for each Lambda function. Please see the diagram below:

 

Tagging Scheme

Once you follow the steps in the implementation guide and everything is done, now it is time to tag your EC2 Instances. Please follow the tagging scheme in the table below:

Tag KeyTag ValueDefinition
toterminateyesThe EC2 Instance will be terminated when the respective CloudWatch Event is triggered
toterminatenoThe EC2 Instance will never be terminated
tostopnoThe EC2 Instance will never be stopped
tostopcron expressionThe EC2 Instance will stop at the specified time represented by the cron expression
weekendstopyesThe EC2 Instance will stop on Saturday at 00:00
weekendstopnoThe EC2 Instance will not stop on the weekend
tostartcron expressionThe EC2 Instance will start at the specified time represented by the cron expression

Cron Expression

Cron expression is a string of characters representing a schedule. Please take note there are different definitions of cron expressions in different libraries which means a cron expression which is valid on a system may not be identified as valid by Roham. Roham only identifies the cron expressions which are defined by the Croniter Python library. The table below briefly explains how you need to format your cron expression to represent a valid schedule:

 MinuteHourDay of Month MonthDay of Week
Possible Values0-590-231-311-12 OR jan-dec1-7 OR mon-sun

Other possible characters

CharacterDefinition
Comma (,)Commas are used to separate items of a list. For example, using "mon,wed,fri" in the 5th field (day of week) means Mondays, Wednesdays and Fridays.
Hyphen (-)Hyphens define ranges. For example, mon-fri means from Monday to Friday
Asterisk (*)Asterisks are used to select all values within a field. For example, * in the Month field means “every month”

Now let’s see some example expressions and their definitions:

ExpressionMinuteHourDay of MonthMonthDay of WeekDefinition
0 18 * * mon-fri018**mon-friEvery day from Monday to Friday at 18:00
30 7 20 2 *307202*On February 20th at 7:30 AM
30 23 * * 53023**5On this coming Friday at 23:30

Please take note the Python code in Roham evaluates these cron expressions against the GMT time. The use of slashes (/) is also prohibited in your cron expression.

It is also important to know in case your cron expression is invalid, the cron tagger will override it with a default value.

 

Implementation

This part will walk you through all the steps to implement Roham in your AWS environment. I have tried to cover every small step to achieve a full working setup. If you have any questions, simply send me an email on e.sarabadani@gmail.com and ask, I try my best to respond in a timely manner. However my biggest advice is to make sure you read this documentation thoroughly first before asking your question, becasue there are many points which you might have already missed.

The goal of this guide is to have the diagram below implemented for every single component of Roham (Tagger, Stopper, Starter, Terminator):

 

So as shown in the diagram above, Roham will be implemented on a central Shared Services AWS Account and manages EC2 Instances in other AWS Accounts. Roham has two different categories for the other AWS Accounts which it manages:

AWS AccountPurpose
DevelopmentThis is your project development environment. This is where you need to manage your cost. This could also be a production environment. EC2 Instances here are never automatically tagged to Terminate by Roham Tagger.
PlaygroundThis is an AWS Account where developers play around and learn. In many companies the costs in such Accounts increase really fast. EC2 Instances here are automatically tagged to Terminate by Roham Tagger unless they are manually tagged not to.

Steps Overview

Assuming that you have already a Shared Services AWS Account as well as a Development/Playground Account ready, follow this guide and take the following steps below:

Phase 1 – On the Shared Resources AWS Account:

  1. Create an IAM Role and Policy which assigns Lambda the permission to execute commands
  2. Create a separate Lambda function for Roham Tagger, Stopper, Starter, and Terminator respectively

Phase 2 – On the Development/Playground AWS Account:

  1. Create an IAM Role and Policy to allow cross-account access to EC2 Instances
  2. Create a separate SNS Topic for the Roham Tagger, Stopper, Starter, and Terminator Lambda functions respectively
  3. Assign the SNS Topic permission to allow the Shared Services AWS Account to use it
  4. Create a separate CloudWatch Rule for each SNS Topic created in the step above and make sure it publishes a message to its respective SNS Topic on a regular interval.

Phase 3 – On the Shared Resources AWS Account:

  1. Subscribe the Lambda functions to their respective SNS Topic in the Development/Playground AWS Account
  2. Allow the created Lambda functions to be executed by the SNS Topic in the other Development/Playground AWS Accounts

Make sure you have AWS CLI on your PC/Mac. In this example the Shared Services AWS Account ID is 111111111111 and the Project1 AWS Account ID is 222222222222.

Phase 1 – On the Shared Resources AWS Account

Create IAM Role and Policy

Create a custom IAM Policy with the following JSON content and name it Roham_Policy_Master:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "*",
            "Resource": "*"
        }
    ]
}

Create a custom IAM Role with the following JSON content and name it Roham_Role_Master:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Create a separate Lambda function for each Roham component

Download the function zip files from here You will need to repeat the steps below one at a time for each of these four packages. let’s say first we do it for Roham Tagger. Run the following AWS CLI command to create the Lambda function:

aws lambda create-function --function-name Roham_Tagger --runtime python3.6 --role arn:aws:iam::111111111111:role/Roham_Role_Master --handler lambda_function.lambda_handler --timeout 300 --zip-file fileb://"C:\Users\esmaeil\Desktop\Roham\Functions\Tagger\Roham_Tagger_Lambda.zip" --profile shared --region eu-central-1

Phase 2 – On the Development/Playground AWS Account

Create IAM Role and Policy

Create a custom IAM Policy with the following JSON content and name it Roham_Policy_Managed:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "sns:*",
                "ec2:*"
            ],
            "Resource": "*"
        }
    ]
}

Create a custom IAM Role with the following JSON content and name it Roham_Role_Managed:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::111111111111:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {}
    }
  ]
}

Create a separate SNS Topic for each Roham component

You will need to run the command below three more times for the other three components of Roham (Stopper, Starter, Terminator)

aws sns create-topic --name Roham_Tagger --region eu-central-1

 

Assign the SNS Topic permission

aws sns add-permission --region eu-central-1 --topic-arn arn:aws:sns:eu-central-1:222222222222:test --label lambda-access --aws-account-id 111111111111 --action-name Subscribe ListSubscriptionsByTopic Receive

Create a separate CloudWatch Rule for each SNS Topic

So here we create the CloudWatch Rule which:

  • Publishes an SNS Message to the SNS Topic subscribers
  • Lambda received the message and is therefore executed
  • The message contains the IAM Role ARN (Roham_Rold_Managed) and the Account type (playground/dev)
  • Roham Lambda functions assume this Role and are therefore able to manage EC2 Instances in that AWS Account
aws events put-rule --name Roham_Tagger --schedule-expression "cron(30 * * * ? *)" --state DISABLED --region eu-central-1

Please take note the cron expression above follows the AWS cron format and has nothing to do with the way we use cron expressions to schedule Roham.

What should be your AWS CloudWatch Rule Schedule cron expression?

It is a very important question and might be a bit confusing at first. The CloudWatch Rule Schedule cron expression is completely different from Roham cron expression tags you assign to your EC2 Instances. To understand CloudWatch Schedule cron expressions, please visit this page which follows a completely different format.

We sugest the following values for the CloudWatch Rule Schedules:

CloudWatch RuleCron ExpressionDefinitionReason
Roham_Starter30 * * * ? *Every hour of every day on the minute 30It is best to check the "tostart" tags on EC2 Instances every hour. It is also suggested to configure it on the minute 30 in case of any delay in Lambda function execution.
Roham_Stopper30 * * * ? *Every hour of every day on the minute 30It is best to check the "tostop" tags on EC2 Instances every hour. It is also suggested to configure it on the minute 30 in case of any delay in Lambda function execution.
Roham_Tagger0 0/6 * * ? *Every day once at 16:30It is best to check tagging compliance only few times per day.
Roham_Terminator15 0 ? * SAT-SUN *Every Saturday and Sunday at 00:15Roham Terminator does NOT use cron expression tags to terminates EC2 Instances. As long as the "toterminate" tag is set to yes, the Instance becomes eligible for termination. So as soon as the CloudWatch Event is executed, all Instances with this tag get terminated. So the best time is at the beginning of every Saturday or Sunday.

 

Now we need to add a target (SNS Topic) to our CloudWatch Rule. There is no simple way to do it via the AWS CLI. So I show you how to do this single step using the AWS Console. Open the Console -> Oen the CloudWatch Rules -> Select the your Rule -> Click Actions -> Click Edit:

Click Add Target. Select and add your SNS Topic and click Configure input -> select Constant and add the following JSON string:

{ "rolearn": "arn:aws:iam::222222222222:role/Roham_Role", "account_type": "dev" }

Click Configure details and then click Update rule.

 

Phase 3 – On the Shared Resources AWS Account:

Subscribe the Lambda functions to their respective SNS Topic

Now we need to make sure each of the four Lambda functions is subscribed to their respective SNS Topic. You need to run this AWS CLI command:

aws sns subscribe --topic-arn arn:aws:sns:eu-central-1:222222222222:Roham_Tagger --protocol lambda --notification-endpoint arn:aws:lambda:eu-central-1:111111111111:function:Roham_Tagger --region eu-central-1

Allow the created Lambda functions to be executed by the SNS Topic

Now we need to assign the Lambda function permission to be executed by the SNS Topic in the other AWS Account:

aws lambda add-permission --function-name Roham_Tagger --statement-id rohamtagger-1 --action "lambda:InvokeFunction" --principal sns.amazonaws.com --source-arn arn:aws:sns:eu-central-1:265722387297:Roham_Tagger --region eu-central-1

If you reached up to here successfully, it means you are done with implementing Roham and now you can sit back and enjoy saving costs…