Objective :

We are required to run a shell script on ec2 using SSM and API Gateway on AWS Stack.

For the background, it was required to do the automation, where we wanted to run a shell script that can take minutes to complete.

We want to invoke that sell script through a front-end button so that we can pass some of the parameters and wanted to have the output and status.

In AWS stack, we required the following service to achieve the same

AWS Services

List of required services

• RUN COMMAND: That will initiate the run document, which will contain shell script detail
• LAMBDA Function: To have logic that will be executing on behalf of REST API
• API GATEWAY: To create REST API
• SNS: that will be setup with run command so that notifications is available on success and failure of command for further action
• S3: to store the logs of the shell script and apart from that we can setup events also
• Dynamo DB Table: making a global database for status.
• EC2, which is having the shell script

Apart from these services, we also need to setup Some of the IAM role and policies so that these services can interact with each other.

let’s assume we have an ec2 instance that has a bash shell script. We will start our process by ensuring that the run command can be executed on that instance. So before doing automation, we will invoke manually.

RUN Command

• AWS System Manager
• Click on Run Command [ Used to execute commands remotely, the only pre-requisite is your machine should be able to connect with system manager]. Although some of the images already have SSM installed and running, you can check the status by running

sudo systemctl status amazon-ssm-agent

on AMI 1 and AMI2, for other OS, please visit, https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-status-and-restart.html

• Search for the document AWS -RunShellScript. Either you can make a copy of this document or you can use this directly. This will require 3 inputs in which is mandatory, which is the command itself.
• Rest two parameters are optional [working directory & Execution Timeout]
• Next is, you need to choose ec2 instance
• Next, you can choose an S3 bucket to store the logs, [whatever is echoed or output of sell script, will be available under logs]
• and You can choose SNS for the notification with default settings [Notifcation executed on basis of status change (in progress, success, failure)]

If your shell script exit with other than 0 exit code, it is assumed as failed. For more about exit code : https://www.shellscript.sh/exitcodes.html

Lambda

• Here you will be writing up the code , so that you can invoke that run command through code

I am using boto3 phyton 3.9 lambda env. I am also using dynamo db to store the status and command id for another purpose.

IAM Role required to run Lambda and SNS

1. one role is required to run lambda
2. another role is required to pass in lambda so that the AWS run command can execute SNS

1. Lambda Role : this role should be having permission for following

• Dynamo DB access
• SSM access
• Lambda Access
• A custom policy to perform “iam:PassrRole” on 2. role arn

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:GetRole",
                "iam:PassRole"
            ],
            "Resource": "< change this to 2nd role arn>" 
        }
    ]
}

2. SNS ARN Role : This Role should have permission to publish messages on the SNS topic.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sns:Publish"
            ],
            "Resource": "<cahnge this to sns topic arn>"
        }
    ]
}

API GATEWAY

So far, lambda should be executing the run command, where notifications are published to SNS, status is written ton dynamo DB, and logs are written to S3.

Now the last thing is, you want to put the execution of lambda behind a REST API call.

You can use the AWS API GATEWAY for this.