Category: AWS

AWS Cloud services and usecases

Running Shell script using SSM and API Gateway AWS

Objective :

We are required to run a shell script on ec2 using SSM and API Gateway on AWS Stack.

For the background, it was required to do the automation, where we wanted to run a shell script that can take minutes to complete.

We want to invoke that sell script through a front-end button so that we can pass some of the parameters and wanted to have the output and status.

In AWS stack, we required the following service to achieve the same

  • AWS RUN COMMAND: That will initiate the run document, which will contain shell script detail
  • AWS LAMBDA Function: To have logic that will be executing on behalf of REST API
  • AWS API GATEWAY: To create REST API
  • AWS SNS: that will be setup with run command so that notifications is available on success and failure of command for further action
  • AWS S3: to store the logs of the shell script and apart from that we can setup events also
  • AWS Dynamo DB Table: making a global database for status.
  • AWS ec2, which is having the shell script

Apart from these services, we also need to setup Some of the IAM role and policies so that these services can interact with each other.

let’s assume we have an ec2 instance that has a bash shell script. We will start our process by ensuring that the run command can be executed on that instance. So before doing automation, we will invoke manually.

AWS RUN Command :

  • AWS System Manager
  • Click on Run Command [ Used to execute commands remotely, the only pre-requisite is your machine should be able to connect with system manager]. Although some of the images already have SSM installed and running, you can check the status by running
sudo systemctl status amazon-ssm-agent

on AMI 1 and AMI2, for other OS, please visit, https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-status-and-restart.html

  • Search for the document AWS -RunShellScript. Either you can make a copy of this document or you can use this directly. This will require 3 inputs in which is mandatory, which is the command itself.
  • Rest two parameters are optional [working directory & Execution Timeout]
  • Next is, you need to choose ec2 instance
  • Next, you can choose an S3 bucket to store the logs, [whatever is echoed or output of sell script, will be available under logs]
  • and You can choose SNS for the notification with default settings [Notifcation executed on basis of status change (in progress, success, failure)]

If your shell script exit with other than 0 exit code, it is assumed as failed. For more about exit code : https://www.shellscript.sh/exitcodes.html

AWS Lambda :

  • Here you will be writing up the code , so that you can invoke that run command through code

I am using boto3 phyton 3.9 lambda env. I am also using dynamo db to store the status and command id for another purpose.

IAM Role required for this,

  1. one role is required to run lambda
  2. another role is required to pass in lambda so that the AWS run command can execute SNS
  1. Lambda Role : this role should be having permission for following
  • Dynamo DB access
  • SSM access
  • Lambda Access
  • A custom policy to perform “iam:PassrRole” on 2. role arn
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:GetRole",
                "iam:PassRole"
            ],
            "Resource": "< change this to 2nd role arn>" 
        }
    ]
}

2. SNS ARN Role : This Role should have permission to publish messages on the SNS topic.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sns:Publish"
            ],
            "Resource": "<cahnge this to sns topic arn>"
        }
    ]
}

AWS API GATEWAY :

So far, lambda should be executing the run command, where notifications are published to SNS, status is written ton dynamo DB, and logs are written to S3.

Now the last thing is, you want to put the execution of lambda behind a REST API call.

You can use the AWS API GATEWAY for this.

How to restore the ec2 root volume from a snapshot

Restoring ec2 root volume from a snapshot is just a two-step process.

Pre-requisite :

  • Make sure you have a snapshot of the attached root volume.

Steps-to-do :-

  • Click on the instance in aws console > Under Action menu > Monitor and Troubleshoot > Replace root volume
  • select the snapshot to restore, and a task will be created
  • Task will be completed within a minute, check the task status from storage tabs.