|

This is another short blog to demonstrate how to backup a Linux EC2 instance.

A couple of things to note before going through the below;

– Your EC2 instance needs to be EBS backed

– You will require a IAM user that has rights to perform snapshots of EBS volumes.

So lets get started….

Mission

– Install AWS CLI tools on Ubuntu

– Setup backup schedule for EBS snapshots for 7 day rotation

Setting Up IAM User

You will need to setup a new IAM user and make a note of their Access Key ID & Secret Access Key. They will need to have the correct permissions to access the EBS volumes. I have included a sample config below:

~~~~~~~~~~~~~~~~~~~~~~~

#{
# “Statement”: [
# {
# “Sid”: “Stmt1345661449962”,
# “Action”: [
# “ec2:CreateSnapshot”,
# “ec2:DeleteSnapshot”,
# “ec2:CreateTags”,
# “ec2:DescribeInstanceAttribute”,
# “ec2:DescribeInstanceStatus”,
# “ec2:DescribeInstances”,
# “ec2:DescribeSnapshotAttribute”,
# “ec2:DescribeSnapshots”,
# “ec2:DescribeVolumeAttribute”,
# “ec2:DescribeVolumeStatus”,
# “ec2:DescribeVolumes”,
# “ec2:ReportInstanceStatus”,
# “ec2:ResetSnapshotAttribute”
# ],
# “Effect”: “Allow”,
# “Resource”: [
# “*”
# ]
# }
# ]
#}

~~~~~~~~~~~~~~~~~~~~~~~

Installing AWS CLI Tools

So these are the tools that will let your instance communicate with the AWS services. To install them on Ubuntu, follow the below;

– Login to your Ubuntu server

> sudo su –
> apt-get remove -y python-pip

> cd /tmp
> wget https://s3.amazonaws.com/aws-cli/awscli-bundle.zip
> apt-get install unzip -y
> unzip awscli-bundle.zip
> ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
> /usr/local/bin/aws –version

Configure AWS CLI Tools

> /usr/local/bin/aws configure

AWS Access Key ID [None]: *************************************
AWS Secret Access Key [None]: *************************************
Default region name [None]: eu-west-1
Default output format [None]: text

The above information is based on Ireland configuration. You will want to enter your region above along with security keys for the user you have created to perform the snapshots.

Test Configuration

> /usr/local/bin/aws ec2 describe-regions

EXAMPLE OUTPUT

~~~~~~~~~~~~~~~~~~~~~~~

{
“Regions”: [
{
“Endpoint”: “ec2.eu-west-1.amazonaws.com”,
“RegionName”: “eu-west-1”
},
{
“Endpoint”: “ec2.sa-east-1.amazonaws.com”,
“RegionName”: “sa-east-1”
},
{
“Endpoint”: “ec2.us-east-1.amazonaws.com”,
“RegionName”: “us-east-1”
},
{
“Endpoint”: “ec2.ap-northeast-1.amazonaws.com”,
“RegionName”: “ap-northeast-1”
},
{
“Endpoint”: “ec2.us-west-2.amazonaws.com”,
“RegionName”: “us-west-2”
},
{
“Endpoint”: “ec2.us-west-1.amazonaws.com”,
“RegionName”: “us-west-1”
},
{
“Endpoint”: “ec2.ap-southeast-1.amazonaws.com”,
“RegionName”: “ap-southeast-1”
},
{
“Endpoint”: “ec2.ap-southeast-2.amazonaws.com”,
“RegionName”: “ap-southeast-2”
}
]
}

~~~~~~~~~~~~~~~~~~~~~~~

Create Backup Script

> mkdir /opt/aws/
> touch /opt/aws/ebs-snapshot.sh
> chmod +x /opt/aws/ebs-snapshot.sh

Enter the blow script into “ebs-snapshot.sh”

~~~~~~~~~~~~~~~~~~~~~~~

#!/bin/bash
# Safety feature: exit script if error is returned, or if variables not set.
# Exit if a pipeline results in an error.
set -ue
set -o pipefail

#######################################################################
#
## Automatic EBS Volume Snapshot Creation & Clean-Up Script
#
# PURPOSE: This Bash script can be used to take automatic snapshots of your Linux EC2 instance.
# – The script will then delete all associated snapshots taken by the script that are older than 7 days
#
#
# DISCLAMER: The software and service is provided by the copyright holders and contributors “as is” and any express or implied warranties,
# including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall
# the copyright owner or contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but
# not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any
# theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this
# software or service, even if advised of the possibility of such damage.
#
# NON-LEGAL MUMBO-JUMBO DISCLAIMER: Hey, this script deletes snapshots (though only the ones that it creates)!
# Make sure that you understand how the script works. No responsibility accepted in event of accidental data loss.
#
#######################################################################

## REQUIREMENTS:

## IAM USER:
#
# This script requires that a new user (e.g. ebs-snapshot) be created in the IAM section of AWS.

## AWS CLI: This script requires the AWS CLI tools to be installed.
# Read more about AWS CLI at: https://aws.amazon.com/cli/

# ASSUMPTION: these commands are ran as the root user.
## START SCRIPT

# Set Variables
instance_id=wget -q -O- http://169.254.169.254/latest/meta-data/instance-id
today=date +"%m-%d-%Y"+"%T"
logfile=”/var/log/ebs-snapshot.log”

# How many days do you wish to retain backups for? Default: 7 days
retention_days=”7″
retention_date_in_seconds=date +%s --date "$retention_days days ago"

# Start log file: today’s date
echo $today >> $logfile

# Grab all volume IDs attached to this instance, and export the IDs to a text file
aws ec2 describe-volumes –filters Name=attachment.instance-id,Values=$instance_id –query Volumes[].VolumeId –output text | tr ‘\t’ ‘\n’ > /tmp/volume_info.txt 2>&1

# Take a snapshot of all volumes attached to this instance
for volume_id in $(cat /tmp/volume_info.txt)
do
description=”$(hostname)-backup-$(date +%Y-%m-%d)”
echo “Volume ID is $volume_id” >> $logfile

# Next, we’re going to take a snapshot of the current volume, and capture the resulting snapshot ID
snapresult=$(aws ec2 create-snapshot –output=text –description $description –volume-id $volume_id –query SnapshotId)

echo “New snapshot is $snapresult” >> $logfile

# And then we’re going to add a “CreatedBy:AutomatedBackup” tag to the resulting snapshot.
# Why? Because we only want to purge snapshots taken by the script later, and not delete snapshots manually taken.
aws ec2 create-tags –resource $snapresult –tags Key=CreatedBy,Value=AutomatedBackup
done

# Get all snapshot IDs associated with each volume attached to this instance
rm /tmp/snapshot_info.txt –force
for vol_id in $(cat /tmp/volume_info.txt)
do
aws ec2 describe-snapshots –output=text –filters “Name=volume-id,Values=$vol_id” “Name=tag:CreatedBy,Values=AutomatedBackup” –query Snapshots[].SnapshotId | tr ‘\t’ ‘\n’ | sort | uniq >> /tmp/snapshot_info.txt 2>&1
done

# Purge all instance volume snapshots created by this script that are older than 7 days
for snapshot_id in $(cat /tmp/snapshot_info.txt)
do
echo “Checking $snapshot_id…”
snapshot_date=$(aws ec2 describe-snapshots –output=text –snapshot-ids $snapshot_id –query Snapshots[].StartTime | awk -F “T” ‘{printf “%s\n”, $1}’)
snapshot_date_in_seconds=date "--date=$snapshot_date" +%s

if (( $snapshot_date_in_seconds <= $retention_date_in_seconds )); then
echo “Deleting snapshot $snapshot_id …” >> $logfile
aws ec2 delete-snapshot –snapshot-id $snapshot_id
else
echo “Not deleting snapshot $snapshot_id …” >> $logfile
fi
done

# One last carriage-return in the logfile…
echo “” >> $logfile

echo “Results logged to $logfile”

~~~~~~~~~~~~~~~~~~~~~~~

Crontab Configuration

> crontab -e

Enter the following at the bottom of the crontab

~~~~~~~~~~~~~~~~~~~~~~~

00 06 * * * root /opt/aws/ebs-snapshot.sh >> /var/log/ebs-snapshot.log 2>&1

~~~~~~~~~~~~~~~~~~~~~~~

After the above has been completed, you EC2 instance will have 7 days of snapshots rotated in the AWS console.

Hope this post helps you.

Marc Esmiley – @MarcEsmiley

SystemsUp Ltd
Waterloo Business Centre
117 Waterloo Rd
London
SE1 8UL