RHEL AMI available on AWS market place have access to RHUI (Red Hat Update Infrastructure) which Red Hat maintains in each AWS region. Amazon EC2 instances running RHEL will access RHUI through the VPC Internet Gateway. This in some instance could be an issue if you don’t want your instances in private subnet to go out through the internet gateway. A solution is to host your own YUM repository on a EC2 instance in AWS. This however will not be a highly available solution so one option would be maintaining multiple EC2 instances hosted YUM repositories. This obviously adds an overhead of maintaining additional EC2 instances. A simpler solution is to host the YUM repository on a S3 bucket. Amazon S3 is designed for 99.999999999% durability and 99.99% availability of objects over a given year.
Amazon have recently released a new feature VPC endpoint. These endpoints provide a secure connection to S3 that does not require an internet gateway.EC2 instances running in private subnets of a VPC can now have controlled access to S3 buckets that are in the same region as the VPC. YUM repository hosted on S3 would be accessible through VPC endpoint providing a secure access.
Another S3 service is static website hosting, this allow web content to be presented directly from a S3 bucket. In this case the Repomd.xml file be presented as the Index document.
Here’s a step by step guide to host a YUM repository on AWS S3 bucket.
First step is to setup an Endpoint on the VPC
From the VPC Dashboard click on Endpoint from the left pane.
Next step select the subnets which require access to S3 bucket using VPC endpoint
Click on Create Endpoint to complete the wizard.
Next step is to launch an EC2 RHEL instance with an additional EBS volume of 100GB for the repository. Refer to this Amazon article for details on how to make an EBS volume available for use.
First check the name of the EBS volume.
Next create file system on the new volume.
[ec2-user ~]$ sudo mkfs -t ext4device_name
Create a directory on the root volume where the volume will be mounted, then mount the volume and update the fstab file.
[ec2-user ~]$ sudo mkdir mount_point
[ec2-user ~]$ sudo mountdevice_namemount_point
[ec2-user ~]$ sudo cp /etc/fstab /etc/fstab.orig device_name mount_point file_system_type fs_mntops fs_freq fs_passno
Run Sudo yum repolist command to review the repositories currently enabled on the instance.
[ec2-user ~]$ sudo yum repolist
Enable the RHSCL (Red Hat Software Collection) Repository
[ec2-user ~]$ sudo yum-config-manager –enable rhui-REGION-rhel-server-rhscl
Install yum-utils and createrepo packages.
[ec2-user ~]$ sudo yum install yum-utils createrepo
Now run repo sync command to download packages to /repository folder. This task will take up to an hour to complete depending on if you are updating your existing repository or creating one from scratch.
[ec2-user ~]$ sudo reposync –gpgcheck -l -p –repoid=rhui-REGION-rhel-server-releases –download_path=/repository/
Once download is complete, run createrepo command to create repomd file(xml-based rpm metadata)
[ec2-user ~]$ sudo createrepo /repository/
Now to sync this repository to a S3 Bucket, S3cmd tool is required. First download the repo file for s3tools to the yum.repos.d directory. [ec2-user ~]$ sudo wget http://s3tools.org/repo/RHEL_6/s3tools.repo
Now Install the S3cmd tool
[ec2-user ~]$ yum install s3cmd -y
Next step is to configure the S3cmd tool providing the access and secret key credentials for the S3 Bucket.
[ec2-user ~]$ sudo s3cmd –configure
Create a S3 Bucket which will be used for hosting the repository, enable it for website hosting and set the Index Document as repomd.xml. Setup permission to allow everyone to access object stored in the Bucket (GetObject).
Now run S3cmd sync command to upload the repository data to the S3 Bucket. This task again can take up to an hour to upload all objects to S3.
[ec2-user ~]$ sudo s3cmd sync /repository/ s3://yum-repo001
Lastly on the instances in private subnet, disable existing RHUI repositories and create a new repo file with http path to the S3 bucket hosting the YUM repository data.
Now run yum repolist to ensure that S3 based repository is accessible and perform a test installation (httpd) using the new repository.