Route53 has been around for almost 5 years and offers a managed DNS solution for connecting your resources in AWS from the outside world. It also now offers Domain name registration so you can purchase and managed your domain from Route 53. Route 53 has option for five types of routing policy which is very handy if you have an infrastructure which span across a number of geographic region or you would like to distribute load across different stack. There is so much you can do using Route 53 however this is not the main focus of this article. Last year AWS announced release of an internal DNS service Route 53 Private Hosted Zone (PHZ) which only answers queries within a VPC. There are a number of good use cases for this however it also has few limitations.
Route53 PHZ has the following limitation:-
There is no option of setting up forwarding or conditional forwarding.
Route53 private hosted zone cannot resolve any resource outside the VPC and also would not answer any queries from outside the VPC. (See this link)
Private hosted zones do not support transitive relationships outside of the VPC. (See this link)
One domain cannot be a subdomain of the other. (See this link)
All these limitations makes it really difficult to use Route53 PHZ as a complete DNS solution when you have a hybrid cloud environment or design an environment which spans across multiple VPCs and needs multiple internal domain names.
One option is to not use Route 53 PHZ at all and use EC2 based BIND (Linux) or Windows DNS server. This obviously give you more flexibility and options however would need lot of work to setup and also maintaining it further. You will have to design it to ensure that solution is highly available and also can auto-heal itself if the instance becomes unhealthy. In a typical master-slave setup for BIND this would pose a big challenge.
There is however a better way to deal with the shortcoming of Route 53 PHZ while making the overall solution more robust and easy to maintain.
Solution comes in the form of using forward-only BIND servers that don’t host any records but forward every query based on the domain being queried to either Route 53 PHZ or to an on premise DNS server. The BIND servers will be easy to manage as once you have created all the forward only zones you don’t need to make changes on a day to day basis. All records will be managed on the Route 53 PHZs or any on premise DNS server. Here is a quick look at the steps needed to configure this solution.
First create all the Route 53 zones forward and reverse, associate them to the VPC with the BIND servers.
Now create CNAME records for ELBs and EC2 instances.
If you have more than one VPC in your environment all you have to do now is to use DHCP option set on the VPCs to point to the BIND servers. This will require VPC peering be enabled and so that DNS traffic from these VPCs can reach BIND server (in a different VPC).
Now associate the DHCP option set to the required VPCs
Ensure that DNS resolution is enabled on the VPCs.
Ideally you should setup more than one BIND server, two or even three. Host them in multiple availability zones and you can survive an AZ failure. Typically this is how you should setup each component of your environment. AZ failures are rare however they do happen from time to time so make sure your environment is design to withstand it.
This is sample configuration on the BIND server.
Now what happens when your BIND server becomes unhealthy, you would probably want to replace it with a new instance with the same configuration and IP? Configuration management can be done using a number of tools – Puppet, Chef, Ansible being few of the popular ones however ensuring that the new instance gets the same fixed IP is a challenge. To get around this issue you can make use of an Elastic Network Interface (ENI). An ENI can be detached and re-attached to another instance, what makes it really useful is that fact that it retain its configuration. So to sum it up, create an ENI with required configuration (IP) and then attached it the BIND server.
You can create a network interface from EC2 console and then can assign it to an EC2 instance.
When the instance is terminated due to a health check failure event ENI is released and can be attached to a new instance. You can do all this programmatically using Ansible/Puppet or any of the other automation tool. Keep in mind that you will have to do custom routing before your instance will start responding on the newly assigned ENI.
I will cover this in detail in another blog. If you mess up the routing on an EC2 instance you will not be able to connect to it any longer and the only way to troubleshoot would be to mount the root EBS volume on another EC2 instance.