Supporting a large network? Less is more! Provide redundancy with Linux; Anycast DNS, Failover DHCP, Peering NTP

A Presentation by Kim Hawtin

At the end of 2005, UofA put a plan in place to upgrade its DNS, DHCP and NTP infrastructure. Hardware was spec'd and purchased and rough plans were put in place for the deployment. In January 2006 I started a contract at UofA to roll out these services.

My first month focused around research on deploying these services with maximum availablity being the driving force behind the upgrade. I was refered to a paper entitled "Three Practical Ways to Improve Your Netowrk" by Kevin Miller from the Carnegie Mello University; http://www.net.cmu.edu/pres/lisa03/ This paper, along with the slides is a great place to start understanding about Anycast and how to use it in mind to deploy redundant DNS services. We wanted to tie in the upgrades with our internal IP, DNS and DHCP management tool, this stores all the information in an SQL database and allows easy configuration deployment.

Thus began the plan for Anycast DNS, Failover DHCP & Peering NTP, running on Redhat AS 4 on our HP servers distributed across serveral campuses.

Anycast DNS;
DNS is often considered the core service on todays internet, converting names to numbers and vice-versa.
So how do we make it more reliable and keep maximum uptimes?
Deploy more DNS servers and make them local to your client base!
But surely configuration becomes a nightmare?
It sure does, so how do we solve that?
Less servers, perhaps?
Thats right less - that the client can 'see', so less is more!
Using a 'feature' of modern routing protocols, like OSPF or BGP, you can offer just two IPs to all your clients, those client can see all of your DNS servers.
Anycast routing is another name for shared Unicast[1] routing.
The client can see a handful of 'localised' servers offering DNS.
These DNS servers can be anywhere on your network, all appearing on the same anycast or unicast IP address.
OSPF and BGP can be setup to route the clients request to the nearest DNS servers.
So you can have more DNS servers to improve performance, reliability and availability.
Each DNS server can be setup to be an authoritive secondary and forward to a local caching DNS service too.
The beauty about this configuration is that every client needs only two IP entries for DNS servers on the network, no matter where they are.
These two IPs can be handed to the client via Failover DHCP service.

Failover DHCP;
To cover a half dozen campuses located around Adeliade, we wanted to have a 'localised' service for the clients.
However we wanted high avaibility too, on distributed servers.
On the new DNS server hosts we trialed the new failover functionality of ISC's DHCP service[2].
This allows two DHCP servers to share the load and update each others lease information.
We generate the configuration information from our network management tool, to population the lease, DNS, NTP and other options.
We have three pairs of servers that are localised to the clients.

Peering NTP;
NTP is probably the easiest sevice to deploy and configure for all the clients.
Large organisations traditionally have a single NTP server that all servers and clients sync their time from.
As part of the drive to build in redundancy, we wanted to take advantage of the server infrastructure that we were deploying for DNS.
Each new server became a Stratum 3 peer, with a the existing Straum 2 server as a prefered parent.
Each client is given several NTP servers according to its location via DHCP.

References;
[1] "Three Practical Ways to Improve Your Netowrk" by Kevin Miller from the Carnegie Mello University; http://www.net.cmu.edu/pres/lisa03/
[2] "Setting up your K12LTSP or LTSP load-balancing and dhcp failover"; http://www.vcsvikings.org/linux/

Direct link to video