Hands on Networking, March 2003

Synopsis:

This Hands On column explains the work necessary in migrating a small network from one ISP to another, and trying to do so with the minimum of hassle.

This article is copyright material. Please do not reproduce it without permission.

Not so long ago, moving from one ISP to another was pretty straightforward - a matter of changing a couple of settings in your Windows network options, and telling everyone your new email address. If you had a domain, you might have to update the mail forwarding options instead, but that's about it.

Now, however, in the brave new world of always on connections, things can be a little more complicated. My network, granted, is a little more complicated than some, but not as much as others. For several years, it's been connected to the internet via a 64k kilostream, at the rather unattractive price of 350 per month - and home users think ADSL is tooexpensive!

The main sticking point as far as moving to ADSL went was the lack of a service level agreement - in other words, it can break, and BT can take a few hours, a few days, or longer to fix it. And you can't complain.

So the switch to ADSL was taken with some trepidation, and then only when a friendly ISP (Wizards was able to offer exactly what I have via kilostream, except much cheaper and considerably faster - a fully routed network connection, with automatic ISDN fallback should the ADSL go walkies.

Having taken the plunge, however, the tricky question was how to manage the switchover with minimal disruption, and ensure that the people accessing services on the LAN don't even notice what's happening.

Equipment

The first step in deciding how to tackle this sort of problem is picking the equipment to use; the kilostream connection uses a Shiva Integrator 150 router, which has ISDN backup built in, but that's not suitable for ADSL.

There's a wealth of cheap ADSL kit out there, some connecting via USB, some via Ethernet, and even some PCI cards. I've ruled out USB and PCI cards, since I'd prefer a single piece of stand-alone kit, rather than a PC running routing software. Firstly, I don't have enough experience with such software to be sure an automatic backup can be made to work and secondly, I want to minimise power consumption.

With a network providing service to the rest of the world - web pages, SMTP, POP and IMAP - a firewall of some sort is necessary too. The solution in the end was to choose the Zyxel Prestige 652R11, which is an ADSL router with built-in firewall, VPN support, 10/100 ethernet, and dial backup via an auxiliary serial port. That gives you the freedom of choosing either ISDN or modem for your backup; in my case, a Zyxel Prestige 2864I that was sitting in the cupboard has been pressed back into service.

There are alternatives - you can buy a much more expensive Cisco router, for example, or a firewall appliance such as a SonicWall, but for now the Zyxel appears to be up to the job; if you don't have the most up to date firmware, however, you won't be able to get the dial backup function to work.

Planning

The most important part of moving your network is planning, especially if you have domains that need updating. Since the network has public IP addresses, each machine will need to be reconfigured. On a Windows PC, that's a trivial exercise. Similarly on the Mac OS X workstation I use day to day - just add a new connection, with a new IP address, and the Mac can see both the old and the new ranges. Which connection it uses by default is determined simply by dragging one to the top of the list.

That enabled the Mac to see and use the new connection right away, but to do everything smoothly means keeping both connections running in parallel for a while - around six weeks, in my case, though you can do it much faster if time or funds are pressing.

The trickier machines to configure are of course the Unix boxes. While many of these can be dual-homed - ie having two IP addresses - there are some instances where it just becomes too much trouble.

For example, on one system, running Netscape FastTrack web server, the IP address of each server - it was set up in the days before HTTP 1.1 - is hardcoded in many places in several configuration files, not to mention system startup files. It's a lot of editing work, and a good candidate for simply reconfiguring a more modern system from scratch.

Doing the DNS shuffle

Step one is tidying up domain names. Fortunately, the primary server for many of the domains is off the network, hosted in a Docklands data centre - but if you do have to renumber your DNS server too, you'll have proceed along these lines. In my case, it was done a couple of months before, when the server moved from one data centre to another, but the principle is the same.

Begin by shortening the life of your DNS records, both expire time and time-to-live. This ensures that instead of expiring after, say, a week, they expire after four hours (14400 seconds). In the short term, this means an increase in DNS traffic, but it also means that when you do update the DNS information with new IP addresses, they will be propagated much more quickly.

If your primary name server is being renumbered, ensure that the secondary one is up to date, and move that at a later stage. Update your domain registration records with a new IP address for the primary server, and when you have confirmation of that - in the case of Verisign-managed DNS, it took a couple of days - you can change the IP address of the system running primary. In the interim, people requesting information will be unable to retrieve it from the primary, so should automatically query the secondary, which will still be updating itself from the old address of the primary nameserver.

When the primary comes back up with its new IP address, make sure that, if it's primary for the domain that includes itself, you update that record right away. Now you can tell the secondary to collect information from the new IP address, before the old data expires. You'll now have working DNS, with the primary server on your new connection, instead of your old.

Don't forget there may be other changes necessary - in my case, the Firewall on the Zyxel router had to be configured to allow appropriate traffic to reach the servers on my LAN via the ADSL connection, and /etc/hosts.allow files on the UNIX boxes needed updating with new information too, for intra-LAN connections.

Moving mail

Stage two is to move the mail server from the old connection to the new one. There are two mail servers on my network, one as the highest level MX for all the domains I host, and one as secondary MX; other machines, which we'll come to later, are the final destinations for messages, but the main hub, holding all the aliases, was the important thing to move over, with as brief an interruption as possible took place.

Not losing mail is simple; if you have two MXs configured in your DNS for a domain, then when messages can't be delivered to the highest priority, they will go to the next highest - with the exception of a few very broken email clients, that try only the highest. But by and large, that's what happens. And if you run a mail system like Sendmail, then when a machine receives mail, it checks to see if it's an MX (Mail Exchanger) for that domain, and will happily accept it, so you don't need to fiddle with other configuration parameters; on some systems you may need to add the domains to a list of 'allowed destination' to prevent anti-spam rules rejecting mail.

I took the brute force solution to reconfiguring the mail server, a SparcStation 2 running Solaris 2.5.1; since there was a second machine on the network acting as secondary MX, I just switched it off after backing up crucial information such as mail alias lists. I then rebuilt the system using OpenBSD, chosen because it's available for Sparc hardware, and has a good security record. When it booted up, I created a sendmail configuration file, based on the old one, and assigned an IP address from the range routed to the ADSL connection.

One other trick here is that I took advantage of the change of ISP to change the main domain name I'll be using for the network. Instead of most systems being under the .org.uk domain I previously used, they'll now be divided up between .com, .org and .org.uk, depending on their primary function. So, the IP address I allocated to the primary mail server on the new connection had a different domain name, under nigelwhitfield.com.

So, all that's necessary to get mail flowing to the new mail server is to update all the domains affected, changing their primary MX records to point to the new system's name. That leaves one problem to resolve - mail that arrived while the system was being rebuilt is queued on the secondary system, destined for a host that no longer exists, on an IP address unreachable via the LAN.

No problem; in common with most UNIX implementations, OpenBSD makes it trivially easy to assign a second IP address to a system, so the command

ifconfig le0 alias 212.47.68.208 netmask 255.255.255.224

is all that's needed to give the mail server a second IP address - the same one that it had before. It's now dual-homed, ie it appears on the LAN with its old IP address as well. The secondary MX can now delivery any mail that's been queued over the LAN.

After a few days, and when everything appears stable, it's time to reconfigure the secondary MX system with its new IP address, and update the DNS records accordingly.

And finally the long slog of reconfiguring web servers: I took the decision to ditch old HTTP 1.0 based software, with its requirement of a unique IP address for each site, and switch to Apache, using the NameVirtualHost directive, and updating the DNS information so that multiple www.something.org addresses all resolve to the given IP number; you use lines like this in your httpd.conf

NameVirtualHost 212.47.68.208

<VirtualHost 212.47.68.208>
ServerName www.mydomain.com
DocumentRoot /web/sites/www.mydomain.com
</VirtualHost>

Once again, to ease the transition, I took the opportunity to move the sites onto new hardware, so that the change would appear completely seamless to all the visitors.

If you don't have that luxury, you could instead configure a new IP address on the same hardware, and force Apache to use just that one with the BindAddress directive, ensuring it doesn't try to grab the IP numbers in use elsewhere on the same system. Thankfully, the POP and IMAP servers don't require reconfiguration - changing the IP address of the system is enough to keep them happy, leaving the whole network running on the new connection. The sole remaining task is to increase the life of the DNS entries.

Moving network connections might sound a daunting prospect at first, but as you can see, all it really takes is planning.

End of article.

Click here for the NigelWhitfield.com home page.