Oxyfish

I manage my own co-located dedicated server, which runs this site, and about 15 others, including Shields Around the World (more on this in another post). I do this mostly because I’m a geek, but also to keep up with the times, and to have a server for trying out infrastructure technologies on my own. I have hosted with meer.net since they had a tiny back room in Mountain View in 1998. At the time, they seemed to be a couple of ex-Netscape guys who had bought some old networking equipment. During the bust, they acquired tons of great gear for pennies on the dollar, and even picked up a couple of fully built-out hosting facilities for incredibly cheap. They seem to be doing very well, and I’m extremely pleased with the service.

Recently they sent me an apologetic note explaining that they had to assign me a new IP address block, because the one I was using eventually needed to be recovered. They are promising this will never happen again, since this block is one they own. It was something of a PITA, but I got through it in a couple hours. If you aren’t interested in the tech details, skip the rest of this post.

My server runs FreeBSD, and I grew up with SunOS, so it’s pretty similar. But I’ve used enough different Linux machines, etc, that I always have to find where things are. I found the rc.conf where I set up the aliases to listen to the new IP addresses. That’s where I made my big mistake – I changed the primary IP address to the new one, but I didn’t change the default router. Oops! As I later found out, packets could get in, but they couldn’t get out.

Fortunately, meer has a nice virtual console feature. Unfortunately, my SSH key was at work, and once I retrieved it, I found I had forgotten my passphrase. Incredibly, they have the ability to update your public key through their support web site, so after generating a new key and updating it, and waiting a few minutes for the servers to sync, I was on the console. While I was on the support site, I figured out that I had to change the default router, and found a couple of other changes – the DNS hosts, and the NTP hosts – that I had to make.

Of course, when the machine came back up, it found some back blocks on one of the disks, so I had to deal with that. Then I changed the default router, rebooted, and everthing worked again. Whew! It’s a bit nerve-wracking to have your main server down for an hour while you try frantically to fix it.

After that, it was easy to change the DNS in resolv.conf, and update the hosts file. When I checked my server logs, I kept getting “ntpdate: no server suitable for synchronization found” in my logs. I checked cron with “crontab -l” but there was no ntpdate. Finally, by chance, I found an /etc/crontab, which had the offending line. I still don’t know why /etc/crontab doesn’t show up when running crontab -l as root.

The remaining tasks were a little tedious, but easy. Apache needed to be reconfigured to listen to the new IP addresses for the virtual hosts. All the DNS files needed to be updated. For this one, I was a little more conservative. I updated the DNS for oxyfish.com and one other domain, but have held off on the rest for now.

So, if you had trouble seeing the site for a few days, you know I screwed it up – DNS problems always seems to take days to straighten themselves out. But if you are reading this – you know I fixed it!

This entry was posted on Saturday, May 27th, 2006 at 9:05 pm and is filed under Technology. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.