Back in October 2009 we decided it was time to add some more capacity to our fleet of servers. Two Core 2 Quad servers were acquired and fitted with 8 GB of RAM. We immediately put the first one into use after giving it two 500 GB hard drives in RAID-1. We decided we’d give virtualisation a shot. Virtualised software licenses are cheaper (i.e. $15/month for cPanel vs. $38) and idle resources on one ’server’ could be allocated to another that needed them more. Made sense to us!

VMware ESXi was the virtualisation platform that we decided to run with. Got some VMs running on it and it seemed fine. ‘Barney’ was born. Named after the character in How I Met Your Mother, we had some hope invested in it/him. It was going to be more reliable than Mulder – which ironically became good after it wasn’t our sole shared hosting server – and potentially faster. I got some customers to volunteer to test out the new server and it seemed fine.

Further down the track Barney becomes the worst server on our network. No downtime or anything on it, the speed of it was just a killer. You’d try to open a WHM session, type in your password, then you’d have a white screen open for about a minute or two before anything really happened. While waiting for it to start working properly, the load would skyrocket to about 15 to 20 from 0. Once WHM had loaded for that first time the pages would be loading fine and everything would perform nicely. It’d just be that first time to load every separate session. Seemed as though the hard disk was spinning down, even though it wasn’t.

Naturally the most logical thing to do is to work out why it’s going slow. Log into an SSH session, make sure it’s all working, then load up WHM and wait for the load to spike. Yep, I did exactly that. SSH was always fast, WHM would always be initially sluggish. I’d have a top session running and nothing would seem odd. No resources sitting at 100% CPU usage. Plenty of RAM free, with a reasonable amount cached. Plenty of disk space. No other VMs on the same node hogging resources.

Ranjit was set up on the same node as a DNS server and was eventually going to be used as our primary cPanel DNS server for customers. It had the same problems as Barney. We were never able to work out the cause of this problem. Disappointing, I know. cPanel/WHM just weren’t liked. I had a Webmin server going on there for some of my own testing stuff and it seemed to work fine overall – a few patches of slowness here and there, but nothing like encountered with Barney & Ranjit.

At the start of this month we provisioned a new server called Dugong. This was to coincide with the change of ownership, going from Curtis to Michael. Michael’s a big fan of Dugongs for some reason. If you’ve seen him on Twitter you’d think he was an absolute lunatic. Some of his tweets (prior to the acquisition) looked like this:

Just you wait; Dugong Bank, Dugong Cars, Dugonalds.

@chombo I demand you rename to Dugong Hosting.

I wanted to choose some sort of awesome name for the new server, but apparently I didn’t have a choice. Dugong has so far been an awesome server. Our best yet, in fact. Incredibly fast. What would surprise you is that it runs the exact same hardware as Barney did – only there’s no virtualisation involved here. We gradually moved everyone off Barney and on to Dugong. So far I’ve moved half of the people who were on Mulder onto Dugong. The exceptions being resellers and dedicated IP address customers.

What’s happening with Barney now? I turned off the VM earlier this evening after the last person on it moved off. I’ll admit, I did feel a little sentimental when running the shutdown -h now command. I had rebooted it many times in the hope of new kernels fixing the slowness, but nothing ever worked. We should soon have that node back up and running soon as ‘Dolphin’. Exactly the same hardware as Dugong. Core 2 Quad, 8 GB of RAM, dual 500 GB hard drives in RAID-1. Same software will be running on it as Dugong – CentOS 5 (64-bit), cPanel, Installatron, PHP 5, MySQL 5 etc. etc. There will be a slight difference to this server in terms of software configuration. What is that difference going to be? I’ll make an announcement when it happens. I can tell you now that Dolphin will be used for reseller customers.