Part 1: Setting up HA Storage using DRBD

by | Jun 4, 2014 | Crazy Idea, Disaster Recovery, General IT, IT, Linux, Misc, SAAS (Software as a Service), Tutorial, Virtualisation

At first I thought this was going to be a fairly easy and straight forward setup. I figured it would be install FreeNAS or Ubuntu and bam we will be away. What has ended up happening was a configuration of DRBD, Heatbeat and other HA utilities to get it up and running and keep it running reliably. This has been rewarding, but I have updated this post to reflect the changes that were needed to get it running correctly.

Recently I have been using a product called VMWare VSA (VMWare vSphere Storage Appliance). What this product does it takes the storage in individual servers and presents it as a shared storage medium which allows items such as HA and DRS to run across hosts without shared storage like a SAN.

The product works really well, I have even been able to recover a cluster that was completely back to basic parts which was a nice thing to be able to do in a stressful situation. During the troubleshooting I noticed that the appliances seem to be running CentOS. Given this I wanted to know if A. I could replicate the features for my home setup and B how it works so I can better troubleshoot issues in the future.

I started looking into products that may be able to replicate the features. I first came across CARP and HAST in FreeBSD (And in particular Nas4Free) After attempting to get this working without much success, I started to read through the documentation and it mentioned a Linux equivalent which is called DRBD. As I have a lot more Linux experience I decided that I would have a better chance and getting that running rather then trying to continue with the FreeBSD method.

With this information I tracked down another workstation for my Home ESX Setup, installed a 2TB Hard drive into the machine and started setting up Linux machines to get the storage working, here is how I did it.

Virtual NAS Machines. 

This setup will require a Virtual NAS machine on each host that allows the storage to be shared. I setup a machine on each host with the following specs

  • CPU – 1 x CPU
  • Memory -1GB RAM (This was increased from 512MB)
  • Hard Drive – OS – 10GB HDD (This is not stored on the 2TB Partition)
  • Hard Drive – Data – 1.8TB HDD (This is stored on the 2TB Partition)
  • Hard Drive – Meta Data – 1GB HDD (This is stored on the 2TB Partition)
  • Network – 1 x NIC – I would like to increase this in the future.
  • Operating System – Linux 14.04 LTS x64 Server

Now that those machines are setup. I installed Linux with the defaults and only the OpenSSH server installed.

The machines will have the following host names and IP addresses. Once setup they will share 192.168.0.210

  • Machine: vNas01
  • IP: 192.168.0.211
  • Machine: vNas02
  • IP: 192.168.0.212

On Both hosts, install the python software properties package to be able to load custom repositories. Then I load the DRBD repository to install the latest version of DRBD. (Which we do later)

sudo apt-get install python-software-properties
sudo add-apt-repository ppa:icamargo/drbd
sudo apt-get update
sudo apt-get upgrade

Then I set the static IP on both machines

sudo vi /etc/network/interfaces

##vNas01##
# The primary network interface
auto eth0
iface eth0 inet static
        address 192.168.0.211
        netmask 255.255.255.0
        network 192.168.0.0
        broadcast 192.168.0.255
        gateway 192.168.0.1

##vNas02##
# The primary network interface
auto eth0
iface eth0 inet static
        address 192.168.0.212
        netmask 255.255.255.0
        network 192.168.0.0
        broadcast 192.168.0.255
        gateway 192.168.0.1

Once that was done, I added a number on entries into the hosts file on both machines to ensure reliable communication between these two hosts.

sudo vi /etc/hosts

######
192.168.0.211   vNas01
192.168.0.212   vNas02

Installed and Configured NTP on both hosts to keep them in sync. (It will update the time each minute)

sudo apt-get install ntp

sudo crontab -e

##Added the following line to the end.
1 * * * * root ntpdate au.pool.ntp.org

Installed VMWare Tools onto both machines and set their CPU, Memory and Disk Priority to High in the ESXi Settings.

By the end of this we should have two machines that are very similar. In the next post, I will install and configure DRBD and Heatbeat to keep them active.