Openfiler 2.99 Active/Passive With Corosync, Pacemaker And DRBD

Openfiler is a Linux based NAS/SAN Application which can deliver storage over nfs/smb/iscsi and ftp. It has a web interface over that you can control these services. This howto is based on the latest version of openfiler at this date, you can download it from the official homepage http://www.openfiler.com/.

Thanks to the Openfiler team that made this howto possible.

hostname: filer01eth0: 10.10.11.101eth1: 10.10.50.101500MB Meta partition4GB+ Data partitionhostname: filer02eth0: 10.10.11.102eth1: 10.10.50.102500MB Meta partition4GB+ Data partition

virtualip: 10.10.11.105 ( don't use on any adapter, we will make this later with corosync )

root@filer01 ~# nano /etc/hosts

Add:

10.10.50.102filer02root@filer01 ~# nano /etc/hosts

On filer02 add:

10.10.50.101filer01root@filer01 ~# ssh-keygen -t dsa

Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:

Do the same on filer02.

root@filer02 ~# ssh-keygen -t dsa

Then exchange the files:

root@filer01 ~# scp ~/.ssh/id_dsa.pub root@filer02:~/.ssh/authorized_keys

root@filer02 ~# scp ~/.ssh/id_dsa.pub root@filer01:~/.ssh/authorized_keys

And now you can exchange files between the nodes without entering a password.

Before we can actually start the cluster we have to prepaire both systems and let the data and meta partition sync before it can be used by corosync/pacemaker as the first cluster config will start drbd and take over the control of this service. So we prepaire our partitions this time before we do the actual cluster configuration as we did in openfiler 2.3.

Edit /etc/drbd.conf:

# You can find an example in /usr/share/doc/drbd.../drbd.conf.exampleinclude "drbd.d/global_common.conf":#include "drbd.d/*.res";resource meta { on filer01 { device /dev/drbd0; disk /dev/sdb1; address 10.10.50.101:7788; meta-disk internal; } on filer02 { device /dev/drbd0; disk /dev/sdb1; address 10.10.50.102:7788; meta-disk internal; }}resource data { on filer01 { device /dev/drbd1; disk /dev/sdb2; address 10.10.50.101:7789; meta-disk internal; } on filer02 { device /dev/drbd1; disk /dev/sdb2; address 10.10.50.102:7789; meta-disk internal; }}

After that create the meta-data on it:

root@filer01 ~# drbdadm create-md meta
root@filer01 ~# drbdadm create-md data

root@filer02 ~# drbdadm create-md meta
root@filer02 ~# drbdadm create-md data

Now you can start up drbd with:

service drbd start

on both nodes.

Make one node primary:

root@filer01 ~# drbdsetup /dev/drbd0 primary -o
root@filer01 ~# drbdsetup /dev/drbd1 primary -o

root@filer01 ~# mkfs.ext3 /dev/drbd0

root@filer01 ~# service openfiler stop

root@filer01 ~# mkdir /meta
root@filer01 ~# mount /dev/drbd0 /meta
root@filer01 ~# mv /opt/openfiler/ /opt/openfiler.local
root@filer01 ~# mkdir /meta/opt
root@filer01 ~# cp -a /opt/openfiler.local /meta/opt/openfiler
root@filer01 ~# ln -s /meta/opt/openfiler /opt/openfiler
root@filer01 ~# rm /meta/opt/openfiler/sbin/openfiler
root@filer01 ~# ln -s /usr/sbin/httpd /meta/opt/openfiler/sbin/openfiler
root@filer01 ~# rm /meta/opt/openfiler/etc/rsync.xml
root@filer01 ~# ln -s /opt/openfiler.local/etc/rsync.xml /meta/opt/openfiler/etc/
root@filer01 ~# mkdir -p /meta/etc/httpd/conf.d

root@filer01 ~# service nfslock stop
root@filer01 ~# service nfs stop
root@filer01 ~# service rpcidmapd stop
root@filer01 ~# umount -a -t rpc-pipefs
root@filer01 ~# mkdir /meta/etc
root@filer01 ~# mv /etc/samba/ /meta/etc/
root@filer01 ~# ln -s /meta/etc/samba/ /etc/samba
root@filer01 ~# mkdir -p /meta/var/spool
root@filer01 ~# mv /var/spool/samba/ /meta/var/spool/
root@filer01 ~# ln -s /meta/var/spool/samba/ /var/spool/samba
root@filer01 ~# mkdir -p /meta/var/lib
root@filer01 ~# mv /var/lib/nfs/ /meta/var/lib/
root@filer01 ~# ln -s /meta/var/lib/nfs/ /var/lib/nfs
root@filer01 ~# mv /etc/exports /meta/etc/
root@filer01 ~# ln -s /meta/etc/exports /etc/exports
root@filer01 ~# mv /etc/ietd.conf /meta/etc/
root@filer01 ~# ln -s /meta/etc/ietd.conf /etc/ietd.conf
root@filer01 ~# mv /etc/initiators.allow /meta/etc/
root@filer01 ~# ln -s /meta/etc/initiators.allow /etc/initiators.allow
root@filer01 ~# mv /etc/initiators.deny /meta/etc/
root@filer01 ~# ln -s /meta/etc/initiators.deny /etc/initiators.deny
root@filer01 ~# mv /etc/proftpd /meta/etc/
root@filer01 ~# ln -s /meta/etc/proftpd/ /etc/proftpd

root@filer01 ~# rm /opt/openfiler/etc/httpd/modules
root@filer01 ~# ln -s /usr/lib64/httpd/modules /opt/openfiler/etc/httpd/modules

Now do a restart and see if Openfiler can still run:

root@filer01 ~# service openfiler restart

service openfiler stop
mkdir /meta
mv /opt/openfiler/ /opt/openfiler.local
ln -s /meta/opt/openfiler /opt/openfiler

service nfslock stop
service nfs stop
service rpcidmapd stop
umount -a -t rpc-pipefs
rm -rf /etc/samba/
ln -s /meta/etc/samba/ /etc/samba
rm -rf /var/spool/samba/
ln -s /meta/var/spool/samba/ /var/spool/samba
rm -rf /var/lib/nfs/
ln -s /meta/var/lib/nfs/ /var/lib/nfs
rm -rf /etc/exports
ln -s /meta/etc/exports /etc/exports
rm /etc/ietd.conf
ln -s /meta/etc/ietd.conf /etc/ietd.conf
rm /etc/initiators.allow
ln -s /meta/etc/initiators.allow /etc/initiators.allow
rm /etc/initiators.deny
ln -s /meta/etc/initiators.deny /etc/initiators.deny
rm -rf /etc/proftpd
ln -s /meta/etc/proftpd/ /etc/proftpd

Change the lvm filter from:

filter = [ "a/.*/" ]

to

filter = [ "a|drbd[0-9]|", "r|.*|" ]

After that we can create the actual used stuff:

root@filer01 ~# pvcreate /dev/drbd1
root@filer01 ~# vgcreate data /dev/drbd1
root@filer01 ~# lvcreate -L 400M -n filer data

root@filer01~# corosync-keygen

( Press the real keyboard instead of pressing keys in an ssh terminal. )

Copy the authkey file to the other node and change the fileaccess:

root@filer01~# scp /etc/corosync/authkey root@filer02:/etc/corosync/authkey
root@filer02~# chmod 400 /etc/corosync/authkey

root@filer01~# vi /etc/corosync/service.d/pcmk

service { # Load the Pacemaker Cluster Resource Manager name: pacemaker ver: 0 }root@filer01~# scp /etc/corosync/service.d/pcmk root@filer02:/etc/corosync/service.d/pcmk

root@filer01~# vi /etc/corosync/corosync.conf

# Please read the corosync.conf.5 manual pagecompatibility: whitetanktotem { version: 2 secauth: off threads: 0 interface { ringnumber: 0 bindnetaddr: 10.10.50.0 mcastaddr: 226.94.1.1 mcastport: 5405 ttl: 1 }}logging { fileline: off to_stderr: no to_logfile: yes to_syslog: yes logfile: /var/log/cluster/corosync.log debug: off timestamp: on logger_subsys { subsys: AMF debug: off }}amf { mode: disabled}root@filer01~# scp /etc/corosync/corosync.conf root@filer02:/etc/corosync/corosync.conf

Openfiler 2.99 Active/Passive With Corosync, Pacemaker And DRBD - Page 2

View the original article here