[[PageOutline]] = HA-NFS = * Highly Available NFS with DRBD and Pacemaker * Ubuntu 10.04 Lucid 2.6.32-30-generic-pae * mdadm 2.6.7.1-1ubuntu15 * lvm2 2.02.54-1ubuntu4.1 * drbd8-utils 2:8.3.7-1ubuntu2.1 * nfs-kernel-server 1:1.2.0-4ubuntu4.1 * pacemaker 1.0.8+hg15494-2ubuntu2 * Tested with VMware Workstation 7.1.3 * see [./log] for verbose log = DRBD on LVM with RAID1 = * Primary * /dev/sd[bc]1 -> /dev/md0 (RAID1) * /dev/md0 -> /dev/vgnfs/drbd (LVM) * Secondary * /dev/sdb1 -> /dev/vgnfs/drbd (LVM) * 全体図 {{{ drbd/Primary (ubuntu-haa) -------------------------------------------------------------- | ------------------------- | | sda | system[ / ] | swap | | | ------------------------- | | | NIC(192.168.234.101) sdb sdc | || | || -------------- -------------- | || ----||------| |----------| |------------ || || | Linux | | Linux | || || | raid | | raid | || || | autodetect<=== RAID1 ==>autodetect | || || |------------| | |------------| || || -------------- | -------------- || || | || || ----------LVM---------- || || | | | || ======================drbd |*1 | *1: snapshot || | | | || ----------------------- || || || || || || drbd/Secondar (ubuntu-hab) || ----------------------------------- || | ------------------------- | || | sda | system[ / ] | swap | | || | ------------------------- | || | | NIC(192.168.234.102) sdb | | || |------------| | ----||------| |--------- || | LVM | || | |----| || |------------| | || -------------- | || | || ----------LVM---------- || | | | ======================drbd |*1 | *1: snapshot | | | ----------------------- }}} == packages == * sudo aptitude install -R mdadm * -R が無いと postfix を同時インストールされる * sudo aptitude install lvm2 * sudo aptitude install drbd8-utils -R * -R が無いと build-essential 等を同時インストールされる == setup block devices == === ubuntu-haa === ==== fdisk ==== * mitty@ubuntu-haa:~$ ls -l /dev/sd? {{{ brw-rw---- 1 root disk 8, 0 2011-03-24 17:20 /dev/sda brw-rw---- 1 root disk 8, 16 2011-03-24 17:20 /dev/sdb brw-rw---- 1 root disk 8, 32 2011-03-24 17:20 /dev/sdc }}} * mitty@ubuntu-haa:~$ sudo fdisk -u /dev/sdb {{{ Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First sector (63-4194303, default 63): Using default value 63 Last sector, +sectors or +size{K,M,G} (63-4194303, default 4194303): +2000M Command (m for help): t Selected partition 1 Hex code (type L to list codes): fd Changed system type of partition 1 to fd (Linux raid autodetect) Command (m for help): p Disk /dev/sdb: 2147 MB, 2147483648 bytes 255 heads, 63 sectors/track, 261 cylinders, total 4194304 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x7faa0456 Device Boot Start End Blocks Id System /dev/sdb1 63 4096063 2048000+ fd Linux raid autodetect Partition 1 does not end on cylinder boundary. Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. }}} * 同じ容量のHDDでも、メーカによって微妙にサイズが異なることがあるので、交換する際のことを考えるとディスク全体をパーティションに割り当てない方が良い * mitty@ubuntu-haa:~$ sudo fdisk -u /dev/sdc {{{ Disk identifier: 0x509a8b0b Device Boot Start End Blocks Id System /dev/sdc1 63 4096063 2048000+ fd Linux raid autodetect Partition 1 does not end on cylinder boundary. }}} ==== mdadm ==== * mitty@ubuntu-haa:~$ sudo mdadm --create /dev/md0 -l1 -n2 /dev/sd[bc]1 {{{ mdadm: array /dev/md0 started. }}} * mitty@ubuntu-haa:~$ cat /proc/mdstat {{{ Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md0 : active raid1 sdc1[1] sdb1[0] 2047936 blocks [2/2] [UU] [>....................] resync = 0.2% (4160/2047936) finish=8.1min speed=4160K/sec unused devices: }}} * mitty@ubuntu-haa:~$ sudo mdadm -D /dev/md0 * see [./log#mdadm-Ddevmd0] * mitty@ubuntu-haa:~$ sudo mdadm -Es {{{ ARRAY /dev/md0 level=raid1 num-devices=2 UUID=32578407:6f33f50b:893cf340:745f5dce }}} * mitty@ubuntu-haa:~$ sudo sh -c "mdadm -Es >> /etc/mdadm/mdadm.conf" * mitty@ubuntu-haa:~$ sudo dpkg-reconfigure mdadm {{{ ┌──────────────────────────┤ Configuring mdadm ├─ (snip) }}} * see [./log#dpkg-reconfiguremdadm] * see also wiki:TipAndDoc/storage/RAID#dpkg-reconfiguremdadm * command result {{{ * Stopping MD monitoring service mdadm --monitor [ OK ] Generating array device nodes... /var/lib/dpkg/info/mdadm.postinst: line 27: /dev/MAKEDEV: No such file or directory failed. Removing any system startup links for /etc/init.d/mdadm-raid ... update-initramfs: Generating /boot/initrd.img-2.6.32-30-generic-pae update-rc.d: warning: mdadm start runlevel arguments (2 3 4 5) do not match LSB Default-Start values (S) update-rc.d: warning: mdadm stop runlevel arguments (0 1 6) do not match LSB Default-Stop values (0 6) * Starting MD monitoring service mdadm --monitor [ OK ] }}} ==== pvcreate/vgcreate/lvcreate ==== * mitty@ubuntu-haa:~$ sudo pvcreate /dev/md0 {{{ Physical volume "/dev/md0" successfully created }}} * see [./log#pvcreatedevmd0] for detail * mitty@ubuntu-haa:~$ sudo vgcreate -s 32 vgnfs /dev/md0 {{{ Volume group "vgnfs" successfully created }}} * see [./log#vgcreate-s32vgnfsdevmd0] for detail * mitty@ubuntu-haa:~$ sudo lvcreate --size 1.5G -n drbd vgnfs {{{ Logical volume "drbd" created }}} * see [./log#lvcreate--size1.5G-ndrbdvgnfs] for detail === ubuntu-hab === ==== fdisk ==== * mitty@ubuntu-hab:~$ sudo fdisk -u /dev/sdb {{{ Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First sector (63-4194303, default 63): Using default value 63 Last sector, +sectors or +size{K,M,G} (63-4194303, default 4194303): +2000M Command (m for help): t Selected partition 1 Hex code (type L to list codes): 8e Changed system type of partition 1 to 8e (Linux LVM) Command (m for help): p Disk /dev/sdb: 2147 MB, 2147483648 bytes 255 heads, 63 sectors/track, 261 cylinders, total 4194304 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x4ab4a81c Device Boot Start End Blocks Id System /dev/sdb1 63 4096063 2048000+ 8e Linux LVM Partition 1 does not end on cylinder boundary. Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. }}} ==== pvcreate/vgcreate/lvcreate ==== * mitty@ubuntu-hab:~$ sudo pvcreate /dev/sdb1 {{{ Physical volume "/dev/sdb1" successfully created }}} * mitty@ubuntu-hab:~$ sudo vgcreate -s 32 vgnfs /dev/sdb1 {{{ Volume group "vgnfs" successfully created }}} * mitty@ubuntu-hab:~$ sudo lvcreate --size 1.5G -n drbd vgnfs {{{ Logical volume "drbd" created }}} * see [./log#lvm2onubuntu-hab] for detail