Building a CentOS 6.2 NAS/SAN with ZFS

From nerdongeek

Jump to: navigation, search

I've had fairly good success with my two Ubuntu-based ZFS NAS boxes running NFS. I have had no end of problems trying to get them working as iSCSI targets, though. I decided to eat my own dog food (as an employee of a large RedHat partner) and build myself a new CentOS 6.2 NAS on our own hardware, and migrate the ZFS pools (SBOD and JBOD shelves in this case) from both Ubuntu boxes over to this one new system.

Hardware:

  • Server/Head - Dell PowerEdge 2950 (Gen 1), dual Xeon LV, 16 GB DDR2 FBDIMMs, Qlogic QLA-2432 HBA, some random SCSI U320 controller
  • SBOD - Xyratex (Xiotech) 16 bay FC2 Fibre Channel shelf with dual controllers and 16 x 146GB FC2 drives
  • JBOD - Dell PV220S 14 bay SCSI U320 shelf with single controller and 14 x 146GB U320 SCSI drives
  • OS - CentOS 6.2 with ZFS-FUSE and all OpenManage tools from Dell YUM repo

I also decided to build a couple of VMs - Ubuntu and CentOS - to replicate the hardware environment I am doing all this in. The idea here is to validate the process of moving individual disks (the *BOD shelves) that are managed by ZFS on Ubuntu over to a CentOS system and import the ZFS pool. In theory, that should prove that I can move the FC and SCSI controllers, cables and controllers to the PowerEdge 2950 without issue. My biggest concern really is that I am exporting the ZFS filesystem via NFS, so I may have to migrate all the VMs to a different datastore and start over. We'll see how it goes! I'll be updating this entry as I get the project done.

Set static IP on eth0

Edit /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE="eth0"
HWADDR="00:50:56:92:6D:1D"
BOOTPROTO=none
ONBOOT=yes
NETWORK=10.1.1.0
NETMASK=255.255.255.0
IPADDR=10.1.1.14
USERCTL=no
GATEWAY=10.1.1.254

Install ZFS

rpm -Uvh http://download.fedora.redhat.com/pub/epel/6/i386/epel-release-6-5.noarch.rpm
yum install zfs-fuse

Install SCSI Target Tools, tgtd, sg3_utils and enable mutipath at boot

yum install scsi-target-utils
service tgtd start
chkconfig tgtd on
chkconfig multipathd on
yum install sg3_utils

Kill the default iptables chain

iptables -P INPUT ACCEPT
iptables -F

Install Webmin so you can easily change the iptables config

Linux, security and iptables purists, feel free to chastise me. I hate host-level firewalls. Deal with it.

yum install wget
wget http://prdownloads.sourceforge.net/webadmin/webmin-1.570-1.noarch.rpm
rpm -U webmin-1.570-1.noarch.rpm

Edit SELinux config and change to permissive

edit /etc/selinux/config and change SELINUX=enforcing to SELINUX=permissive

Personal tools
Google AdSense