CompleteFusion

Recently, I needed to implement a low-cost, enterprise grade storage system for backups and to provide network attached storage for vserver-based Debian servers. After considering the requirements and projected short-term future growth, I decided to build a storage server with at least 12TB of raw storage, which would allow me to create two 4TB RAID6 arrays.

Ultimately, I was able to get 13TB of raw storage for less than $6300 (~$484 per TB, compared to many vendors starting prices of $1000 or more per TB).

Hardware

1. Supermicro CSE-933T-R760B Black 3U Rackmount Case – $769.99
2. Supermicro MBD-X7DBN-O Dual LGA 771 Intel 5000P Extended ATX Motherboard – $359.99
3. 2x Intel Xeon E5405 Harpertown 2.0GHz LGA 771 80W Quad-Core Processor – $224.99 ($449.98 for 2)
4. 4x Kingston 8GB (2 x 4GB) 240-Pin DDR2 667 FB-DIMM ECC RAM – $212.99 ($851.96 for 4)
5. 13x Western Digital RE3 1TB 7200 RPM SATA Hard Drive – $189.99 ($2469.87 for 13)
6. 2x Western Digital VelociRaptor 300GB 10000 RPM Hard Drive – $229.99 ($459.98 for 2)
7. HighPoint RR3540 RAID Controller – $799.99

Total: $6161.76 (plus shipping)

After reviewing the current Linux distributions targeted towards NAS systems, like FreeNAS and OpenFiler, I decided to stick with a simple system using Debian with LVM2 and AoE. My next post, Debian NAS using AoE (Part 2), will dig into the details of the OS and software setup.

I recently decided to install Plone to test its' usefulness as a content management system. When installing Plone on Debian Lenny, I ran into the following error:

pycentral: pycentral pkginstall: not overwriting local files

After a quick search, I found an easy way to resolve the error is to run the following command:

echo "overwrite-local = 1" >> /etc/python/debian_config

Then just install Plone as you normally would, and remove the added line from /etc/python/debian_config after the installation completes:

apt-get install plone3-site

I ran into an interesting situation recently — I had a Compaq BL10e blade server enclosure that was powered on and connected to the network, but no one knew the IP address for the enclosure's management interface. The enclosure also had a serial console option, but either the manual wasn't helpful enough in figuring out the correct connection settings, or it had been disabled/broken previously. To find the IP address for the enclosure, I made a cross-over ethernet cable and connected one of the blade server's ethernet ports to the enclosure's management ethernet port.

Wireshark is a network protocol analyzer, which allows you to view/capture network traffic. I setup Wireshark on the blade server (on a debian server, just apt-get install wireshark), specified the interface I wanted to watch, and watched the packets from the enclosure's management interface show up in Wireshark. Since Wireshark displays the IP address for the network traffic it captures, I was easily able to grab the IP and connect to the enclosure's management interface.

The same procedure could be applied to any machine that you don't know the IP address for, and can afford to temporarily disconnect from its network connection.

I run CompleteFusion, and several other sites, on a virtual server hosted at VPSLink.  Recently, I upgraded the server from Debian Etch to Lenny.  After the upgrade, I've noticed errors like the following in the output from dmesg:

4gb seg fixup, process imap-login (pid 29663), cs:ip 73:002d9226
printk: 121 messages suppressed.

Some basic investigation revealed that this is apparently a fairly common issue with Xen domU instances, but it should be easy to fix:

• First, make sure libc6-xen is installed: apt-get install libc6-xen
• If /lib/tls exists on your system (it didn't on mine): mv /lib/tls /lib/tls.disabled
• echo 'hwcap 0 nosegneg' > /etc/ld.so.conf.d/libc6-xen.conf && ldconfig
• Reboot your domU

I've experimented with a wide variety of virtualization software in the past, but have recently settled on two. VMWare, VirtualPC, and others may have their uses, but I've found VServer and Xen to be my new favorites. Not only are they both free, but they also offer near-native performance. VServer makes it incredibly easy and fast to set up new guest systems, but also can meet more complex requirements with some additional minor configuration. The only feature that VServer can't provide is the ability to run Windows, but Xen will on any processor that can do hardware virtualization, or HVM (Intel's VT or AMD-V will).

Since VServer is easier to use and faster to setup for general Linux virtualization, and Xen can run Windows, the need to use both quickly becomes apparent. They both require kernel patches to work, but a well-oiled Debian setup makes that as easy as running apt-get install. One thing I would suggest is to use LVM – it makes giving guest machines disk space easier. Assuming you have a box already set up running Debian Etch, just apt-get install the following packages and you're well on your way to virtualization with Xen and VServer:

libc6-xen xen-hypervisor-3.0.3-1-i386-pae xen-ioemu-3.0.3-1 xen-linux-system-2.6.18-4-xen-vserver-686 xen-tools xen-utils-3.0.3-1 xen-utils-common bridge-utils linux-headers-2.6-xen-vserver-686 linux-image-2.6-xen-vserver-686 util-vserver vserver-debiantools vncserver vnc-common

For more information, check out:

• The Perfect Xen 3.0.1 Setup for Debian Etch
• Linux-VServer on Debian Etch
• VServer Documentation

Note: The first two links describe installing Xen and VServer separately — the only change required to use both is to grab the -xen-vserver kernel packages instead of only -xen or -vserver alone.