[H-GEN] Linux file server

[David Jericho]

Paul Gearon wrote:

> I don't have much experience with ATA RAID, but I'd definately recommend 
> going SATA over PATA.  Also, we're using SATA RAID here at work on an 
> Opteron, and it's great (ask David Jerico for details).

I'll share.

A single Opteron 240 driving a PCI-X (that's X, not Express) 3ware 8506 
8 port controller. 6 7200 RPM Western Digital 200 GB JB series drives, 
running ext3 on Fedora Core 2 using the mount options of noatime and 
data=writeback. Formatted using the switches "-o 
dir_index,sparse_super,filetype -R stride=96 -b 4096". It's a 1.2 TB 
RAID 0 array, we have no need for data reliability on this machine.

I used bonne++ 1.03a to test generic application performance on disk, 
but this does not reflect the speeds we get with our custom 
applications, or what you may receive using Samba or other better 
written (or Linux specific) applications. bonnie++ does not test mmap(2) 
or sendfile(2) performance, and using the above I've seen burst read 
speeds with large files around 440 MByte/s.

The 2.6 kernels have significantly lower throughput than the 2.4 
kernels, and it's also worth noting that Ext3 had the slowest read 
throughput by around 50 MBytes/s, with Ext2 beating it. XFS formatted to 
match the RAID 0 array had the fastest at approximately 220 MBytes/s. 
2.6 however did have the lowest latency, as did Ext3 with the above options.

Slightly trimmed bonnie++ 1.03a output for the above Ext3 config returned:

------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
   8G 33580  96 171900  80 42479  13 33077  79 93868  12 575.3 1

------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
    16 24480  98 +++++ +++ +++++ +++ 19843  79 +++++ +++ +++++ +++

FYI, + characters mean the test completed too quickly for bonnie++ to 
return meaningful data. I have yet to rerun the above bonnie++ with a 
larger dataset.

It maybe important to note that as of the latest 2.6 kernels, Linux on 
the AMD64 platform can only memory map a 340GB per process. This is due 
mainly to a VM paging system ported from the ia32 platform that should 
have been left on the hillside at birth to die. I have not tested *BSD 
because we have not done enough research to confirm if the Linux 
emulation works on AMD64 for AMD64.

And yes, before anyone says "Why would you want to do that?" we do hit 
this limitation.

-- 
David Jericho
Senior Systems Administrator, Tucana Technologies