I have some older systems that need new hard drives. I came across a great deal on hard drives with the caveat that they were sata. Well, I would prefer sata drives in the machines for the cabling if nothing else. However these systems didn’t have sata ports so I decided to try and find out if I could find a sata controller card that was cheap enough to not ruin the value of the drives. It’s not easy to find out what chipset most cards use and then check the Linux compatibility but here’s what I found.

When using smartctl the return value can provide some useful information but it requires looking up what the individual set bits mean. After running smartctl on a large number of drives here are the codes I found and looked up:

• 32 = SMART status check returned “DISK OK” but we found that some (usage or pre-fail) Attributes have been <= threshold at some time in the past
• 64 = The device error log contains records of errors
• 96 = Both of the above

I’ve recently been enabling the monitoring of SMART via Munin. Most of the disks had SMART enabled, presumably thought the BIOS setting. When I came across one that wasn’t I realized I didn’t know how to enable it except through BIOS. Since this machine was located remotely I quickly figured out how to enable it via this command:

• smartctl -s on /dev/sdX

The ext3 filesystem supports write barriers which are designed to allow a filesystem to take advantage of a disk’s write cache without fear of compromising the ingetrity of the filesystem on a power failure or kernel panic. I’ve seen reports that ext3 write barriers have a significant impact on performance. Of course, the alternative is to disable the write cache (as is recommended many places) and suffer the performance loss from that. Given that the use of write barriers means forsaking many Linux storage tools which do not support write barriers (lvm, dmraid, dmcrypt, md raid except raid1, etc) it seems like the best option for flexibility is to just turn off the hard disks write cache and keep one’s options open. However, if the difference in performance between write barriers enabled and write cache disabled is significant then using write barriers would be a better option when IO performance is important. I often find IO to be the bottleneck on my servers so any improvement in IO performance is worth investigating. I ran some tests with bonnie++ and came up with these numbers.

I've been quite happy with my purchases of Seagate's 7200.11 drives. The 640GB and 750GB are quite cheap and the 1.5TB makes a great external drive. I've replaced my 2 500GB external drives with a 1.5TB drive so that I can have 1 drive for the sake of simplicity. The 640GB drive is a 2 platter design and I suspect that the 750GB is always a 3 platter despite Seagate having the tech for 375GB platters for the 4 platter 1.5TB. Given my experience with drives with 1 year warranties failing within 2 years I'm pleased that Seagate offers a 5 year warranty on all of these drives. The only other drive I will be purchasing in the near future is the Western Digital RE3 for servers. The multiuser performance of the RE3 is somewhat above the 7200.11. However, the 7200.11 has performed quite well in my tests and I'm also impressed by it's low noise and low temperature.

The best article to read about these drives is the one from AnandTech though the TechReport one is decent also. The AnandTech review does a great job investigating the cheaper solid state drives to uncover their performance issues with high latency. Their conclusion is to avoid any JMicron JMF602 based SSD’s. That’s too bad for me since I’m laptop shopping and I was thinking of supplementing my laptop with a reasonably priced SSD. And I still may but it’s probably just as likely that I’ll wait a while and see how this data shakes out. The folks at OCZ are investigating these issues. As for the Intel X25-M I’d say it’s out of my price range. Of course I wouldn’t mind a smaller capacity drive, say half the capacity for the half the price and I could become interested very quickly.

If there’s one thing about hard drives I’ve learned in the last couple of years is that hard drives with a 1 year warranty are not worth buying. The likelihood of a drive failure before 3 years is fairly high in my experience. Right now I’m going through drives which have failed or are failing and checking their warranties to see if any can be replaced before I take them to freegeek for disposable. I figured it would be wise to make a list of links of warranty checkers.

I thought there was an easier way to do this but perhaps I have forgotten. I am a fan of Linux software RAID because of the fact you can transport the drives to other systems without having to worry about hardware RAID compatibility. Linux software RAID 1 also possesses good performance and reliability. However, sometimes you need to repair file system damage or create an image of the filesystem or perform other tasks needing the arrays to be accessible from either a Live CD or from another system. Here’s how I accomplished this recently.

Write barriers have gotten some discussion in the Linux world recently which is a good thing due to a lack of documentation about what write barriers are and when they should be used. Let's first start with the assumption that a user wants file system integrity over performance. This often means turning off write caching on all disks. The reason is that file systems rely on the disks not reordering write requests in order to maintain the integrity of the file system. Of course, turning off write caching harms write performance substantially so another option was invented: write barriers.

Just thought I’d make a note of these for further research. Not sure I’ll actually use them in a project but it is a possibility.

• Gfarm
• GlusterFS
• Hadoop - write-once-read-many, not production ready
• MogileFS - write-once-read-many
• HAMMER - not production ready

I suppose that growing a RAID 1 array is not very common but I had a client that wanted to go from a 2 disk array to a 3 disk array so that one disk could be rotated out periodically and kept off-site. Here were the steps:

• mdadm --add /dev/mdX /dev/sdXX
  • This adds the partition to the array as a hot spare.
• mdadm --grow /dev/mdX -n 3
  • This increases the number of disks from 2 to 3.

Prices on SSDs are coming down and performance is going up. It may not be time to be buying yet, but that time may be sooner than people think.

• 64GB SSD on the Desktop: Samsung and OCZ go mainstream
• Memoright SSDs: The End of Hard Drives?

I’ve been doing a lot of research on storage solutions for a couple of projects. I’ve been delving into RAID setups for large numbers of disks and trying to determine in what situations a hardware RAID controller is appropriate instead of Linux software RAID. I’ve also been considering various nested RAID levels and their value.

WD’s new 10,000 RPM drive has been sent to various hardware websites for testing. The drive has an unfinished firmware but so far seems to score at the top of performance tests, especially for multiuser benchmarks. However it suffers 2 flaws: low capacity and high price. For the same price one could buy 2 750GB Seagate 7200.11 drives and run them in RAID 1 or RAID 0 in order to match the performance of the VelociRaptor and have 2.5 or 5 times as much capacity. I think the only use case is for a large disk array where performance is important but larger number of disks is impractical. I would think a 16-disk array of VelociRaptors might be a better setup than a 32-disk array of 7200.11 drives. I am happy about the low power consumption of the new drives due to their 2.5” design. I’ve compiled a list of reviews and further comparisons below.

I have tended to setup Xen systems with LVM storage due to the performance increases I’ve observed compared to file-based storage. Recently I’ve decided to largely transition to file-based storage primarily for the increase in manageability that file-based virtual machine images provides. Files are much easier to manipulate for backups, migrating to other systems, and similar tasks. While I will still use LVM in certain situations I think the file-based images is a better choice until one runs into performance bottlenecks. Even at that point there may be other alternatives available.