Evaluating the performance of ext3 using write barriers and write caching

The ext3 filesystem supports write barriers which are designed to allow a filesystem to take advantage of a disk’s write cache without fear of compromising the ingetrity of the filesystem on a power failure or kernel panic. I’ve seen reports that ext3 write barriers have a significant impact on performance. Of course, the alternative is to disable the write cache (as is recommended many places) and suffer the performance loss from that. Given that the use of write barriers means forsaking many Linux storage tools which do not support write barriers (lvm, dmraid, dmcrypt, md raid except raid1, etc) it seems like the best option for flexibility is to just turn off the hard disks write cache and keep one’s options open. However, if the difference in performance between write barriers enabled and write cache disabled is significant then using write barriers would be a better option when IO performance is important. I often find IO to be the bottleneck on my servers so any improvement in IO performance is worth investigating. I ran some tests with bonnie++ and came up with these numbers.

I just did a couple of runs in each test with bonnie++ and rounded the numbers.

ext3 unsafe - write cache on, write barriers off

• Sequential Block Write - 40MB
• Block Rewrite - 18MB
• Sequential File Create - 28,500
• Sequential File Delete - 16,000
• Random File Create - 27,500
• Random File Delete - 6600

ext3 safe and fast - write cache on, write barriers on

• Sequential Block Write - 37MB
• Block Rewrite - 17MB
• Sequential File Create - 30,000
• Sequential File Delete - 18,500
• Random File Create - 32,000
• Random File Delete - 6400

ext3 safe and slow - write cache off, write barriers off

• Sequential Block Write - 24MB
• Block Rewrite - 14MB
• Sequential File Create - 21,000
• Sequential File Delete - 7300
• Random File Create - 18,500
• Random File Delete - 2000

Well, I think I see where ext3 gets its reputation for slow deletes. With the write cache off the delete performance is terrible, nearly 70% lower. It’s clear that enabling write barriers does something as the numbers are lower on a number of items (though not all). However it’s clear that write barriers is minor loss of performance compared to turning the write cache off. I think this leads me to consider how many servers I can run with just ext3 and md raid1 so as to keep the write cache enabled and the filesystem safe. I’ll have to weigh the performance gains against the benefits of using LVM (especially snapshots) and dm-crypt (which might have limited benefits on a server anyway).

And of course I’ll be first on the list for testing a Linux filesystem that can do snapshots, volumes, checksums, and handle write barriers effectively. I’m looking at you Btrfs and Tux3.