cn=Directory Manager

ZFS also provides RAID-Z, which is a parity scheme like RAID-5, but doesn't require NVRAM for either performance or correctness. If you also enable compression, you can actually have compressed, replicated data that consumes less total disk space than a single copy of the original would! For example, four 500GB drives (2TB total space) configured as a RAID-Z group can typically store about 4TB of logical data (assuming 2-3x compression, which is typical).

Posted by Jeff Bonwick on January 20, 2006 at 03:04 AM CST #

Thanks for your comment about RAID-Z. I'm aware of it and have read your comments on it. The features and flexibility that it offers are certainly attractive, but unfortunately I haven't really gotten a chance to test it. My guess is that because it is similar to RAID-5, RAID-Z would exhibit the same kinds of performance degradation and we would still probably want to stick with a 1+0 approach to sacrifice space (but compression would help alleviate that) for performance and also better redundancy because we could lose multiple disks and still be OK as long as they were in different mirrors. The performance of RAID 1+0 in a failure state should also be much better given that there shouldn't be a need to read from all the disks and calculate parity.

If RAID-Z were able to provide both the space of RAID-5 and performance of RAID 1+0 (including in a failure state), then that would be a truly impressive feat. I can't see how that would be possible, but it also wouldn't be the first time that ZFS has shown counter-intuitive performance results where it can still be faster in the face of doing more work (e.g., compression and checksums).

Posted by Neil Wilson on January 20, 2006 at 12:36 PM CST #

I don't think that RAID-Z would give the same performance as a RAID 0+1 but, with the variable chunk size of the writes, you actually get a big performance increase over RAID-5. One of the biggest performance penalties of RAId-5 is when you don't get to fill exactly everything in a Disk block (with happens pretty much every time) with forces a read to every disk that will not have a block completelly filled so the parity can be calculated. If you avoid those extra reads, you'll improve the Writting performance a lot. Anyway, have you actually tested the RAID-5 performance in LDAP operations? From what I've been understanding from your previous posts, the Directory Server doesn't need a very big sustainable performance since most of the Disk accesses are made in Bursts so, with the use of a few 6130, I don't think you would saturate the caches. What do you think?

Posted by Jaime Cardoso on January 20, 2006 at 06:14 PM CST #

It is true that under ideal conditions, disk performance isn't that important. As a rule, the faster the disks the faster that write operations will be. Usually, when the server needs to perform a read-only operations like searches, binds, or compares, it is best if the targeted entries can be retrieved from memory. However, for cases in which you have a huge database that can't possibly fit in memory (even in the DB cache or filesystem cache), then you can't help but perform disk reads.

In this particular case, we tested multiple directory sizes, with the smallest one taking about 210GB of disk space. The largest system that we have in our lab is the Sun Fire 6800 with 192 GB of memory, so there will always be parts of the database that won't be in memory. In that case, whenever something is requested that isn't in memory, it will perform a random disk read. It doesn't take many of those to fully saturate your storage.

Posted by Neil Wilson on January 20, 2006 at 07:03 PM CST #

The Importance of RAID