Those of you who have to deal with audit trails from busy systems know that they can get really big, and when you need to store them for a couple of years they consume a considerable amount of disk.
To minimize the disk usage that you can compress the audit trails, and it works really well (I've adjusted the output to right-align the size):
root@warlord# ls -l total 19447638 -rw-r--r-- 1 root other 1353214369 Aug 9 09:27 20070728065900.20070730163539.warlord -rw------- 1 root other 62209268 Aug 7 08:25 20070728065900.20070730163539.warlord.gz -rw-r--r-- 1 root other 1965073391 Aug 7 09:35 20070728065900.20070730163539.warlord.txt -rw-r--r-- 1 root other 71460194 Aug 7 09:35 20070728065900.20070730163539.warlord.txt.gz
As you can see the compression ratio is really good (over 90%),
but one problem still remains:
if you need to work with the files you have to uncompress them before you can run your scripts to go an find who edited /etc/passwd.
Uncompressing one file doesn't take that long,
but when you don't know exactly in which file the audit records you are looking for are,
things start to take time.
Enter ZFS: with on-the-fly disk compression it is the perfect file system to store audit trails. First of all you have to enable compression:
root@warlord# zfs set compression on pool/audit
That only makes future writes compressed, so the files you already have needs to be rewritten to be compressed. After having done that it is time to look at the compression ratio:
root@warlord# ls -l total 19447638 -rw-r--r-- 1 root other 1353214369 Aug 9 09:27 20070728065900.20070730163539.warlord -rw-r--r-- 1 root other 1965073391 Aug 7 09:35 20070728065900.20070730163539.warlord.txt
Wait a minute! That didn't compress anything, or did it? ls -l shows the size of the uncompressed file, so you have to use du to see the compressed size:
root@warlord# du -k 554067 20070728065900.20070730163539.warlord 732399 20070728065900.20070730163539.warlord.txt
Much better! (Note that it displays the size in kilo bytes while ls -l displays it in bytes) But it still is not as good as when I ran gzip on them. Why?
ZFS uses the
LZJB compression algorithm
which isn't as space effecient as the gzip algorithm, but it is much faster.
If I had been running Nevada I could have used:
root@warlord# zfs set compression gzip pool/audit
And have gotten the same compression ratio as when I "hand compress" my audit trails. This thanks to Adam who integrated gzip support in ZFS in build 62 of Nevada.
[Technorati Tags: OpenSolaris ZFS Security ]
