Storage R Us

One thing I appreciate in ZFS is the ability to quickly check system limits, or at least 64 bit code completeness, since making sparse volumes or sparse files is pretty easy. As most shells support the bitshift parameter, this is just a couple trivial oneliners:

To make an sickly huge 8EB sparse file (and yes that's 8 exabytes) on a zfs volume:

killer:bigpool jone# mkfile -n $(((1<<63)-512)) /bigpool/sickfile

And now to make a block aligned 8EB sparse volume:

killer:bigpool jone# zfs create -s -V $(((1<<63)-512)) bigpool/sickvol

let's take a look

killer:bigpool jone# ls -lh sickfile
-rw------T   1 root     root        8.0E Feb 29 17:11 sickfile

killer:bigpool root# format -e /dev/zvol/rdsk/bigpool/sickvol
selecting /dev/zvol/rdsk/bigpool/sickvol
No defect list found
[disk formatted, no defect list found]


FORMAT MENU:
        disk       - select a disk
        type       - select (define) a disk type
        partition  - select (define) a partition table
        current    - describe the current disk
        format     - format and analyze the disk
        fdisk      - run the fdisk program
        repair     - repair a defective sector
        show       - translate a disk address
        label      - write label to the disk
        analyze    - surface analysis
        defect     - defect list management
        backup     - search for backup labels
        verify     - read and display labels
        volname    - set 8-character volume name
        !     - execute , then return
        quit
format> ver

Volume name = <        >
ascii name  = 
bytes/sector    =  512
sectors = 18014398509481855
accessible sectors = 18014398509481855
Part      Tag    Flag     First Sector                 Size                 Last Sector
  0   reserved    wm                34            8388608.00TB                  18014398509481855    

format>

egad! have i just boiled the oceans?? of course to fill a real volume like this is another story since at an unrealistic sustained 10GB/s it'd take you over 27 years .. enjoy!

forgive me father for i have sinned .. it's been over 2 years since my last blog entry - i must confess that it's just one of those things that's easy to put off for another day, but that day never comes until you seize it .. (carpe diem and all that corpus crapus) ..

anyhoo - there's a lot of exciting things that've been happening more recently in our storage software stack that i hope to delve into in the coming days, minutes, weeks, months and years .. but perhaps i should [re]-introduce myself and what i've been doing here the past 10+ years for the illustrious fireball in the sky:

year 1: internal system administration for the sun java centers (ah, the old professional service oriented java consulting days) .. this was an interesting exercise in promoting corporate communication and cooperation with a very small budget .. we focused a lot on building up aliases for cooperation and had a few machines that we found in various closets and through other sources to collaborate on some projects and internal development .. (let me just say that i thought i'd never see the day when sun consultants were welcome to tout whatever laptop and operating system they so desired, and our source would be completely opened)

years 2-10: various professional services engagements across the country .. ah from network solutions at their high time when they were split, to many hours at AT&T Wireless, AT&T corporate, many gov't jobs, hospitals, media companies, financial svc industries, and more .. always a blast implementing, testing, filing bugs, fixing, working around, proving and breaking many things

ever since the LSC acquisition back in 2001 (bringing us a wealth of experience from Control Data along with SAM-QFS) i've always had a special place in my heart for the filesystem and storage problem .. particularly as it relates to storage performance and global access issues .. it's been fun to see ZFS take the stage and integrate some lovely cache related goodness (ah - slab allocator on the VOP) and watch as the system folk took on a greater interest in storage issues (it's much more than an accessory you know) .. so i guess that's where I've really been focusing much of my efforts over the years .. for those of you who've i'd had the pleasure to work with over the years - it's always been a treat to hear of your problems and issues you're trying to solve as well as your experiences with sun (i have many of my own)

Ok .. mostly testing to see if this works (.. slid it to the sidebar .. I gotta start putting some better pictures on my flickr site) .. I should also start posting again

If you haven't played with it yet: Google Maps really is one of the better interfaces out there. Smooth zoom, and clear detailed road layout with a preload, clean recentering, and smooth dragging makes this all pretty nice. Of course it is rather US centric, and if you zoom on the default map you'll find that the coffeyville country club in kansas is the center of the world! The relevance of results from their local search is a little funny if you don't have an exact business name .. for example a search for "thread stores" in Soho gave us things like "The MoMA design store", "the Apple store" (presumably matching on store), and KidRobot .. none of which carry any sort of thread. A search on just "thread" seems to yield much better results.

Overall with google, I guess Sergey Brin and his team seem to be setting a number of standards. I do remember his geeky face from many late nights in the old WAM labs - hacking on the old NeXT black boxes. I do appreciate their initial work they did back at Stanford on deterministic relevance. It's a pity that they haven't developed a better business model, and seem to have forgotten most of their academic roots. It's nice that the ads are a little less intrusive looking, but just like Red Hat, motives often become questionable when you cross over into that corporate realm. Good to see his dad still teaching math at UMD though. I guess the contributions back into academia are probably key for real innovation and growth .. funny how much you can lose focus when you put on the million dollar corporation hat (and yes i know it's ironic writing this on a sun blog .. i guess i'm just jealous i don't have the $4B and my stock [of which i have admirably fewer shares] are at 1/45th their value with worthless options i can buy at 10x our current price)