ZFS Puts Net App Viability at Risk?

About a month ago, Network Appliance sued Sun to try to stop the competitive impact of ZFS on their business.

I can understand why they're upset - when Linux first came on the scene in Sun's core market, there were some here who responded the same way, asking "who can we sue?" But seeing the future, we didn't file an injunction to stop competition - instead, we joined the free software community and innovated.

One of the ways we innovated was to create a magical file system called ZFS - which enables expensive, proprietary storage to be replaced with commodity disks and general purpose servers. Customers save a ton of money - and administrators save a ton of time. The economic impact is staggering - and understandably threatening to Net App and other proprietary companies. As is all free innovation, at some level. So last week, I reached out to their CEO to see how we could avoid litigation. I have no interest whatever in suing them. None whatever.

Their objectives were clear - number one, they'd like us to unfree ZFS, to retract it from the free software community. Which reflects a common misconception among proprietary companies - that you can unfree, free. You cannot.

Second, they want us to limit ZFS's allowable field of use to computers - and to forbid its use in storage devices. Which is quizzical to say the least - in our view, computers are storage devices, and vice versa (in the picture on the right - where's the storage? Answer: everywhere). So that, too, is an impractical solution.

We're left with the following: we're unwilling to retract innovation from the free software community, and we can't tolerate an encumbrance that limits ZFS's value - to our customers, the community at large, or Sun's shareholders.

So now it looks like we can't avoid responding to their litigation, as frustrated as I am by that (as I said, we have zero interest in suing them). I wanted to outline our response (even if it tips off the folks at Net App), and for everyone to know where we're headed.

First, the basics. Sun indemnifies all its customers against IP claims like this. That is, we've always protected our markets from trolls, so customers can continue to use ZFS without concern for spurious patent and copyright issues. We stand behind our innovation, and our customers.

Second, Sun protects the communities using our technologies under free software licenses. As an example, Apple is including ZFS is in their upcoming "Leopard" OS X release. This is happening without any payment to Sun (that's how truly free software works). Under the license, we've waived all rights to sue them for any of the patents or copyright associated with ZFS. We've let Apple know we will use our patent portfolio to protect them and the Mac ZFS community from Net App. With or without a commercial relationship to Sun.

That's true for any licensee - in fact, Net App could adopt ZFS today and receive the same protection. The port is done to FreeBSD, the OS on which Net App's filers are built. They could use it without owing us a dime, and they'd be protected from our portfolio. (The quid pro quo? They'd have to agree to offer reciprocal protection to Sun.)

Third, we file patents defensively. Like MySQL or Red Hat, companies similarly competing in the free software marketplace, we file patents to protect the communities from which innovation and opportunity spring. Unlike smaller free software companies, we have one of the largest patent arsenals on the internet, numbering more than 14,000 issued and pending globally. Our portfolio touches nearly every aspect of network computing, from multi-core silicon and opto-electronics, to search and of course, a huge array of patents across storage systems and software - to which Network Appliance has decided to expose themselves.

And to be clear, once again, we have no interest whatever in suing NetApps - we didn't before this case, and we don't now. But given the impracticality of what they're seeking as resolution, to take back an innovation that helps their customers as much as ours, we have no choice but to respond in court.

So later this week, we're going to use our defensive portfolio to respond to Network Appliance, filing a comprehensive reciprocal suit. As a part of this suit, we are requesting a permanent injunction to remove all of their filer products from the marketplace, and are examining the original NFS license - on which Network Appliance was started. By opting to litigate vs. innovate, they are disrupting their customers and employees across the world.

In addition to seeking the removal of their products from the marketplace, we will be going after sizable monetary damages. And I am committing that Sun will donate half of those proceeds to the leading institutions promoting free software and patent reform (in specific, The Software Freedom Law Center and the Peer to Patent initiative), and to the legal defense of free software innovators. We will continue to fund the aggressive reexamination of spurious patents used against the community (which we've been doing behind the scenes on behalf of several open source innovators). Whatever's left over will fuel a venture fund fostering innovation in the free software community.

And on that note, I want to thank the free software advocates from across the world who've offered expert testimony, and reams of prior art to defend ZFS, and the community of which Sun's a part. Please rest assured we will use this opportunity to highlight the futility of using software patents to forestall competition - in the commercial marketplace, and among the free community.

In the interim, if you're a Net App customer looking for alternatives, we would be pleased to talk to you about lowering the cost of proprietary storage - if you're a technical sort, start by trying out ZFS in software form. (There are also lots of reviews available, this one just posted). We'd also be happy to send you a free trial Storage System based on ZFS (pick the x4500 here). And remember, we indemnify our customers.

The shift to commodity infrastructure is as inevitable as the rising tide - although for some, I'm sure it feels like a rogue wave.

Posted on 10:00AM Oct 24, 2007 | Comments[132]

Share this post  | | | | | |

Hugging Customers (Not Trees)

(Update - Interview with Dave Douglas, Sun's VP, Economic-Responsibility at bottom of this entry)

A few folks know I like to cook. It pairs well with liking to eat. A good friend gave me a beautiful cookbook last year, Thomas Keller's The French Laundry Cookbook.

It's gorgeous book, but I will admit to having prepared a dish from it only once - the recipes are complicated, and take far more time than I typically have (the author admits, up front, people like me aren't his target demographic). But one of the things I love about the book is Keller's focus on efficiency - out of respect for the ingredients, and the economics of running a commercial kitchen. "Should a cook squander anything, ever?" Great chefs and kitchens waste nothing.

Now, waste is a generic term. It means one thing to a consumer frustrated with the 60,000,000 plastic bottles we dispose of daily in the US. It's another thing to a CIO who just realized she's running datacenters at 10% efficiency. At that level, as in a commercial kitchen, it's not an annoyance, it's a waste. Of money.

About five years ago, we made a simple but important bet - that our customers would eventually look at waste in their datacenters with a far more scrutinizing eye. We bet the cost of powering a computer would eventually exceed its purchase price - and thus focusing on energy waste would be a profitable pursuit. (Just imagine if your gasoline cost more than your car... oil hit $86/barrel today, btw.)

We introduced our first energy efficient server system about 18 months ago, known as Niagara 1 (the name is a nod to the machine's throughput capacity). We did the introduction in London, home to the world's most expensive real estate - space being a precious asset for most of our customers, as well.

But after years of R&D, we showed up with something confusing: we rejected the notion of speed at any cost (a first in the industry), and optimized the system not for speed, but for efficiency, reducing power waste. Like a bus, not a Formula 1 racer (the former getting radically better per passenger mileage). We similarly suggested the world should move away from running one application per machine, and instead collapse many tasks onto the same machine - heresy at the time has since been renamed virtualization.

Those who love desktop computers thought we were daft. Here we had what looked like a slow chip, optimized for something no home user really cared about (lowering power bills, running multiple OS's and minimizing space). And to make matters worse, we removed support for floating point precision math on the chip - to save more power and space. Desktop users (who play games that often feast on floating point processes) thought we were loons, but most datacenters didn't notice (very few datacenters use floating point).

Net result? Within two quarters, revenue hit $100m per quarter. And by last quarter, systems based on Niagara 1 were nearly a billion dollar annualized business - from a cold start 18 months prior. It wasn't for every application (don't simulate nuclear fission on a Niagara 1 system), but for internet workloads (databases, web and app servers), it set new records for work/watt.

Were we hugging trees? That wasn't the only point. We were hugging our customers. Customers out of space and power who, like great chefs, feel there's no point in waste.

So a couple weeks ago, we introduced the second generation of Niagara based systems, known as Niagara 2. Niagara 2 adds an incredible breadth of new features and performance enhancements (this is a good summary).

We shrunk the process used to make the chip, and upped the clockrate to boost overall performance. We doubled the thread count (8 cores x 8 threads = 64 threads), and via xVM (formerly, Project Virginia) virtualization: a single Niagara 2 chip can collapse 64 independent operating systems onto a single system. Not separate application partitions, 64 separate operating systems - from Solaris and other real time OS's, to Linux or BSD. (As far as we know, no one else can do that without a huge performance penalty.)

We added cryptographic ciphers (algorithms used to encrypt/decrypt data for secure storage or transmission over the internet) onto the microprocessor - so users don't have to power, or provision space for extraneous security functions. We added (dual 10 Gigabit ethernet) networking onto the chip, eliminating yet more waste. Niagara 2 isn't simply a server, it's a true system - in the spirit of the Systems team introducing it. Our first Niagara 2 systems are as wondrous a head-end for storage farms (using wire speed encryption), as they will be PBX's, firewalls, routers - and with floating point added back in force, wonderful rendering engines and high performance computing machines.

And on equivalent workloads, we dropped our power draw - that is, we lowered the power required to do the same amount of work.

Again, tree hugging? Nope, customer hugging. Does that make Sun green? Not by a long shot, but it's a good step forward.

So our investments in eco-responsibility are starting to pay off - and the trends we were in front of a couple years ago (power efficiency and virtualization) have only become more relevant. Not to everyone, certainly - only those that care about minimizing waste. If you'd like a free trial Niagara 2 machine, just click here.

And much though I appreciate Kevin Maney's column, I think he's wrong in this opinion piece - to dismiss a focus on eco-responsibility as a fad. It's one thing for waist-conscious consumers to avoid high carb diets. It's another thing entirely when you're talking to a government seeking to avoid new coal fired power plants, a C-level executive committed to extreme efficiency, or the employee of a company whose CEO has just said, "We will be carbon neutral."

There is, and there will always be commercial opportunity in eliminating inefficiency - to be clear, that's our primary motivation. Paint it any color you want.

Along the way, if we reduce our carbon footprint, minimize our waste stream, and get crisper about our views on corporate social responsibility - does that have an impact on customers wanting to do business with us? Absolutely yes. I've seen it firsthand.

It also has an impact on our competitiveness as an employer. Do our current and potential employees care about the efficiency and responsibility of our business? At least as much as the chefs in Keller's kitchen, if not more.

(Interview with Dave Douglas here - Dave is leading the effort to reduce waste and environmental impact at Sun, and downstream in our customer base.)

Posted on 12:00AM Oct 17, 2007 | Comments[33]

Share this post  | | | | | |

Going Bollywood

I just got back from a series of customer meetings with technologists from the telecommunications, media and entertainment industry (they are, after all, converging on the same market, monetizing consumers). I was joined by Greg, and a number of folks from Sun, notably Jeff Bonwick and Bill Moore, along with Matt Ahrens, the co-inventors of the ZFS file system.

All three industries are largely underserved by innovation - their requirements and opportunities are vastly outstripping the rate at which the industry is innovating. (That's code for, "their technology budgets are growing.")

There were a number of choice anecdotes from the sessions, my two favorites being the following.

The first was relayed by the CTO of a major movie studio, who explained the value of very, very long term deep storage archives.

His company had recently pulled a more than fifty year old movie out of a salt mine - where it was stored on 35mm film color separations. Before you ask, vinyl film outlasts the standards that the industry produces (a point on which I've written previously), and salt mines are environmentally more stable than datacenters). They pulled the separations to remaster and reissue the film on DVD.

In so doing, the resulting movie actually increased in resolution for the viewer - the devices used to display the movie today (likely a laptop or high definition TV) offer better resolution than what was available to the original viewers (likely a 50's era TV or movie theater). The movie improved with time - the modern viewer saw the movie in greater detail than the original viewers (suggesting one should always store higher resolution data than the display you see in front of you).

After release of the digitally remastered version, the DVD rose to number 8 on the Amazon best seller list.

Cost of production? Near $0. The effort was nearly pure profit.

And they have a library of approximately 30,000 films.

That's a ton of value in a salt mine (if all the titles are as compelling, which is unlikely, but an interesting thought exercise, nonetheless).

The second anecdote relates to the advent of very high resolution digital cameras - the highest, and most desirable are currently known as "4K," offering 4096 x 3112 pixels (!) per frame - yielding cameras that spew 100's of megabytes of imagery per second).

The director of one feature length film wanted to keep all the footage from a soon to be introduced new movie. He wanted to preserve outtakes and all, for the eventual "Behind the Making of..." or "Director's Cut..." versions of the movie. The digital master for an average 4k film is roughly 9 Terabytes - that's for the version you and I would see in a theater.

But the total archive including out takes and secondary/tertiary angles (bits are a lot cheaper than film, so why not set up three or four cameras for every shot?) was roughly (drum roll please)... a PETABYTE (or a thousand terabytes, or roughly 500,000 iPods). Equivalent to about a million feet of 35mm film.

That's a lot of data. To be archived for... well, to the first anecdote, likely forever (like health records or airport surveillance). And now you know one of the (many) principle motivations behind a file system we built at Sun, ZFS. The focus of the ZFS team is both the scale, simplicity and quality of storage (on Mac OS X, BSD, Solaris and Linux).

This is a great overview of ZFS by Jeff and Bill - of why ZFS matters to the media and entertainment industry, and likely anyone concerned with high quality, high scale and high productivity, storage.

As I was told by Jeff (parroting a storage executive), there are only two types of disk drives in the industry.

Drives that have failed, and drives that are about to fail. And now you know what inspired ZFS, and what's inspiring interest in it.

Posted on 08:50PM Oct 07, 2007 | Comments[47]

Share this post  | | | | | |

All the Wood Behind One Arrow

I'm radically increasing Sun's focus on storage today.

Why? Because the market's only going to grow, for as long as we're on this earth, and I believe our talent and assets give us a big sustainable advantage - that we're planning on exploiting. Aggressively.

How? First, I'm going to be combining our Storage and Server product teams to create a new converged group at Sun known simply as our "Systems" team. The Systems team will focus on the evolution and convergence of computing, storage and networking systems. Talk to any datacenter adminstrator, and that's what they want to hear - they live in a world managing the (often idiosyncratic) interactions of that trinity (computing, storage and networking - and just wait until they're virtualized). We want to be in a position to innovate on their behalf, at the system level, beyond the boxes - across blades, racks, disk and tape.

So we'll still be strongly focused on being a multi-platform storage provider (just as our servers run multiple operating systems, and our operating system runs on every vendor's servers), but we're also going to start talking at a higher level to customers that see more standardization and integration in their future datacenters. That's not everyone, but it's definitely a trend we're going to accelerate (and again, that's what virtualization portends).

Now, why do I believe combining groups make sense? It's a recipe that works for us. We combined our high volume x64 server group with our traditionally high scale SPARC server group over a year ago - leveraging the volume skills of the former with the scaling skills of the latter. What did that collaboration yield? The highest scale x64 systems in the market. And a refreshed lineup of volume systems powered, interchangeably, by SPARC, Intel and AMD. We also combined our networking expertise to build the best general purpose blade platform below (known internally as C10 or Constellation 10 - 10 blades, vs. "C48," with 48), with integrated networking and seamless management. Can you tell where the server stops, and the storage starts? I can't (and in a virtualized world, it's not terribly important).

Secondly, as our servers clearly show, we're heading to a general purpose world - in which open and general purpose platforms will be the dominant drivers of growth, for us and the market broadly. The first general purpose storage system from Sun was Thumper (our x4500) - powered by an open source operating system (Solaris), and file system (ZFS - soon to be parallelized by Lustre, a recent acquisition from Cluster File Systems). Thumper rocketed to a $100,000,000 annual runrate within its first two full quarters of shipment (on a $13 billion dollar revenue base, that's hard to see, but we certainly took notice - at least one competitor did, too).

Combine these assets with some of our recent network innovations (like Magnum, the world's largest Infiniband switch - which is not the smallest variant we'll build, btw), the Crossbow community in Solaris - and it begins to look like we've got all the right ingredients to reinvent the datacenter.

So in this instance, I'm expecting our Systems team to be just as focused on standalone storage and networking - leveraging disk, tape (and all future removable media) - as they are on building great integrated systems (like the Constellation System, above, or our Thumper platform). I'm expecting to see more innovation, faster time to market, and a breadth of opportunities emerging from serving our current customers better than ever, while inviting new customers with a constant stream of high value innovation.

And before I end, I want to focus on one particular group, whose value only grows to Sun every day - our Tape and Archive business. From a market perspective, some data lasts forever - surveillance video, health and insurance records, trading histories, etc. In our view, the market for permanent data will only grow. Today, only tape can maintain the integrity of that data without electricity. And for the datacenters we serve, many are seeing the cost of electricity threatening to eclipse their hardware budgets (yes, I'm serious). For disk storage, over a decade, that's easy to see - just look at the power bill to run a SAN, mulitply it by a decade.

So while we've been selling very large libraries to the largest companies in the world, in mainframe and open system shops, we've also begun meeting a variety of startups and web 2.0 companies (some household names, even). With the need for very, very large pools of archived storage (when you collect user generated high definition videos or satellite imagery for planetary social networks, it's easy to find yourself with peta-scale archive problems).

Tape, with effective indexing and retrieval, represents the most economically responsible (that is, eco-responsible) archive platform for long term storage. Broadly speaking, tape (and in the future, other forms of removeable media) are a core part of Sun's archive plans. We think there's a ton of innovation we can bring to that market - now that we've done the basic integration (did you know a Solaris powered server is now embedded in our libraries - bringing new meaning to Free Inside!). As we converge high performance networking, virtualization and file system innovation - along with an overall Systems approach - our archive business will benefit just as much as our blades and rackmount systems. They are all, after all, members of the Systems family.

So like I said in the opening, I'm dramatically increasing Sun's focus on storage today. By bringing to bear the talent and assets we have from across Sun to ensure our success. From where I sit, we have the right leaders and assets, and the right target in front of us.

Now's a great time to put all the wood behind one arrowhead.

Posted on 10:30AM Oct 01, 2007 | Comments[46]

Share this post  | | | | | |