Coffee, Keynotes and Linux on Intel vs. Solaris on Opteron

Webcast of a keynote I just delivered at the Open Source Business Conference (OSBC) can be found here. I did have a fair amount of coffee before the speech, which someone in the audience suggested was evident in the pace at which the speech was delivered. Listen for yourself.

There were quite a few startups in the audience - and given that we're beginning to turn our focus to recruiting those startups back to Sun, I thought I'd live up to my (now delinquent) commitment to publish an email I received from Marc Andreesen about his startup's experience with "Linux on Intel" vs. Solaris. Here you go:

-------- Original Message --------

Subject: followup
Date: Thu, 02 Feb 2006 15:51:28 -0800
From: Marc Andreessen < >
Organization: Ning Inc.
To: Jonathan Schwartz < >, Anil Gadre < >

Hi guys -- below is the writeup of the data I spoke about on stage yesterday.

We (Ning) are fine with you all using this however you want.

Marc

Ning Server Platform Analysis

The following data is based on an analysis we did in support of the implementation of our production environment. Ning had deployed a beta system based on a commercial Linux distribution running on whitebox (aka local system integrator PC clone) AMD Opteron servers. This infrastructure is hosted in our own cage at a commercial co-location facility. We pay a monthly fee based on the total sqft of the cage as well as the total power delivered to the cage (measured in amps).

The space and power costs are blended averages for our area (San Jose, CA):

Monthly space cost: $27.00 per sqft
Monthly power cost: $17.00 per amp

A 4-post rack (or cabinet) occupies approximately 20sqft. Each rack/cabinet has approximately 40 rack units (RU; 1RU = 1.75") of useable space. Thus:

Monthly cost per rack: 20 sqft * $27/sqft = $540
Monthly cost per RU: $540/40 = $13.50

For purposes of our analysis, let's assume all servers are of equal cost:

Server price: $3,000.00

Next we need to consider the operating system for the server, so we need to include the purchase price plus any maintenance fees. We compared a commercial Linux distribution (enterprise class) vs Solaris:

Commercial Linux annual subscription fee: $900.00
Commercial Linux per-incident charge: $380.00
Total: $1,280.00

Sun Solaris subscription fee: $120.00
Sun Solaris per-incident fee: $0.00
Total: $120.00

Given that we are in a hosted environment which imposes certain constraints (total space and power available), the acquisition cost is only a part of the equation. As Paul Harvey would say "this is the rest of the story".

Our co-location provider imposes a per-rack limit as to how much power they will provide. In our case this is 60A. The rule of thumb is to only load a circuit to 80% of the total capacity. Thus:

60A * 80% = 48A useable per rack

Now we need to determine how many systems we can fit in a rack. We know we have 40RU available and 48A of power available per rack. But how much power do the systems require? Let's see:

Sun x2100 (model 175; dual-core Opteron): 1A
Whitebox AMD (2x AMD Opteron 248): 2A
Intel dual Xeon: >3A

To determine the maximum number of systems we can deploy per rack:

Intel: 48A per rack / 3A per system = 16 systems per rack
Whitebox AMD: 48A per rack / 2A per system = 24 systems per rack
Sun x2100: 48A per rack / 1A per system = 48 systems per rack.

Given there are only 40 RU in a rack, the real limit for the Sun systems is 40 systems per rack. Next we need to determine the monthly cost of operating the systems:

60A per rack * $17 per A per month = $1,020.00 per month
Monthly rack cost: $540.00

Intel (@ 16 systems per rack):
power: $1,020 / 16 = $63.75 per month
space: $540 / 16 = $33.75 per month
total: $97.50 per month

whitebox AMD (@ 24 systems per rack):
power: $1,020 / 24 = $42.50 per month
space: $540 / 24 = $22.50 per month
total: $65.00 per month

Sun AMD: power: $1,020 / 40 = $25.50 per month
space: $540 / 40 = $13.50 per month
total: $39.00 per month

Looking at a straight line analysis of the combined costs of server + OS + space + power over 36 months we end up with:

Intel + commercial Linux:
$3,000 + (3 * $1,280) + (36 * $97.50) = $10,350.00 over 3 years

whitebox AMD + commercial Linux:
$3,000 + (3 * $1,280) + (36 * $65.00) = $9,180.00 over 3 years

whitebox AMD + Solaris:
$3,000 + (3 * $120) + (36 * $65.00) = $5,700.00

Sun AMD + Solaris:
$3,000 + (3 * $120) + (36 * $39.00) = $4,764.00

Based on this analysis the Sun solution is less than half the cost of running Linux on Intel hardware.

A key consideration: data center space is a limited commodity over which we have minimal control. We can "reserve" space for expansion, but this requires additional expenses. A solution that allows us to maximize the space we purchase is a significant advantage, as this allows for much more cost effective growth.

------------------

ps. note comments have been turned on.

pps. And to that point, The Register just published an excellent analysis - and given the increasing scrutiny under which Gartner and IDC now find themselves, it'd probably help their credibility if they, like the financial analyst community, started disclosing revenues they receive from the vendors they cover. At their scale, vs. the smaller boutiques, I think it's going to become an imperative.

And personally, it may end up being a determinant of whether we're willing to do business with such firms. Transparency's a good thing.

Posted on 10:00PM Feb 17, 2006 | Comments[22]

Share this post  | | | | | |