What happens downstream?: Scott Mattoon's blog

Over the past few months I had heard exclamations of amazement regarding a storied new data center in the Nevada desert called SuperNAP. I was a bit skeptical of the superlatives about scale and efficiency that embellished these stories. My skepticism turned to exuberance last week when I joined a group of architects from Sun for a tour.  The goal of our tour of this Mega Data Center was to see first hand the state of the art as implemented by Switch Communications, where Sun operates it's cloud computing business.

Switch, and it's customers, which include several operating units within Sun, are beneficiaries of the collapse of Enron. The former utility giant had designs on trading network bandwidth using models similar to their energy trading systems. When Enron's flimsy financial structure gave way, their financial backers and the U.S. government stepped in to auction these assets. Switch CEO Rob Roy was the only one that showed up at the auction block.  In an uncanny twist of fate, he managed to side step what could have been a formidable bidding war to control this hub of communication that is unparalleled in North America.

Here are the vital stats that only begin to describe the phenomenal facility that Switch has managed to assemble:

• 407K square feet of data center floor space
• 100 Mw of power provisioned from two separate power grids
• Fully redundant power to every rack, backed by N+2 power distribution across the facility
• Enough cooling and power density to run at 1,500 watts per square foot (that's 10x the industry average 150 watts).
• 27 National network carries

This describes the capacity of the SuperNAP, which is just one of the eight facilities operated by Switch within a 6-7 mile radius in a no-fly zone south of Las Vegas.

Sun to Reveal Cloud Plans Tomorrow

Some details of Sun's Cloud Computing business will be revealed tomorrow (March 18) in New York at the CommunityOne East event.

---

 Additional Resources

• The Register article on Sun's cloud in the SuperNAP
• Another article in The Reg with pictures of the SuperNAP
• CommunityOne site
• Cloud Computing at sun.com
  

In the maelstrom of preoccupations that kept me awake last night, self-service in the cloud was a strangely prominent theme. A sad commentary on my slumber time, I know, but it was eerily coincident with news of OpenSolaris freed from a special registration process - when I woke this morning I found this announcement in my Inbox:

News Flash for Our OpenSolaris 2008.11 on Amazon EC2 Users!

We are happy to inform you that the latest OpenSolaris 2008.11 Base AMIs on Amazon EC2 in the US and Europe are now available to you and your users with no registration required! Please stay tuned for more OpenSolaris 2008.11 AMI stacks coming soon for you to quickly access. The registration process for pre-OpenSolaris 2008.11 AMIs is still in effect.

For your reference, here are the AMI IDs:
OpenSolaris 2008.11 (US) 32-bit AMI: ami-7db75014
OpenSolaris 2008.11 (Europe) 32-bit AMI: ami-6c1c3418

To read about what's new in OpenSolaris 2008.11, please visit the OpenSolaris Web site.

OpenSolaris on EC2 had been available for months, but it was cloistered behind a registration process that involved waiting for a human to get back to you with approval of your request. But no more. Now OpenSolaris on EC2 is a first class citizen with all the other *nix and Windows distros, available self-service to anyone with an AWS account.

Amazon.com released on Thursday a web GUI for AWS: the AWS Management Console.

It's pretty slick, although still clearly a beta - some rough edges around navigation, and EC2 support only (no S3, SQS, Cloudfront, etc. yet). If you're running lots of diverse AMI's, this single view is a great decision making tool. Once they add Tagging (Label and group Amazon EC2 resources with your own custom metadata,) companies will be able to quickly see opportunities for optimization and grouping of operations, etc.

The AWS announcement probably hurts RightScale, but this is their positive spin on it.

This news from Amazon raises the bar on ease of use and the relative importance of self service in the cloud market. Once they've experienced the AWS Management Console or RightScale's dashboard, many enterprises will want their own private clouds to be built with clean UI's and web2.0 ease of use too. While a quality programmatic interface is vital to the scaling needs of cloud users, a simple and useful set of GUI controls is equally important for those primarily seeking the self service benefits of cloud computing.

Entering the market without a comparable console will be a disadvantage for upstart public clouds, but this new prerequisite for clouds also creates an opportunity to up the ante further.

A couple opportunities for value add come to mind:

1. Social networking integration - one clear opportunity is to enhance cloud console functionality with existing social networks. Imagine a 37Signals interface that let's you plot the sequence of operations required to upgrade your complex app running across 1000 instances in Basecamp, a message to Twitter followers when specific operations complete, and a Livejournal post summarizing the status of the upgrade after completion - a social RESTful SOA for datacenter operations if you will.
   
2. Modeling and Design tools - I expect companies like Smugmug wont use the AWS Dashboard and Control Panel features as is, but would use a GUI that could help model different deployment patterns and quickly sort through sequencing and dependency issues and compare performance characteristics of alternative architectures. (If you haven't read how Smugmug uses EC2 for their Skynet, check out Don MacAskill's post on it. A modeling tool might give Don a way to compare an SQS implementation with his home grown solution, and make a decision informed with real financial and performance inputs.)

---

Other Cloud Computing news for the week ending 10-Jan-09:

• Sun Acquires Q-Layer
• Beta release of Cloud Foundry for deploying Java applications in EC2

In a post on his Wisdom of Clouds blog last week, James Urquhart proposed five phases of Cloud Maturity.

I fully concur with the phases of this maturity model, and with Urquhart's assessment of the current state of enterprise IT on the scale, i.e., most have consolidated, some have abstracted, fewer have automated, and only a handful are experimenting with metering and self service. None have achieved open Internet-based Market capability, yet.

This maturity model is useful, but I can't say I find the first four phase names, order, or definitions to be novel in any way - I've been writing these exact same phases on client whiteboards for about three years.

Maturity Model (MM) Hopping Disallowed

The fifth phase (Market) however, is new and insightful, and it reshapes the preceding four in interesting ways. In other words, if you're on a path to reach Market maturity then certain capabilities must be addressed in preceding phases that weren't necessarily required in preceding models that stopped at Utility. For example, elements of service level management must be addressed in the Automation and Utility phases that were not essential prior to the proposed model. In a pre-Market maturity model enterprise IT could deliver automatic provisioning and pay-for-use to their customer without demonstrating compliance to specific service levels. That won't fly in a price arbitraged cloud Market, so these capabilities important to the Market phase must be built in the preceding phases that correspond to the capability. Maturity models are only useful if the phases inherit all the preceding phase capabilities. If additional Automation capabilities are required to achieve Market capability, then Automation was not really achieved at phase three.

What of Elasticity?

I'm not convinced that this is a comprehensive maturity model, and whether we can fit clouds, both public and private, into one vector such as this. For instance, where does Elasticity fit? Auto-scaling relies on Automation, but would we require it of any environment claiming to be Automated? The pay-for-use implication of Utility does not necessarily mean resources are acquired and released in conjunction with use - metering is not intrinsic to provisioning, and vice versa. Whereas Elasticity, I submit, implies growing and shrinking resources synchronously with customer demand. So, does Elasticity warrant it's own phase in the Cloud Maturity Model? What are the implications of this model for private clouds? Does a private cloud ever reach Market phase maturity?

MM Drafted, Now Where's the Magic Quadrant?

In any case, the proposal of a Cloud Maturity Model is a valuable step in the evolution of cloud computing, and the Market apex of the model seems like a reasonable goal. And there is an army of consultants forming to help enterprises address the climb.

Ian Kallen over at Technorati wrote a nice post about the cloud computing ontology and the subtleties of Infrastructure as a Service (IaaS).  I'm glad to know he's still working on the hard problems there at the blogsphere search engine after their recent cost cutting measures.  As he has said to me previously, he writes, "What I foresee is that the first vendor to embrace and commoditize a standard interface for infrastructure management changes the game."  I think he's right, particularly in his prediction that these standards will enable a market place in which workloads can be moved from cloud to cloud according to price, capacity, and feature criteria.  A few companies are jockeying for the pole position in the race to provide the arbitrage for this meta cloud that Ian envisions.  Rightscale is perhaps in the best spot for that right now.  But who's going to set the standards for interfacing with clouds.  It's still pretty early in the game, but there's no question that Amazon has a good leg up with the AWS API's, which are further butressed by Eucalytus's emulation of those interfaces in their open source Xen based IaaS stack.  Meanwhile, Ruv over at Enomaly is fostering a Unified Cloud Interface (UCI) standard to be submitted to the IETF next year.  Conspicuously, it appears that Amazon is involved in neither the Eucalyptus nor UCI standards efforts.  Meanwhile, Rightscale is working closely with Rich Wolski's Eucalytus team, and both of these standard bearers are advising on Sun's Network.com model.  It will be interesting to witness the evolution of agreed upon standard interfaces in the presence of the defacto standard that is AWS.  Until there's a cleaner and/or cheaper way to develop on OpenSolaris in the cloud, I'll continue to write to the AWS interfaces to launch and extend instances of OpenSolaris on EC2.

With California's first real rain of the season forecast for Friday, it's time to take stock of another weather system affecting the West (and other places connected to the Internet): cloud computing. Summer 2008 saw a downpour of cloud offerings. We've witnessed whole business ventures billow up and evaporate on the cost and agility promises of cloud computing. While storm systems continue to build off the Pacific coast, the long range forecast is for an unstable system to dump on the landscape for a couple quarters before a high pressure cell clears the air. Despite the instability (and, as if this hackneyed weather metaphor needed more abuse,) it don't take a weatherman to know which way the wind blows - adoption of cloud computing will continue to rise.

The storm of demand is fed by startups

In the climate of "fail fast" startups, the appeal of cloud computing as a means of containing cost and improving productivity during the fragile stages of germination is obvious: skip over the infrastructure "muck" and keep your costs tied to your growth.  "Fixed costs are taboo" is the principle directive from many VC's investing in Web startups - put the employees on a sustenance + equity compensation plan, and, for God's sake, don't spend anything on compute infrastructure you don't absolutely need. 

A major front accumulates in the enterprise

But what about the enterprise?   Enterprises differ from startups in how they evaluate risk and how they spend on IT services.  In the enterprise computing landscape, risk averse business leaders are concerned with reliability and control over their services and their data.  Control is not one of the attributes primarily associated with cloud computing, security risks are a major barrier to enterprise adoption, and 99.9% availability is often not good enough for business critical and mission critical services.  Further, and for the time being, fixed costs are already baked into the equation in most IT business models.  In fact, most large enterprises treat IT as one big fixed cost, which it parcels out to business units according to some "fair share" cost allocation scheme.  

Rarely are the business units of a large enterprise satisfied with their cost allocation, let alone the IT services it pays for, but they're captives of myriad barriers like technical complexity, regulatory compliance, data provenance, spending constraints, and limited organizational imagination. One or more of these factors are impediments to any serious consideration of public cloud computing for existing enterprise IT needs.  Business consumers of enterprise IT would like to have a secure, reliable, pay-as-you-go public utility service customized to their unique needs, but such a service does not exist. They'd use a public cloud for the cost and agility benefits if they perceived the risks to be acceptable, if their complex needs could be managed, and if they weren't already paying for IT services with funny money. Public cloud service providers are working on the availability concerns by committing SLA's, and certain security concerns by providing VPN's, but the reality is that the major refactoring of their huge software investments required in order to work in the public cloud will drive many enterprises to build their own cloud-like private infrastructure instead. In fact, any large enterprise is probably already doing this - the practice of building cloud-like infrastructure has been evolving for years under the cover of consolidation and virtualization initiatives.

High clouds are approaching

If predictions of mass consolidation onto public clouds proves true, then enterprise IT might be a dying breed of industrial infrastructure.  But just as it took electric power distribution decades to transition from local DC power generation to utility grids, traditional data center bound enterprise IT won't die easily.  Enterprises will strive for the  kind of efficiency that propels public cloud adoption by continuing investment in consoldiation and virtualization in their own data centers.  But consolidation and virtualization alone does not a cloud make, and will leave the consumers of enterprise IT with the same bucket of bits, still wanting for a cloud.  So when does one confer cloud status on a consolidated, virtualized environment?  The following simple criteria gives a pretty decent working definition:

1. When it delivers IT resources as a metered service (rather than an asset or share of an asset,) and
2. When all it's services can be accessed programatically.

Yes, the implication here is that cloudhood can be achieved in a private implementation. (This potentially violates certain tenuous claims that cloud services must be provided offsite and by a 3rd party, and that clouds are accessed over the Internet, but we'll not constrain the discussion with those seemingly arbitrary distinctions.)

Of course, the devil is in the details, so the next posts in this series will address more nuanced definitions of cloud computing. In particular, we'll examine the attributes of cloud computing as put forth by other aficionados, and what value and relevance these attributes have to business consumers of enterprise IT.

---

Related reading