Taylor's Take on Sun Storage
My storage team and I focus on three of the most important aspects in any industry: customers, competitors and market trends. There is insight to gain and share in this role, so here is our take on Sun and Storage - Taylor Allis
Open Storage: Size & Growth
Both EMC and IBM have chimed in on what their thoughts are on Open Storage's future. Tony's analogy made me think about a simple way to explain what I think Open Storage's future will be like:
Open Storage & Builders Square: Let's rewind to the 1990's. At that time I knew very little about home improvement. In my mind, the only option was to save up to hire an expensive contractor to update my kitchen/bathroom/bedroom, etc. But then a new store opened up close to my home - Builders Square. That shop was subsequently replaced by a new store called Home Depot. I was intimidated when I first walked in - there were only serious contractors around and I couldn't believe the amount of stuff they had.
Well the time came to update our bathroom - and as I was looking up contractors and plumbers in the phone book my wife said, "why don't you see how much it would cost to do it yourself by going to that new home store down the street?" Fast forward to today and the people at our local Home Depot and Lowe's know me by name. I have installed bathrooms, kitchens and built our basement. I had help from store employees, instruction guides and affordable house "components" to do everything. And when I walk into a store today; I see families make up the majority of customers - not contractors. The simple fact is this - I could not have done what I wanted to do with my house if it wasn't for the introduction of these new DIY home stores. Open storage will help users do more with less - and reference architectures, solution blueprints, online communities and consultant services will help customers deploy open storage.
Open Storage & Stay-at-Home Moms: Let's go back to the 90's. I was also an early Web developer back then. In fact, a quick Google search of my name will find a question I posted to a developer's help desk in 1996 where I ask, "Do you know when Netscape 2.0 for mac will be able to pick up Java applets?" Pretty funny; and a testament to the longevity to open standards like HTML. But my point is this - back then my wife was working in consumer PR and had absolutely no idea what I was doing for a living in high tech. The Internet? The WWW?
Today my wife has taken on the full-time job of raising our kids. And I am happy to report that she is looking into starting her own blog - and she is looking at a blog from another stay-at-home mom as an example. Now, if I told my wife she would be "developing" her own WWW journal back in the 90's she would have thought me crazy (which happens often) or the task impossible. But Internet tools have evolved in away that let's anyone self-publish on the web for little to no cost.
I believe open storage will find its way in early markets; and the support resources and tools around it will evolve over time, bringing open storage to more traditional, mature markets.
Open Storage Size & Growth: (From the Open Storage Adoption White Paper)
How do we know this short of getting out our crystal ball? Well, we don't - we need to estimate - and this is how we did it: We took multiple IDC forecasts and rolled them into an internal Sun model - forecasts which included revenue from industry standard storage (JBOD, SSD, etc.) and open source software. Then we made our own assumptions on how the market will behave - for example, we believe the fastest open storage adoption rates will appear in the entry and midrange NAS, unified storage (iSCSI, FC, etc.) and object-based storage markets. We assume adoption will be slower in more traditional, higher-end markets. So, this is NOT an IDC model - but an internal Sun model based on IDC and industry data (to be clear). And this is an industry/market revenue estimate - it includes ALL vendor revenue from industry-standard hardware + open source software used in ANY storage system and does not give ANY info on Sun revenue or performance (to be clear again).
So...we estimate open storage products and services will represent just under 12% percent of the total storage market in 2011. With IDC estimating the total storage market (hardware, software and services) generating approximately $90B in 2011, the open storage portion could be just over $10.6 billion.
We also predict that open storage will represent more than 20% of the external disk market by 2011. At $5.2 billion, hardware represents the largest portion of the open storage market. Tape storage and storage networking products are not included in the open storage forecast (although one could argue that LTO is industry standard hardware).
Next Blog: Sun Open Storage vs. Other Vendor Efforts
Posted at 05:00PM Jun 12, 2008 by Taylor Allis in Storage Intelligence | Comments[1]
Open Storage: Early Markets
As promised, below is a second White Paper on Open Storage. This second one addresses Open Storage market drivers and growth and is titled Open Storage Adoption. It can be downloaded here:
Again, I would like to thank Bruce Norikane, our Sr. Analyst, for
his help as well as our market research manager, Chris Ilg, for his forecasting work. And again, I'll use this blog to post the CliffsNotes for those short on time. Below I will cover the need for a new storage architectures and early target markets. In subsequent blogs I'll cover the Open Storage future market forecast, other vendor initiatives and customer case studies - early adopters who have used Open Storage to solve their critical business needs...
The Need for a New Storage Architecture
Bruce, mentioned above, made a profound statement during our Open Storage planning that ended up in the White Paper. He said, "Google and Amazon would not exist if they hadn’t built their own storage infrastructures." They certainly wouldn't exist in their current state. When they started, traditional storage architectures were too expensive and inflexible to support the business model they had in mind. So what did they do? They had to buy commodity components and developed their own software like the Google File System (GFS).
Certainly not everyone can build their own file system today. But the requirements that drove Google to build their own file system have done nothing but increased. Consider the following facts:
- IDC’s report “The Expanding Digital Universe" states that new digital data will grow 6X between 2006 and 2010 (Emerging market data is growing up to 40% faster)
- 70% of this data will be created by individuals (me, on this blog for example)
- Enterprises will be responsible for storing 85% of this new digital data (Sun's IT dept has to manage the data I create)
- YouTube was founded in February 2005, and by May 2006, 50,000 videos were being added every day
- A YouTube search today can return about 72.1 million videos and 2.89 million user channels
A new, more economic and scalable storage architecture is desperately needed - enter Open Storage...
Open Storage Growth Markets
Open Storage can (and will) compete with traditional storage architectures. But Open Storage won't "take over the world" overnight. Most likely the data center mix of open storage architectures vs. closed storage architectures will change over time and vary data center to data center (if history is our guide). But what markets will adopt sooner? What are the Open Storage "sweet spots"?
Web 2.0: I count Web 2.0 apps as applications delivered via the Web. Apps like blogs, wikis, podcasts, RSS feeds, mashups, and social-networking sites like MySpace, Facebook or SmugMug. Consider this:
- Technorati stated in March 2008 that there were 112.8 million blogs
- Bloggers update their content with 1.6 million posts EVERY day (or more than 18 updates per second!)
- There are currently 3,115 mashup applications on the Internet
And Web 2.0 apps are not just for up starts - Forrester surveyed 2,200 IT decision makers from traditional enterprises and found that 33% were planning on investing in Web 2.0 applications. Web 2.0 storage requirements differ from traditional storage requirements as well. They need massively scalable but low-cost systems. Web 2.0 users are even willing to trade high availability for lower costs. Everyone needs high scalability at lower costs - but the need in the Web 2.0 space is acute. Thus, Web 2.0 will be the key driver for Open Storage architectures.
HPC Storage: IDC estimates that HPC storage systems added about $3.9 billion to the 2006 server revenue total and will undergo faster annual growth than HPC servers. Maximizing I/O bandwidth and minimizing latency while scaling storage capacity is the top priority for HPC storage users. Because of this, data locality is an issue for many HPC implementations. What's data locality? HPC services provider Instrumental, Inc.explains:
Data locality is a big issue in some architectures. Sometimes you need to know where data is in memory to get the best performance. Locality issues are compounded by the enormous amount of software ‘in the middle (OS, file system, volume management, failover, host bus adapters, and so on)’.
To manage issues such as data locality, an open storage architecture is needed. The one thing that HPC storage deployments have in common is that they are all custom built. HPC users need direct access to their storage components and software along with the flexibility to swap components and customize software to optimize their storage. This is difficult to do with closed storage systems.
Additionally, parallel, shared or clustered file systems that leverage global namespace technologies are used in most HPC storage environments. This includes the HPC open source file systems Sun offers - like Lustre. In fact, Lustre is used in 15% of the top 500 supercomputers in the world and in six of the top 10 supercomputers.
Lastly, an additional top storage requirement in HPC is Hierarchal Storage Management (HSM) software (moving data from disk to tape). 
Why? Just look at the massive amounts of data HPC applications generate. The San Diego Supercomputer Center states their earthquake simulations alone generate 47TB every week! By 2011, they expect archived data to grow to more than 100PB. HPC centers must leverage the economics of tape to store such massive amounts of data. Sun offers tape as well as open-source HSM software for disk-to-tape data migration - Sun's Storage Archive Manager (SAM) software.
To see the real-world benefits an open storage architecture can offer HPC customers, see the Texas Advanced Computing Center (TACC) implementation of Sun Constellation - aka Ranger.
Server Virtualization: Open storage introduces more flexibility and consolidation benefits to the server-virtualization market. This added functionality can be realized in two ways:
- By running open storage software inside a virtual machine (VM)
- By running any vendor’s storage software on an open storage server
Storage users can now consolidate three servers and a storage appliance onto a single server. In a closed architecture, storage software cannot be separated from the hardware.
In the second scenario, users can use an open storage server, such as the Sun Fire X4500, as a storage target or shared appliance. What’s unique is that users can repurpose their storage appliance as their needs change. For example, customers can repurpose the same Sun Fire X4500 into a NAS device, a Virtual Tape Library (VTL) or a data replication appliance without buying more hardware. Now that's investment protection!
In the following diagram, a customer has taken a Sun Fire X4500 server running Linux-based VTL software and has repurposed it into a remote replication appliance by leveraging server virtualization and open source Sun StorageTek Availability Suite software.
Sever Virtualization and Open Storage can deliver better investment protection and significant cost and consolidation advantages ...
Next Blog...
What we predict the size and growth of the Open Storage will be
Posted at 04:06PM Jun 10, 2008 by Taylor Allis in Storage Intelligence | Comments[1]
EMC's Big Bet
So EMC just announced an all-cash acquisition of a small startup as well as a new "Cloud Infrastructure and Services" division. And while they do A LOT of acquisitions - EMC's Chuck Hollis doesn't think this is your everyday acquisition.
I agree with him...
As we know, Sun and others in the industry believe the future will hold utility-like
"Information Data Centers" - where customers can buy/rent/lease CPUs and GBs. An emerging buzz term for this is Cloud Computing or Storage. Sun is building next-gen technology (another buzz word, thank you) that can be leveraged by these emerging mega-data centers - for more on this, see Greg Papadopoulos' SAS 2008 Keynote on building network-scale computing.
So, while Sun is providing the technology for future "Information
Data Centers" - EMC is trying to become one.
Let's look at what EMC has been up to these past couple years in this space (dates are close but not exact):
- 2/21/08: EMC announces new Cloud Infrastructure and Services
division to be run by ex-Microsoft exec Paul Maritz
- 2/21/08: EMC announces all-cash acquisition of Pi
Corporation, developer of Personal Information Management
Technology
- 1/22/08: EMC announces EMC Fortress SaaS infrastructure & Mozy online backup service
- 10/1/07: EMC acquires online storage provider Berkeley Data Systems (Mozy) for $76M
- 11/1/06: EMC acquires online storage provider Avamar
Technologies for $165M (Avamar is a de-dup technology, but it sits
behind online storage service offerings like Arsenal's)
- 6/30/06: EMC buys RSA Security for $2.1B - they have
security and authentication software, key in providing storage services
So, EMC's big bet looks like it is to become a "Utility Information Data Center" (for lack of a better term).
More power to them if they can pull it off. And while EMC may have already made its investments in storage infrastructure, software, server virtualization and security for this - they are going to need some pretty innovative server and networking technology to pull it off. I think I may know a company ;-)
Posted at 01:11PM Feb 22, 2008 by Taylor Allis in Storage Intelligence | Comments[0]
Server Virtualization's Impact on Storage
Today Sun announced it entered
into a stock purchase agreement to acquire innotek.
In a nutshell, innotek develops VirtualBox, an open source desktop
virtualization platform. On the server virtualization side, Sun has
had an alliance with VMware for 2.5 years now, and Sun has also
delivered its own Sun's xVM
platform
with some pretty unique self-healing and management capabilities. (Get OpenxVM here). innotek will add to
Sun's xVM platform, it allows laptops or desktops running Windows, Mac,
Linux or Solaris to run multiple OSes side-by-side. Cool stuff. To
learn more on innotek see the Weblogs of Steve
Wilson and Joe
Bonasera or download
VirtualBox here.
So all this server virtualization talk got us to thinking...
What is server virtualization's impact on Storage?
I admit, my team and I have discussed this and we believe that the true impact is yet to be determined - this is new stuff after all. This is what we do know:
Server Virtualization's link to storage: The most important link b/w server virtualization and storage is application mobility. In server virtualization, customers can ultimately move applications from box-to-box and system-to-system much easier than ever before. But as applications move to different systems, customers need to maintain the links to storage. If customers have to maintain links to storage as they move their applications around, it would make sense for virtualized environments to leverage networked storage - maintaining the links through the network.
Which storage network benefits the most from Server
Virtualization? SAN, iSCSI or NAS? All of the prominent storage networks, FC, iSCSI and NAS, are fighting
for virtual server market share. After reading several IDC briefs, all
three show signs of growth. In one brief, IDC claimed that Server
Virtualization contributed to the increase of industry FC SAN sales in
2007. IDC also predicts that ~50% of virtualized servers will be
attached to iSCSI in the future - citing that server admins are
generally more comfortable with IP-based storage and networks. NAS
vendors are also pushing file server networking to support virtual
servers.
Server Virtualization has the potential to dramatically impact Storage customer requirements: Server Virtualization is still emerging and maturing, but it will impact storage purchase patterns. This will (or should) impact how storage is marketed and sold and will most likely disrupt analyst's long-term forecasts of the storage market. The amount and type of impact to vendors and customers should be interesting to watch.
I'd love to hear any comments on how others think server virtualization will impact storage....
Posted at 03:36PM Feb 12, 2008 by Taylor Allis in Storage Intelligence | Comments[3]
Top 10 Storage Technology Trends
1. Open Storage Platform (aka general purpose storage, open source storage): Trend #1 is a term we coined, so it may not sound familiar. It is a combination of market trends as well as a direction Sun is taking with its newer products. The concept of a common platform is not new – several vendors have tried to build one platform that can run multiple storage applications, saving users time and money. “Open” is a relatively new concept for storage, but not for software or servers. There are generally three components that make up an Open Storage Platform:
- General Purpose Components: General-purpose servers, processors, storage and operating systems are now being deployed in enterprise-class storage devices. Previously, storage vendors had built proprietary operating systems, ASIC chips and other custom-built components. As commodity chips, components and software have matured, storage vendors are now using general purpose components in their systems. The cost savings are significant - however, most vendors' prices remain the same (contributing to vendor margin). Sun's philosophy is to pass these savings to the consumer. Enterprise-class systems based on general purpose components give customers higher-value systems at a fraction of the cost (see graphic below). They are also much more flexible, as they can be re-purposed for other uses.
- OS/File System Storage Services: Traditional appliances charge customers extra in software licensing costs for data management services like administration, replication or volume management. What if this functionality came already embedded in the storage system itself? Sun's newest file system, ZFS, has started to incorporate these services at the File System level. ZFS deploys point-in-time-copy, volume management, administration and data integrity features like Copy-on-write and RAID. Storage services deployed at the OS-level have several benefits, including efficiency, performance, reliability and affordability (no more licenses for extra software needed). And before we think this is a new concept, mainframe has been doing it for years. The new concept is doing this in the open systems space – and Sun Solaris ZFS is leading the charge.
- Open Source Storage Software: The ability to download software, test it and add features to it is critical to developers building new applications – we’ve seen this in the OS market. But what about storage? This has been an investment area of Sun’s - Sun actually offers one of the most complete open source storage software stacks (from protocols to drivers to data management software). Developers can build their own storage solutions and sell them by leveraging Sun’s open source software (see the Nexenta Storage Appliance for a perfect example). Open source has an added benefit to customers with in-house development resources - customers can deploy new software features by searching for it in the open source community or even developing it themselves (why wait for vendor roadmaps?). This is not possible with a proprietary appliance.
- Midrange Disk = $10.39/GB
- Open Storage Platform = $1.50/GB
(7x less than Midrange and 13x less than
- Server Virtualization: The ability to host Windows, Linux and Solaris operating systems on one platform has literally taken the market by storm. A huge boon to IT managers and developers alike. VMWare led the charge with some start-ups and open source initiatives in tow. On October 5, 2007 Sun entered the race with Sun xVM. (And if virtualization has hit the systems world, expect to see it hit the desktop market soon…)
- File
Virtualization (a.k.a Clustering,
Global Namespace, Unified Namespace, NAS Virtualization): A few years ago we were talking about
Grid Storage – a bunch of storage nodes acting as one, single system. File virtualization is shaping up to be the
technology that will take us there. In
2005 & 2006 we saw EMC buy Rainfinity, NetApp buy Spinnaker and Brocade buy NuView. In recent years we we have seen EMC announce Rainfinity,
NetApp announce a rough start to Data OnTap GX (based on Spinnaker technology)
and file virtualization start ups Ibrix, Isilon and LeftHand Networks continue
to grow. Expect this market to heat up,
and consolidate as start ups are either bought, or simply run out of
runway.
- Virtual Tape: A disk system that emulates a tape library. Virtual Tape has been in the mainframe market for years, and Sun is the market leader here. Virtual Tape Libraries (VTL) in the open systems market are relatively new – disk prices have eroded to a point where backup administrators can now take advantage of disk’s access speeds without replacing their exiting backup infrastructure. IDC pegs this market as small, but growing at 16.2% (2006-2011 CAGR).
- Virtual Disk (aka Storage Virtualization): A disk system that aggregates 3rd-party disk and offers disk partitioning. A necessity for anyone that needs to consolidate their environment and/or make data migration between independent systems easier. Virtual Disk Systems include IBM’s San Volume Controller (SVC), FalconStor’s IPStor and Sun’s award-winning StorageTek 9000 arrays.
3. Thin Provisioning: Better system utilization is the name of
the game. Most admins know that the
utilization rates on their disk systems are not where they need to be. Thin Provisioning allows admins to allocate
or provision space to specific applications, making full use of their system’s
capacity. 3PAR spearheaded open systems
Thin Provisioning and NetApp offers it as a part of Data OnTap. Sun announced Thin Provisioning on its
StorageTek 9990V system in May – meaning consumers can have the world’s fastest
enterprise array, Virtual Disk AND Thin Provisioning all on
one platform.
Pretty cool…
4. Data Deduplication
(aka De-dup, Single-instance storage): In a world where there is more data coming
into a company than can possibly be managed – data compression ratios ranging
from 10:1 to 50:1 sound pretty darn nice
(See how De-dup works here). Data
Domain, Diligent, FalconStor and other upstarts get credit for bringing this
new technology to market and larger vendors are quickly following suite. De-dup is still emerging, can have
performance issues and does not work perfectly for every application – but
economics dictate its worth consumers investigating where it can work for them.
There are two emerging de-dup architectures: “Inline” – where the de-dup magic happens in real-time, as data comes into the system, as found in Diligent's ProtecTIER appliance. Or “Post-Processing” where the magic happens as a secondary process after the backup job, as found in FalconStor’s Single Instance Repository (SIR) software. Both have their pros and cons, and deciding which approach to use depends on balancing your performance vs. complexity needs. For the record, Sun sells both….
5. Data Encryption: One need only read the horror stories of lost
tape and disk drives to see the importance of data encryption. While it has been around for a while – the
need has never been greater. Growing
storage capacity has caused another problem – one can store a lot of personnel
records on a single cartridge or drive.
In an age of identify theft, losing one storage device can put a company
out of business. The new trend is not
how to encrypt, but where to encrypt… On the host server? On an appliance in the network? In the storage device itself? Decru (since bought by NetApp) benefited from
this trend with their encryption appliance.
I once worked with a brilliant engineer whose favorite saying was “never put a product where a feature should be.” I’d say this was Sun’s philosophy when we delivered the Sun StorageTek T10000 tape drive. Put simply, Sun put an encryption chip next to the compression chip on the drive – so data is encrypted as it is fed onto the tape. Simple and affordable – no extra appliance needed. Sun also offers the StorageTek Crypto Key Management Station to centrally authorize, secure and manage encryption keys.
6. Eco Storage (aka Green Storage/IT): I freely admit that when I was first approached with “Green Storage” I was a skeptic. I would have also never guessed Al Gore would win the Nobel Peace Prize! But Eco also stands for Economics. If you save power and footprint, and the world while you are at it – who can argue with that? But the challenge for storage customers will be sorting through the vendors who make REAL Eco investments vs. the ones that just add “Eco” or “Green” to their marketing collateral. Sun’s in the “real” category, investing heavily in Eco IT. Sun’s Eco efforts can be seen here...
7. Object Archive
(aka CAS, Application Aware Archive): The dizzying array of regulations,
compliance requirements and influx of data have made the archive market one of
the fastest growing markets in IT and storage.
And customers must continually evaluate which archive approach will work best for them. The trend here is to “build a better mousetrap archive.” The challenge is this, an archive system
must:
- Store a lot of data affordably
- Have WORM functionality so documents show up un-altered in an audit or court of law
- Be easily and quickly accessible
- For more than 100 years…
But do keep in mind for deep archive; Sun’s StorageTek
SL8500 Tape Library is tough to beat – just one library's max raw capacity is
56 Petabytes, and data sitting on tape consumes 0 kilowatts and generates 0 CO2… (see above trend #6)
8. New Interfaces, Protocols & Configurations: There is a lot of change happening in storage systems and how they are configured. The three primary ways storage is attached is Direct Attached Storage (DAS), Storage Area Networks (SAN) and Network Attached Storage (NAS). A disk system can also be configured in a couple different ways. RAID configurations stripe data across multiple drives and impact a system’s reliability and performance. JBOD (Just a Bunch of Disks) is more affordable because it does not require a disk controller, but provides no data redundancy. New interfaces and protocols will impact each of these markets significantly.
- SAS: In DAS, Serial Attached SCSI (SAS) is a newer serial communication protocol that will make data transfer speeds much faster and at a lower cost. SAS drives have replaced parallel SCSI for internal storage, and SAS HBA external host interfaces are just starting to ship in volume. SAS disk arrays started shipping this year and SAS JBOD are starting to ship. To complement the SAS HBA's and SAS arrays, there are new network components called expanders which are basically switches. Generally SAS disk will be cheaper than Fibre Channel (FC) with similar performance and capacity. (Sun has SAS host I/F, disk I/F and SAS disk drives)
- iSCSI: In the SAN market, the iSCSI protocol is making a significant impact by taking the cost out of expensive Fibre Channel SANs. iSCSI allows users to send SCSI commands over their existing IP networks.
- File Virtualization: In the NAS market, file virtualization is making NAS farms much more scalable and manageable (See trend #2)
- Clustered RAID: A new innovation to watch in the RAID market is horizontally scalable RAID or clustered RAID for large applications. Digi-Data is a small storage company pushing this innovation.
- JBOD: Keep your eye on Sun in the JBOD market. JBOD is more affordable than RAID, but does not have RAID’s redundancy and reliability features. But what if you had an infinitely scalable file system with data integrity and RAID features running on JBOD, say, something like ZFS?
- Unified Storage: Another trend in this area are unified or hybrid devices – storage systems that can handle multiple protocols and interfaces, including iSCSI, Fibre Channel, and NAS – all in one unit. Makes sense to users with a dizzying array of choices in the market.
- 10GbE: Lastly, 10 Gigabit Ethernet or 10GbE is the latest and fastest of the Ethernet standards that will re-shape data center networking – offering a fast, common and affordable network technology for IT and Web 2.0 applications from supercomputing to networked storage.
9.
10. Storage as a Service:
Storage as a service offered over the Internet has been talked about for years – but poor performance and implementations have cooled this trend. However, Amazon has given Storage as a Service a power boost with its Amazon Simple Storage Service (Amazon S3). By leveraging Amazon’s existing e-commerce and storage infrastructure, the company is offering customers storage capacity for $0.15 per GB-Month of storage used – possibly the cheapest $/GB on the planet. And while this may have more play in the consumer market, Amazon could re-invigorate the storage as a service trend. Also keep an eye on Sun’s Internet Service offerings over Network.com...
--- Updates ---
- For more color on Trend #1 and open storage platforms, read about IBM's latest acquisition...
- For more on Trend #9 (SSD) see EMC's SSD announcement
- An update on #10 (SaaS) see EMC's announcement of MozyEnterprise and EMC Fortress
Posted at 12:51PM Nov 30, 2007 by Taylor Allis in Storage Intelligence | Comments[6]
Today's Page Hits: 107