Flash memory gets a lot of press nowadays touting it as the cost-effective solution for your storage performance needs. With this blog entry , I'd like to explore some of the rationale behind this enthusiasm. Let's start with some numbers.
According to this IDC whitepaper by 2011, the digital universe will be 10 times the size it was in 2006. Another interesting aspect is that approximately 70% of this digital universe is created by individuals, but enterprises are responsible for the security, privacy, reliability, and compliance of 85%.
With that, I think it's safe to assert this data explosion will cause a need to increase storage performance.
Interestingly, the server world has reacted to the steady increase for compute performance with ever faster CPUs, higher memory capacities, multi-core designs, increased server utilization through virtualization, etc. Despite some innovation, it seems that the technology in storage land lags behind. Hard disk drives, even the fastest 15K RPM can not feed servers fast enough. They are some hundreds of times slower than what today's servers are capable of so they spend most of their time waiting for data after a request. They are starving (see Dave's blog entry here for quantitative example). The traditional remedy of adding more expensive DRAM may no longer suffice as data sets double every couple of years. Today, some applications are being chocked by spinning disk drives which which are causing storage latencies and I/O bottlenecks.
What is the solution here ?
A good initial example is Sun's approach, which is to place Flash-based storage in the form of SSDs to accelerate the overall storage I/O performance of the Sun Storage 7000 family with the use of Hybrid Storage Pools. By strategically putting SSDs in hot areas on your storage subsystem, Hybrid Storage Pools make great use of the performance characteristics of SSDs. On top of that, current SSD capacities are well suited for this approach.
But what if your need both : performance and capacity ? or in other words, what do you do if you need the performance of SSDs and the capacity of HDDs ?
Well, I guess we'll see what innovation will surface in this space. For now, keep watching Sun announcements.