Chalfant's Blog

Disk only Solutions are Suffering from Atrophy

It seem only fair that if the disk only companies insist on propagating the ridiculous assertion that tape is dead, that I can launch my own ridiculous campaign, also with a medical metaphor.

Here it is – Disk only solutions have Atrophy.

Like human Atrophy, Disk only solutions are the partial or complete wasting away of a part of the disk. Causes of atrophy arise from disuse, mismanagement, or lack of exercise.

This results in a visible bloating of a data center, and has long been feared and shunned by senior managers as a dangerous to the overall health of their organization. Even though this condition can be effectively treated today, the disease still causes widespread anxiety. As a result, organizations suffering from Disk Atrophy often suffer psychological and social problems. Vendors will offer free tickets to major sporting events to help minimize the impact, but when IT professional return, the Atrophy has grown. Sad really.

Yesterday I mentioned that 70% of disk space is wasted on average. Let’s take a closer look at that.

Typically there is far more storage available to users than they appreciate or understand. Sun completed a study performed at 200 user sites globally. In the study, tools were installed to uncover the allocation and utilization efficiency. The results were surprising.
• 10% of all the capacity found had once been allocated to a server, but the server had gone away for various reasons, without the space been returned free. We define that as allocated but orphaned.
• 15% of all the capacity was allocated but unused. Typically someone had requested space from a storage administrator, but no file system or file was stored. It can also be represented as unused database table space.
• 5% of the space is inappropriate use. This is typically unstructured data commonly JPEG, MPEGS, WAV, AAC, etc., that should not be on the system.
• 40% of the space was inert. This was data that had not been referenced for six months or more.
• That left only 30% of the data been well used, along with some capacity for growth.
There are some interesting lessons here. The first one is that the combination of orphaned, unused, and inappropriate space represents some 30% of the typical capacity that is wasted. That being said, when we look at the role of the typical storage administrator, much of the focus of what they do is yielding fairly significant amounts of wasted capacity, which has a large financial impact on their company. Commonly, this issue is never addressed.
The opportunity is to identify the parts that are being wasted though allocation inefficiency and then reclaim it. While that sounds simply, and actually is, remember, literally every site we examine is struggling with this issue (Atrophy is everywhere!). There is a cure fear not.
The second interesting lesson is in the next category of ‘inert space’, which represents another 40% of the wasted space. The inert data space is where effort should be applied to move unused data to more appropriate tiers of storage to reduce capital costs. The basic premise is to implement a policy manager that can migrate active data inside of a storage hierarchy based on specific usage behavior while improving costs.
A Hierarchal Storage Manager can move infrequently referenced, unimportant or old files to less expensive storage without changing the way users or applications access the files. There are a variety of policy based migration controls, among them are Age, Last Touched, Size, and Extension. You can even use HSMs to Speed up Database Performance by moving the index from high performance disk to capacity disk. Again, this isn’t secret sauce; various companies can do the same thing. Sun and Veritas are examples. What’s important is that you use them.
Besides health, what is there to gain for this effort? Plenty!
Customer Example
Here is an example. There was a large international bank that had a growing Enterprise Resource Planning (ERP) business application. Their growth rate was at about the industry expected standard for commercial business which is 30%. They initially went to their primary storage provider to ask for recommendations for how much more storage they would need to sustain the growth rate of their business. The disk vendor came back with their proposal which indicated that the expected need was 100 Terabytes. We suggested to the CIO that they give us the opportunity to assess the real requirements when efficient storage management practices are employed. The customer agreed to the assessment. The results were dramatic. After discovering all of the allocation and utilization inefficiencies, we mapped their growth into the misused space. We then did a gap analysis to determine how much more they would need, and discovered that the real requirement was for 2 Terabytes. This resulted in a savings of $3.8M over three years versus the first vendor’s recommendation. The key to this value was understanding the true nature of data usage behaviour, and mapping that over an appropriate storage infrastructure.

The Paradox
The truth often sounds paradoxical.
- Lao Tsu (c.604–531 B.C.) Great Chinese philosopher, founder of Taoism

While there is no doubt that managing the infrastructure will make the operation staff more productive, not everybody will happily follow this idea. The concern most operations staff will have is based on the fact that they are stretched too thin now to manage what they currently have, not to mention the growth. They often feel that any change is tantamount to adding additional complexity to the environment which will have serious operational impact, and will therefore result in them pushing back on any changes. This is exacerbated by the fear that the introduction of any new technology will only add to the problem. If things are done with a parochial mentality, often they are right. But not always.

The solution to this dilemma is found in doing the right things, not the popular things. Doing the right things is dependent not only on good technology, but also a deep understanding of the operational characteristics, the environment, business processes and application deployments across business applications. With these things in mind and some practical experience, one can architect a solution that will realistically reduce operational complexity, ultimately reducing operating expenses and capital expenses.
It is a sort of forest and trees challenge. Compounded by the lack of resources in a fast paced and growing environment, once the general architecture of an IT infrastructure is implemented, seldom do people go back and question the architecture implemented. The net impact is that often the benefits of the architecture and infrastructure are no longer efficiently delivering peak value to the business. It is no surprise considering that many businesses are growing, and that of course has a direct impact on the load and growth placed on the IT infrastructure. There are always a shortage of qualified people, which is what leads this paradox, people begin to simply buy and use more capacity as a way of dealing with the overload of efficiently managing what they have.

Slowly all people realize the infrastructure is strained, as the business has grown beyond the abilities to efficiently deliver IT services from what was originally conceived and implemented. That’s where things get exciting. With the lack of improved insights on how to manage the infrastructure, and with the lack of enough resources to manage the complete environment, the strained technology staff is forced to adopt a plug the dike strategy for the growth of the infrastructure, in an attempt to keep the business applications running at expected service levels.

This is a wonderful opportunity for self serving vendors to over sell capacity. Even that is understandable. If servers and storage were free of cost, one way to approach the issue is to hide the complexity of managing resources behind vast amounts of capacity. However, not only is this expensive from a capex point of view, from an opex point of view, eventually this will fail also.

Instead of a methodical analysis of how the infrastructure is tied to the needs of the business, and mostly because of industry hype, the effort is always focused on data backup, migration, archive, and tiers of storage for gaining value in the inert data category. All of which are good and interesting, however in the implementation phase mostly inefficient and costly.

Here is a basic outline for what needs to happen to gain efficiency.
1. Identify and manage the wasted space to achieve allocation efficiency
2. Identify and manage inert data to achieve utilization efficiency
3. Build a protection architecture that is justified based on the contribution the application makes to the business ( I will cover this in another BLOG)
4. Consolidate, which will refine the value achieved above
5. Virtualize to further refine

The road to health requires an examination, diagnosis, and the proper treatment. Disk Atrophy is ugly, but it is also curable, and is a big win for you when it is.

Posted at 08:22AM Jun 01, 2007 by chalfant in Sun  |