Chalfant's Blog

Follow the Money

Most people remember the manta of Jerry McGuire – Show me the Money! When it comes to storage, customers would be wise to 'follow the money'. Everyone in the vendor community understands the insider quip regarding sales people – they are said to be ‘coin operated’. While this is a friendly jab at the sales people, it is also fairly true. Sales managers know that when you build a compensation plan for the sales force, you have to be really careful. Sales people will figure out how the plan works and do what it takes to best leverage that plan.

Business is very much the same. I was recently out on a sales call where I was once again reminded that businesses will also do what they are compensated for. Here is the situation.

About a year ago, I was a speaker at a large golf tournament. Everyone that has seen me present knows to expect brutal honesty and some sound advice on how to bring their operating infrastructure into alignment with the needs of the business. And so it was at the event a year ago. One long term customer heard some new things we are doing which involves an assessment that peals back the layers of complexity in the data centers, and exposes how inefficient storage and servers can be. They decided to give it a try, and so we contracted to perform a business value assessment.

The interesting thing about this engagement is that this customer has multiple data centers. One of their data centers is managed by a service provider, which has lots of disk from one vendor in particular. Our team showed up at that data center with tool kits in hand to begin the collection process which leads to an irrefutable exposure of how much space is commonly wasted. The cool news is with this in hand, the customer can cut their costs in a big way, because they can re-allocate and use space that is otherwise hidden, undiscovered and unused.

Only one problem, the service provider and the disk vendor get compensated based on how much capacity they use. If we go in there and uncover how much space is wasted, and can reclaim it, the service provider and the disk company have a major revenue hit. They don’t like that. So instead of inviting the team in for the greater good of the customer, they blocked the doors and wouldn’t allow any work of any significance.

Now interestingly enough, I have met with this service provider, over and over and over. They are afraid the approach we tout will make them loose business. I have told them this is an ‘eat your own children’ problem, and that if you don’t – someone will do it for you. But – in a time when corporate pressures push revenue and operational efficiency at the same time, it seems impossible to change the business model enough to not only make a reasonable profit, but also do what is right for the customer. So, at the end of the day they blocked the effort.

Well the day of reckoning may be on the horizon for them. I had a meeting with that same companies CIO today, and exposed the game that is played by vendors pushing capacity (as a value for their own top line growth, from the expense line of our customer). I showed him and his technical team how to expose it, and what to do about it. Right in the middle of my presentation he stopped me and asked if we knew the guy that is in charge of this service provider locally. He said he wanted to get up ‘right now’ and bring him into the meeting to find out what they were doing.

Follow the money. If your service company and your vendors are comp’ed to sell capacity, they will sell capacity. What you need to figure out is how to comp them for selling the minimally correct amount of infrastructure that allows the business to operated at peak performance. Period, full stop.

I love the awakening I see going on. Business leadership knows fair well that the IT infrastructure is critical to business. They have suspected for years that there is a better way to align the needs of the business with a balanced implementation of the infrastructure. Now that the tools, technologies, and consulting practices are in place to deliver that, the game is up.

Fair warning to all vendors that are over selling for personal gain, the game is drawing to an end.

Posted at 08:27AM Jun 27, 2007 by chalfant in Personal  |  Comments[2]

Disk only Solutions are Suffering from Atrophy

It seem only fair that if the disk only companies insist on propagating the ridiculous assertion that tape is dead, that I can launch my own ridiculous campaign, also with a medical metaphor.

Here it is – Disk only solutions have Atrophy.

Like human Atrophy, Disk only solutions are the partial or complete wasting away of a part of the disk. Causes of atrophy arise from disuse, mismanagement, or lack of exercise.

This results in a visible bloating of a data center, and has long been feared and shunned by senior managers as a dangerous to the overall health of their organization. Even though this condition can be effectively treated today, the disease still causes widespread anxiety. As a result, organizations suffering from Disk Atrophy often suffer psychological and social problems. Vendors will offer free tickets to major sporting events to help minimize the impact, but when IT professional return, the Atrophy has grown. Sad really.

Yesterday I mentioned that 70% of disk space is wasted on average. Let’s take a closer look at that.

Typically there is far more storage available to users than they appreciate or understand. Sun completed a study performed at 200 user sites globally. In the study, tools were installed to uncover the allocation and utilization efficiency. The results were surprising.
• 10% of all the capacity found had once been allocated to a server, but the server had gone away for various reasons, without the space been returned free. We define that as allocated but orphaned.
• 15% of all the capacity was allocated but unused. Typically someone had requested space from a storage administrator, but no file system or file was stored. It can also be represented as unused database table space.
• 5% of the space is inappropriate use. This is typically unstructured data commonly JPEG, MPEGS, WAV, AAC, etc., that should not be on the system.
• 40% of the space was inert. This was data that had not been referenced for six months or more.
• That left only 30% of the data been well used, along with some capacity for growth.
There are some interesting lessons here. The first one is that the combination of orphaned, unused, and inappropriate space represents some 30% of the typical capacity that is wasted. That being said, when we look at the role of the typical storage administrator, much of the focus of what they do is yielding fairly significant amounts of wasted capacity, which has a large financial impact on their company. Commonly, this issue is never addressed.
The opportunity is to identify the parts that are being wasted though allocation inefficiency and then reclaim it. While that sounds simply, and actually is, remember, literally every site we examine is struggling with this issue (Atrophy is everywhere!). There is a cure fear not.
The second interesting lesson is in the next category of ‘inert space’, which represents another 40% of the wasted space. The inert data space is where effort should be applied to move unused data to more appropriate tiers of storage to reduce capital costs. The basic premise is to implement a policy manager that can migrate active data inside of a storage hierarchy based on specific usage behavior while improving costs.
A Hierarchal Storage Manager can move infrequently referenced, unimportant or old files to less expensive storage without changing the way users or applications access the files. There are a variety of policy based migration controls, among them are Age, Last Touched, Size, and Extension. You can even use HSMs to Speed up Database Performance by moving the index from high performance disk to capacity disk. Again, this isn’t secret sauce; various companies can do the same thing. Sun and Veritas are examples. What’s important is that you use them.
Besides health, what is there to gain for this effort? Plenty!
Customer Example
Here is an example. There was a large international bank that had a growing Enterprise Resource Planning (ERP) business application. Their growth rate was at about the industry expected standard for commercial business which is 30%. They initially went to their primary storage provider to ask for recommendations for how much more storage they would need to sustain the growth rate of their business. The disk vendor came back with their proposal which indicated that the expected need was 100 Terabytes. We suggested to the CIO that they give us the opportunity to assess the real requirements when efficient storage management practices are employed. The customer agreed to the assessment. The results were dramatic. After discovering all of the allocation and utilization inefficiencies, we mapped their growth into the misused space. We then did a gap analysis to determine how much more they would need, and discovered that the real requirement was for 2 Terabytes. This resulted in a savings of $3.8M over three years versus the first vendor’s recommendation. The key to this value was understanding the true nature of data usage behaviour, and mapping that over an appropriate storage infrastructure.

The Paradox
The truth often sounds paradoxical.
- Lao Tsu (c.604–531 B.C.) Great Chinese philosopher, founder of Taoism

While there is no doubt that managing the infrastructure will make the operation staff more productive, not everybody will happily follow this idea. The concern most operations staff will have is based on the fact that they are stretched too thin now to manage what they currently have, not to mention the growth. They often feel that any change is tantamount to adding additional complexity to the environment which will have serious operational impact, and will therefore result in them pushing back on any changes. This is exacerbated by the fear that the introduction of any new technology will only add to the problem. If things are done with a parochial mentality, often they are right. But not always.

The solution to this dilemma is found in doing the right things, not the popular things. Doing the right things is dependent not only on good technology, but also a deep understanding of the operational characteristics, the environment, business processes and application deployments across business applications. With these things in mind and some practical experience, one can architect a solution that will realistically reduce operational complexity, ultimately reducing operating expenses and capital expenses.
It is a sort of forest and trees challenge. Compounded by the lack of resources in a fast paced and growing environment, once the general architecture of an IT infrastructure is implemented, seldom do people go back and question the architecture implemented. The net impact is that often the benefits of the architecture and infrastructure are no longer efficiently delivering peak value to the business. It is no surprise considering that many businesses are growing, and that of course has a direct impact on the load and growth placed on the IT infrastructure. There are always a shortage of qualified people, which is what leads this paradox, people begin to simply buy and use more capacity as a way of dealing with the overload of efficiently managing what they have.

Slowly all people realize the infrastructure is strained, as the business has grown beyond the abilities to efficiently deliver IT services from what was originally conceived and implemented. That’s where things get exciting. With the lack of improved insights on how to manage the infrastructure, and with the lack of enough resources to manage the complete environment, the strained technology staff is forced to adopt a plug the dike strategy for the growth of the infrastructure, in an attempt to keep the business applications running at expected service levels.

This is a wonderful opportunity for self serving vendors to over sell capacity. Even that is understandable. If servers and storage were free of cost, one way to approach the issue is to hide the complexity of managing resources behind vast amounts of capacity. However, not only is this expensive from a capex point of view, from an opex point of view, eventually this will fail also.

Instead of a methodical analysis of how the infrastructure is tied to the needs of the business, and mostly because of industry hype, the effort is always focused on data backup, migration, archive, and tiers of storage for gaining value in the inert data category. All of which are good and interesting, however in the implementation phase mostly inefficient and costly.

Here is a basic outline for what needs to happen to gain efficiency.
1. Identify and manage the wasted space to achieve allocation efficiency
2. Identify and manage inert data to achieve utilization efficiency
3. Build a protection architecture that is justified based on the contribution the application makes to the business ( I will cover this in another BLOG)
4. Consolidate, which will refine the value achieved above
5. Virtualize to further refine

The road to health requires an examination, diagnosis, and the proper treatment. Disk Atrophy is ugly, but it is also curable, and is a big win for you when it is.

Posted at 08:22AM Jun 01, 2007 by chalfant in Sun  | 

The Death of Tape – Oh really??

Hooray for youth, for they shall inherit the world. For all of us parents, it is a test of patience to observe our children learning life’s lessons as they proclaim truth to be something we have learned already, and have found that ‘truth’ to be materially different.

I just read another article that mentioned that ‘old-timers’ will scoff at the idea that tape is dead. I read ‘old-timer’ as synonymous with a child enlightening a parent with their perception of truth.

Every time I listen to a customer telling me that disk is cheaper than tape, I can’t help but see the image of young Linda Blair playing the role of Regan MacNeil in the movie ‘The Exorcist’. Her eyes are creepy lights that glow, and when her mouth opens you hear the voice of Disk only vendors saying, “tape is dead’. Gee, I wonder why a disk only vendor would send out the message that tape is dead?

Look, there is some simple marketing math here folks. IDC reported that between 2005 and 2008, the installed capacity of disk will increase by 367%. Yet the amount of money that will be spent for all that growth is near flat. Also, most enterprises expect that they will hold their head count flat to manage all of that.

Business is pretty simple. Executive leadership cares about three things; 1) Making Money, 2) Saving Money, and 3) staying out of a little orange suit. It is the same set of rules for every single company that is publicly held. If you get this algorithm right, Standards and Poors, and Moodys will raise your ratings, institutional buyers will buy large blocks of shares, that creates the supply and demand problem, and that causes share holder value to increase. Simple right?

So guess what. If you are a disk only company, just like every other company, that means you have to grow faster than the market, and faster than the competition. If the total amount of money that is available in the market is flat based on what you are willing to spend, that means the revenue pie that is available to all of us vendors is fixed, and it is small. So how do you grow faster than the market to impress Wall Street? Well, you have two strategies you can use. 1) Saturation – where you have to displace everyone you can to own market share. If you have done a good job of that, and the hunting is getting thin, you need to 2) find new markets. 65% of the world’s data is on tape. That looks like a target rich environment to grow to the disk only companies. And so we hear the cry “tape is dead”. Of course they will tell you tape is dead.

If they can convince you to change from what is an incredibly cheap storage media, to move to something that will have a minimum of a 10X increase cost, which they collect – that’s called top line growth. The only problem is that it is taken from your bottom line. By the way, that’s just the Capital costs, the Operational expense to manage that disk is three times the cost to buy it. On the other hand, the cost to manage tape is 10 times less than disk.

Look, there is a massive amount of vendor hype and confusion that is out there on this. It is created for the benefit of the disk only companies. Calmer heads realize this and are doing the right things.

Part of our approach to this, is to lead with a business value assessment, where we can look at the operational needs and practices of you business, and then map the technology into a supporting foundation for the demands of your business.

In a survey of over 200 customers we looked at every single disk volume installed, and found that on average 70% of that space was wasted. If the disk only companies gave you the tools for you to see this, they would be in serious financial trouble. Instead they cry ‘tape is dead’, buy more disk. For those of you that have met me, you will remember the first thing I tell people is to stop buying storage. Better to focus on managing better what you have first.

We have seen companies realize savings on Net Present Value (NPV) calculations in the 10s of millions if not 100 million dollars. So can you.

I will be focusing this blog on the before and after of companies that use their infrastructure to optimize their business – so stay tuned.

In the mean while, let all of the youngsters that have a need to go with what’s cool and trendy have their head. Volume and reach don’t prevail on a balanced view, only time can provide that.

Posted at 09:39AM May 30, 2007 by chalfant in Sun  | 

Welcome

Welcome to my Blog! Like you have enough time or interest to read another one, right? On the other hand, we always have time for the things that are important to us. My business life involves visiting with IT professionals all over the world. One thing I have observed is that IT physics are all the same. Computers are computers, networks and storage all do basically the same things. However different people do find innovative ways of solving IT challenges. In this Blog, I will share the experiences I have when I observe new and different ways of meeting the challenges of IT. So check back and see what’s new.

Posted at 03:12PM May 29, 2007 by chalfant in Sun  |  Comments[5]