Technology and the Environment DD's Eco Notes

Yahoo! has been in the news with its new plans for an energy efficient datacenter in upstate NY, along with plans to discontinue purchasing carbon offsets.

First, nice to see Yahoo! join the ranks of those who find they can do more for the climate by investing in their own GHG reduction, rather than buying carbon offsets and investing in others.

Second, the article mentions that Yahoo! may patent their datacenter design. (Note: getting a patent doesn't mean you can't give it away - the question is whether they will share the work or not). If they really want to do something for the climate they will give the design away - let others do the same thing! We've published our green datacenter plans on the Internet in hopes that people will copy them and help reduce the overall load of datacenters on our energy infrastructure.

Want to have an impact? Use your expertise to teach others how to reduce theirs!

The Green Grid has cool new tools and maps for datacenter efficiency:

• Fresh air cooling map of North America (shown above)
• Water side cooling map of North America
• Free cooling calculation tool

The image above is a snapshot of the fresh air cooling map

All of them are here. Green Grid members have access to the higher res versions. Great job, Green Grid!

The Sunday Times in London caused a big news splash a week or so ago with their coverage of Physicist Alex Wissner-Gross' comments about the GHG emissions of Google searches, which apparently turned out to not be Wissner-Gross' comments, and so on and so on.

But the story got our team thinking about the carbon impact of common internet activities, and wondering about the impact of each email, YouTube, iTune, SMS text, and tweet that flies across the net. We all have day jobs, so we couldn't estimate all of them, but we had some good internal data on email so Mark Monroe led the team in an analysis of the emissions of the internet's original (warning: bad pun coming) "killer app".

We broke the email process up into 4 parts: composition, processing, and storage. This made the analysis easier and highlighted some big differences in where the carbon loads are. The other big area to look at would be transport (the energy used by switches in the network to move the bits around). That's the one area we didn't have good data, but the time involved in transit in the network is so small that we believe it is much lower than the storage or processing costs. (anyone have any data?)

For processing, we looked at the energy consumed by one of our large enterprise email systems. This system serves over 12,000 employees, processing 12.6 million emails per week. For all that work, we ended up estimating the energy per email processed at about 0.13 watt-hours, which in the US would average about 0.08 grams of COe2.

Storage was next. The same server farm has over 330 million emails stored, with an average size of 76KB and and average age of 18 month on disk. That's roughly 27,000 emails per employee, stored mostly on redundant, highly available spinning disks within the complex. When all was included, storing the average email used up another 0.36 watt-hours, or about 0.23 grams of CO2e. With storage we're now up to 0.31 grams of CO2e.

On to composition. Looking at a random sample of about 62,000 emails, we found the median number of new characters typed into an email was around 300. If the average English word is 6 characters, and the average typing speed while composing is around 19 words per minute, our typical user is spending 2.6 minutes in front of a laptop, desktop, or thin client to compose each message. We estimated the average power draw for these 3 types of devices at our company is about 66 W, so the 2.6 min of composition time would consume about 2.9 watt-hours of energy, producing 1.8 grams of CO2e (obviously it would be less if you only used thin clients, more if you only used workstation-class PCs).

Note that that's six times the amount of energy and carbon produced in the processing and storage steps. But it highlights the fact that the back end email systems are pretty efficient. It's the time sitting in front of the console that chews up the big energy. As an example, the 25 minutes it takes to compose this blog entry on my 32 watt laptop results in 8 grams of CO2e. The same pattern probably holds for Google searches, where the 0.2 seconds the search spends inside Google's data center is nothing compared to the minute or two you spend reading it on your screen and clicking through to several references.

Add up the emissions to compose, process and store a message for 18 months and it totals 2.19 grams CO2e. To put this in perspective, humans emit 0.5 grams of CO2e per minutes by through breathing, so a 2.6 minute email is 1.3 grams of CO2e from breathing. Doesn't seem like much until you consider the total volume of email is estimated at 97 billion messages per day, which turns our little puffs of CO2e into 212,000 metric Tons of CO2e per day.

If the amount of spam keeps rising, maybe this will be our "killer app"!

I had to laugh at this blog in Information Week titled "Going Green Merits Yellow Light". I see this every once in awhile - people so caught up in the green aspects of a product that the "goodness" of the energy savings or lower impact spills over distorts their perspective. In this case the energy efficiency was somehow supposed to have made up for the fact that it was designed by a small company who apparently had some serious engineering (and, ultimately, business) issues.

For procurement, supply chains, vendor selection, etc, environmental responsibility has to be an 'AND', not an 'IN PLACE OF'. Make sure that as your teams increase their focus on eco, they don't forget the stuff that they're already good at, like evaluating product reliability, vendor's financial viability and product fit. If a supplier can't meet your volume needs, all of the green-ness in the world won't help meet customer demands.

But you can also go a step farther and get proactive. Take your best suppliers and vendors today and tell them where you need them to be in a few years. Maybe you're a little ahead of them in thinking about green strategy. Maybe they're putting their green emphasis somewhere that's not as useful to you.

From our customers we're getting more and more eco hurdles to clear in RFPs, but they're all about what we're doing today. Let's talk about datacenters 5 years from now and what you think you'll need. If you're trying to have that discussion with us and we're not listening, send me an email directly. We've got plans, but I want to make sure they're what you need. I'd hate to miss the mark and cause our customers to have to go buy exploding computers from some fly-by-night startup....

I had some time blocked off this morning to finish off a post I've been working on about energy measurement, when IBM got me so confused with their announcement on Friday that I had to put my other project aside (there's also a video here if you're interested).

The announcement starts off simple enough: a) there are lots of opportunities to save energy and money in datacenters, b) customers want to validate that they're actually saving energy. This makes sense. We're seeing the same opportunities in our datacenters and in our customer's datacenters, and we all want to make sure the savings are real.

The next step is where it gets confusing. Most of the people I know are having good luck reading monitoring their equipment, reading their electric meters and looking at their power bills to verify that they are in fact saving energy. But for some reason, IBM says that's not enough. What you really need is an "IBM Efficiency Credit Certificate", which are based on the relatively obscure concept of Energy Efficiency Certificates, which are the lesser known cousin of Renewable Energy Certificates, or RECs. And to do these, IBM has partnered with a (tiny?) company called Neuwing Energy, LLC.

Basically it appears that when you finish a project you pay Neuwing some money, which they turn around and pay to one of their verifiers (see their website), who then verify your efficiency gains. In exchange you get a certificate saying what you saved. In theory there is a market that you could sell or auction these certs off at, but after a couple of hours on the web I've been able to find out almost nothing. The presentation on Neuwing's site says that the power companies in three states are required to deal in these, but there's no evidence of a functioning market. Note that EECs are neither required nor sufficient to trade on the Chicago Climate Exchange, which is as close to a real efficiency market as we have in this country today. Nor are these required to report to EPA Climate Leaders or the Carbon Disclosure Project.

So I get the theoretical concept of EECs, though I'm unclear on the practical issues of their value and how you sell them. If you don't plan to monetize them, it seems like a waste of time and money - there's a lot easier ways to understand your energy savings than paying Neuwing. But that leaves me with some questions for IBM:

Does IBM endorse EECs as a full-blown public climate policy? Are they lobbying in Washington and in state houses across the country for a system based on EECs, as opposed to carbon cap-and-trade, carbon taxes or other market-based policies? Whether they wanted to or not, their announcement has brought this obscure cert to the forefront, and they're telling their customers that they should be paying for these things. Is this what they really want to happen?

These things are so obscure that there's lots of opportunity for misunderstanding, confusion and claims of greenwashing (just ask Whole Foods about their REC program). Is IBM really so convinced that EEC's are good environmental policy and good for their customers that they are staking their green credibility on it?

Finally, these questions aren't all rhetorical. I'd appreciate some real data on EECs, and if IBM or someone else with a datacenter has actually acquired some and sold them, I'd be interested in how it works and how much they were worth relative to the energy savings. Maybe this is the future and IBM's just out ahead of everyone else. Or maybe it isn't.

Last week I visited Milano, Brussels and Berlin in a quick trip through the EU. The trip was a mix of customer activities, government discussions and press interviews. Here's some random and unordered notes:

• I got to drive the Eco Rider scooter/bike thing that the German team has been driving around the country. It was lots of fun, and really captured people's imaginations. I'm big on gimmicky promotions, but the Eco Rider somehow transcended gimmickry and put the right kind of images in peoples minds.
• My sense is that Europe is 6 to 9 months farther along in "eco awareness" than in the US (California exempted - I think the EU and CA are about equal). Your average European is just more up to speed and thoughtful about the environment than your average American. That said, the Europeans aren't to the point where they'll do a project just for the environmental benefit, eco as ecology AND economics is just as true in Europe as elsewhere.
• The eco financing problem that I find rampant in the US is just as prevalent in Europe. People paying the energy bills are not the same as those making purchasing decisions. I'm now convinced this is a global problem that needs real attention.
• I kicked into high gear with the EU policy folks I met with in Brussels, as well as the press I met with in Berlin about the need for computer companies to publish accurate anticipated energy usage for their products. There's a lot of interest in importing Energy Star, but many people that they will get energy reporting as a result, and its not true. I know this isn't easy, and standards are still coming, but we're making good headway on this. I've really come to believe that having your vendor inform you of expected energy usage is Consumer Right #1 at this point in our history, and Energy Star just doesn't go far enough in this regard. I'm sure you'll hear more from me on this going forward, as I'm wound up about it.
• In Berlin I met with a large ISP and Sun customer, Strato AG, who is doing some awesome work. Step 1 was to move to Niagara servers to cut energy usage, and that's allowed them to get their power low enough to look at going for totally green power. Next step is upcoming T2 systems, which will hopefully give them room for significant growth within their current power envelope.

Overall a very busy and tiring trip, but also quite rewarding!

I've been enjoying the posts by Dave Stangis over on the CSR@Intel blog. In particular, I really resonated with his post "What Exactly Is the Social Responsibility of Global Corporations?", and "The color of money....maybe it's not so tough to be green.". Check out the blog if you haven't, its another good place to understand what's going on as big companies, and in particular, big tech companies like Sun and Intel reason through the Eco and CSR issues.

One place I'd like to differ with Dave is on his use of the word 'green'. Is 'green' a state of mind, good intentions, or is it a result of actions? Dave says that Intel has been green for years, but that statement only makes sense if you use the first two definitions. Intel's carbon footprint is at the end of this article, and its not small. (Kudos to them for being one of the early reporters through the Carbon Disclosure Project, ours is submitted and will show up when the publish the CDP5 report.)

Maybe we're being too conservative, but at Sun we've decided to not use the word 'green' when referring to ourselves. We're not green. Our products and operations use lots of energy (255,000 tons of CO2 from our US operations, lots more from our products), we ship lots of lead in our products (solder), etc etc. We are, however, making big improvements throughout the company, so I will say we're 'greener', but 'green'? No way, not yet.

Speaking of big tech companies, the article above noted that Google won't disclose their CO2 footprint. So lets guess! After polling lots of people (no insider knowledge among them), the consensus is that Google's datacenters are in the 500MW to 1GW range. Using the US average for CO2 emissions from electricity, that would put them in the 3M to 6M tons/year range. That puts it in the range of Intel and 25% to 50% of GE, who's 13 times bigger in revenue. Interesting...

Congrats to AMD on exceeding their initial GHG reduction goal, and on setting another aggressive goal. Great job!

As the article mentions, transparency is something that we believe in when it comes to environmental impact, and AMD is setting a great example.

Last night I got thinking about the industry analyst firms and the environment. This was sparked by Gartner's critique of the Green Grid (accessible here, but only if you have a Gartner account) and Simon Mingay's comments in the press (see "Gartner Dumps on Green Grid").

After my initial reaction ("Jeesh, I know that Green Grid's not perfect, but did it really deserve that at this early stage?"), I started thinking about the value that someone like Simon could bring to the process if he was involved, instead of sitting in the stands and critiquing the calls on the field.

So my question to Gartner, IDC, Forrester, etc, is this: how can you guys get involved in the Green Grid, either directly or indirectly? Its a shame to have a group working together voluntarily on such an important problem, and then to have folks with important data and expertise sitting on the sidelines.

Also, in the course of seeing what some of the analysts are up on the environmental front, I found a ton of whitepapers. So I looked deeper for what they were doing themselves, and I came up totally empty handed. IDC (and their parent, IDG). Gartner. Forrester. Carbon footprint disclosures? Nope. CSR reports? Nada. Environmental impact statements? Zip. CO2 reduction goals? MIA. Collectively billiions in revenue, over 10,000 employees, etc. Did I miss something? (if so, please let me know!)

I found this kind of ironic, since discussions with these organizations have given us some really valuable input on the eco and CSR parts of our website.

Rajesh and Jonathan forwarded me an awesome article on the irrationality of not pursuing energy efficiency. It features this chart:

The article goes on to explain:

"The result is a testament to economic irrationality. The measures below the horizontal line have a negative abatement cost—in other words, by carrying them out, people and companies could both cut emissions and save money. At a macroeconomic level they would boost, rather than reduce, economic growth."

Recently, in our Broomfield datacenter we did an $80K capital improvement that should yield around $100K of savings in electricity per year. There's nothing that unique about our datacenters, and I suspect that similar savings exist in most others. So why aren't we seeing a mad rush to do datacenter efficiency projects?

The article hints at the reason in the middle: in most organizations the people paying the energy bills aren't making capital decisions, and the people making capital decisions aren't paying energy bills. For example, if our IT guy in charge of the improvement above can get credit for the energy savings, his project has a positive return in under a year. If he can't get credit for the energy savings, this project never shows a positive return.

This should be easy to fix, right? Well, not really. We're dealing with it internally on a case-by-case basis, but I'd love to see us institutionalize it better. Others have more trouble. I was recently at a major university which gets lots of government grants, and electricity gets paid for out of "overhead" payments that come with lots of strings attached. And, oh, by the way, these payments are based on a fixed percentage and don't change if you make yourself more efficient.

I gave a talk at a conference last week which included practical advice on how to reduce carbon emissions in businesses. Step 1 was to figure out where your energy goes today. Step 2 is to get your finances lined up so that you can factor energy savings into the ROI on capital improvements. Starting efficiency projects without this isn't the end of the world, but its sure helps. There's nothing worse than having efficiency projects get a bad name because they always cost money and don't have a return. This is especially painful when these projects, in fact, often have fantastic ROIs.

So if you're involved in IT, or in eco in general in your organization, make sure you work this issue up front. Energy efficiency makes financial sense, but only if you can bring the finances under one umbrella.

In a recent post I laid out the various areas that we need to account for in the overall environmental footprint of a product. One thing you'll notice is the the product travels at least twice (once from the manufacturer to the customer and once from the customer to the recycler), but the bulk of the product may travel more times than that. For example, parts may flow through a supply chain such that the heavy parts take two or three hops before the manufacturer, and may take an additional hop after being initially sorted for recycling.

While the number of hops isn't important, its important to know they exist. You may think you're buying a product from someone in California, but if all of the subcomponents originated in China, then all of that weight started there, even if they weren't assembled yet.

As noted a year or so ago, the environmental cost of shipping varies wildly depending on the method used (per "Let My People Go Surfing"):

• Rail or boat: 400 BTUs per ton per mile
• Truck: 3,300 BTUs per ton per mile
• Air cargo: 21,760 BTUs per ton per mile

So lets make up a scenario and see what the impact is. First, assume a 250W server which will run all year long (roughly 9000 hours/year) for 3 years. That makes a total of 6,750 kWh. Second, assume that the parts start in China, end up as a product in New York, and meet their final resting place in a recycling facility in Oregon, for a total of approximately 13,000 miles. Finally, lets assume that the server weighs 50 pounds.

If we use only one type of transportation, the three options above would check out as follows:

• Rail or boat: 130,000 BTUs or 38kWh
• Truck: 1,072,000 BTUs or 314kWh
• Air cargo: 7,072,000 BTUs or 2,072kWh

So in this case, if we use either of the less costly ways, the overall product impact is not overly affected. However, if we fly the parts around for each hop, the environmental cost is the same as using the product for a whole additional year!

Apparently our corporate "greenness" is going to be measured by the size of our investment in the environment. Bank of America came out first with a $20B commitment, but was soon out-greened by Citibank at $50B. Most recently, IBM came out at $1B.

So what do these numbers mean? Is IBM really 50 times "less green" than Citibank, and 20 times less than BofA?

Obviously not, but apparently that's the conclusion that their marketing department is begging us to make. I know for a fact that they have some really good programs going on, but some marketing person decided that their press release would somehow be more interesting if it had said $1B. Details behind all of these large dollar figures are vague, with key assumptions and allocations unstated, so comparing them is silly. Why throw it out there if you aren't going to back it up and you don't want people to use it as a yardstick of goodness? Maybe that's why IBM's environment group doesn't have a link to this release.

The world is already getting savvy about green marketing, and even companies with a good story to tell, like IBM, can hurt themselves when someone tries to tell a "bigger story". Fun "who's greener than who" articles like this one are starting to crop up.

At Sun we're trying hard to stick to the facts, and I think we're succeeding most of the time. We debate a lot about specific words and phrases, and worry that our message won't break through. Its tricky today and its going to get trickier. Its important to us that people see what we're up to, and we hope that it helps us in the marketplace, but also that others can learn from what we've done.

Recently we got some nice indirect feedback from a post in Green Wombat about our work with JavaOne - it's good to know that a stunt-free press release can still get some attention.

I took a little break from blogging over the holidays, but am back in the saddle again.

The flurry of press and activity at the end of 2006 continued straight through the holidays. If the number of eco conferences in 2007 is a leading indicator, attention to datacenter power will continue to grow as a topic through the new year. Here's a few tidbits from over the holidays:

• Project Blackbox...relentless. The last two calls I did prior to breaking away with my family for XMAS were on Project Blackbox. Oh, and then I did one more even after I'd officially left. I'm still accepting bets for those of you who think this thing won't ever make it into the mainstream.
• EPA announces plans for Energy Star for servers. Energy Star is a wildly successful program, and we're excited to see it expanded into the server space.
• Customers are taking notice. Interest in our energy efficient Niagara servers continues to rise. And keep an eye on the press, because we're starting to talk more about Niagara 2.

We're hitting the new year running. We made a dent in datacenter power in 2006, and we've think we can take it a lot farther in 2007. Quoting the motto of my home state: Forward!

Yesterday the President signed the bill "H.R 5646":

H.R. 5646, which directs the Environmental Protection Agency to study the energy consumption of Federal and private computer data centers

Our Washington team worked hard, along with others in our industry, to get this through the House over the summer, and then through the Senate this fall. We were interested in this bill for a number of reasons:

1. It helps to raise awareness to the growing issue of datacenter energy consumption
2. The EPA has a strong history of facilitating the energy discussion between producers and consumers. If you've ever looked at an energy sticker on an appliance or the MPG sticker on a car, then you know what I mean.
3. The federal government is the largest IT consumer in the world. Bringing energy into their procurement process would have a positive impact on their energy usage, would setoff a big ripple through the industry, and I believe the results would set a strong example for other large IT consumers.

The EPA report that is called for in the bill will be out in 180 days or so, and I'll report out on it then.