Sun hosted a well attended Silicon Valley Leadership Group (SVLG) event focused on Energy Efficient Datacenters last Thursday at Sun's Santa Clara campus. The agenda consisted of case studies, an energy demand management program and a facilities tour. I'll share some of the highlights here.
Small Retrofit Investment Yields Big Savings
NetApp Facilities Director, Dan Hoffman presented the results of some very targeted energy efficiency measures in a mid-sized (6764 sq ft.) data center. The net results of a $146k investment were impressive:
- Estimated Energy Savings: 1,042,000 kWh/year
- Cost Savings: $125,000 -$145,000 year
- GHG emission reduction: 3.6 million lbs./year (equal to removing 150 cars from highways annually)
And under a PG&E rebate program they were able to recoup their entire initial investment, so the project had a zero year payback.
The key measures, developed with the assistance of Lawrence Berkeley National Labs (LBNL), that made these savings possible included:
- Supply right amount of air at correct temperature where it is needed, i.e., to the inlet of IT equipment
- Install curtains between hot and cold aisles to minimize Source Air (SA) and Return Air (RA) mixing
- Replace SA registers that direct air to top of racks with grills that direct air downward to reduce stratification & hot spots
- Install array of wireless temperature sensors strategically placed on cold aisle side of racks to enable raising SA temperature (i.e., raise the set point on the thermostat by 2-4 degrees)
- Allow RA to rise to 80-90°F to enable a substantial reduction in RA/SA fan speed
- Optimize “free-cooling”effect of economizer by raising SA temperature and allowing dampers to modulate longer to reduce the load on the cooling system
I always hear the question, "Why don't we build data centers in northern climates where we can use the outside air for cooling?" So it was of key interest to me to hear Dan's experience with "free cooling" and the economizers' effect on airborne particle concentrations. When they increased "free cooling" to use 85% outside air the particle concentrations jumped to 11 micrograms/m3, nearly double their normal level. This is well below ASHRAE standards, but still of potential concern to data center managers worried about contamination. The environment used 40% filters, which is typical of modern data centers. It was estimated that the particle count and size could be reduced significantly (over 50%) by replacing the 40% filters with 85% filters. The energy required to force air through these finer filters, of course, is higher, so may not be rationalized in terms of net energy use.
Demand Response Coming to a Data Center Near You
LBNL Computer Systems Engineer, Girish Ghatikar shared details of the Demand Response (DR) programs being developed in conjunction with utility companies, and gave some compelling data on why voluntary participation in PG&E's AutoDR plan could be beneficial to medium and large IT equipment users.
The current PG&E DR Program provides technical assistance and requisite equipment to help facilities managers shed electrical load during critical peak usage periods. Savings through discounts on electricity can be significant (up to $50 per kw saved) under the program if the facility can reduce usage during "events" (project to be between 12 and 15) during the summer (May 1 - October 31). Participants are notified day before or day of an event, and participants control the level of curtailment during the period.
For most data center managers, the idea of switching off IT equipment during business hours and on short notice, is akin to fingernails scratching a chalkboard. The mere mention of cycling the power on a server gets a visible shiver from the average system administrator. But many of the concerns are becoming manageable, and the notion that cycling the power reduces MTBF is dated - most modern IT equipment does not suffer any significant reduction in reliability due to moderate frequency of power cycles.
In response to the management problem - how to efficiently and gracefully respond to peak event notification - the LBNL folks have developed a systems architecture that can be integrated into existing environments. The infrastructure includes a Demand Response Automation Server and Energy Management Control System (EMCS), which transmit event notifications and trigger load shifting actions. The architecture leverages a new SOA standard, OpenADR based on SOAP, that enables a publish and subscribe model. The payload of the OpenADR messages contain Event Pending, Price levels, and Price Schedule. This architecture, while promoted through the AutoDR program can be used for finer grained control that can be integrated with other event notifications besides PG&E's critical peak days. For example, if price exceeds $0.20/kWH then initiate a load shed activity.
State of The Art - Sun's New Santa Clara Labs
When I first met the distinguished LBNL Engineer Bill Tschudi at this meeting, he said, "You guys are doing some great stuff, but why aren't you using free cooling in your new rooms?" Good question, Bill. I don't know, but I suspect it has something to do with the somewhat unique mixed use environments that are characteristic of the new labs being built in Building 12 on the Santa Clara campus.
As Dean Nelson, Sun's Shared Lab Services Director, pointed out on our tour of these nearly completed rooms, engineers will be working in these environments as part of their normal day to day routine. For the most part, these are not lights out environments. Systems engineers and Services engineers will be using these large labs to do everything from prototype component testing to troubleshooting customer problems. They're designed with ergonomics and frequent physical movement of equipment in mind. A lot of the space in these labs is dedicated to benches where gear can be disassembled and tested easily.
Two huge cooling towers are the centerpiece of a mechanical plant (pictured here) that sits behind an office building facade just outside the labs. These 1000 ton chillers use adaptive frequency drives that change the frequency and voltage to optimize the load to what is sensed. Dynamic cooling within the data centers takes the cooling intelligence down to the rack level, and spot cooling minimizes unwanted air mixture.
One of the highlights of the tour was a look at the Hot Aisle Containment solutions by Liebert and APC. There were several installations of each, and Dean expected huge savings in cooling costs due to their efficient circulation of chilled air. The Liebert Solution has a higher total CFM, so was chosen for the more dense lab needs, while the APC Solution with some sensing and adjusting capability and lower maximum CFM were chosen for the less dense, or not always on environments. Dean's team will be generating benchmark data that compares both solutions once they get populated and running.
Watch this space for more on the magnitude of savings and reduction in greenhouse gas emissions as a result of the new Santa Clara labs project.
Related reading:
