Wednesday Sep 26, 2007
I'm seeing lots of bad comparisons abound in articles and tech analyst reports
around pricing and power.
I see them talking about system price by only referring to processor/cores, and then...
- not specifying memory size, or using lowest cost slow memory size
- not configuring reasonable size fast memory
- not specifying GHz (or talking Speed of at one GHz and cost of another)
I see them talking about importance of system wattage by only referring to processor TDP (Thermal Design Power of the chip, yes ignoring the whole system, making people assume the rest is all the same, which it isn't), and then...
- not specifying system wattage
- not configuring reasonable size fast memory
- not specifying wattage on actual performance benchmark runs.
included above are hints on how they can fix them.
Monday Sep 24, 2007
There is a ZDnet article that goes into issues with hosted datcenters called, ""Building a data center circa 2007"
It goes into power issues and now they are coming to dominate real estate (and performance) perspectives.
This clearly goes beyond hosted datacenters. All IT Management (or their bosses) should really focus on getting power budgets into IT so the right decisions are being made.
Thursday Sep 20, 2007
More good advice on datacenters was given by Mark Monroe(Sun) in at
eWEEK.com article entitled, "Finding the Economic Green in Green Computing" By Chris Preimesberger September 18, 2007. You can read about it at:
http://www.eweek.com/article2/0,1895,2184747,00.asp
In this blog we covered other important factors:
http://blogs.sun.com/bmseer/entry/the_register_needs_to_ask
and remember measure it in watts/performance, just like $/perf.
Friday Sep 14, 2007
I know AMD & Intel love to focus on the power used by CPUs
in their high-stakes battle to gain server chip dominance. Both
started talking TDP (Thermal Design Power) and getting people to judge
systems based on TDP.
...wait a minute, buying a system based on the power of a CPU is
a bit of misdirection unless the CPU is most of the power. This isn't
the case any more. So CPU TDP should be ignored, unless you are
designing your own product and are just buying CPUs.
First of all system power is what datacenters care about, so Intel and
AMD should be talking about system power in realistic memory configurations.
Memory draws lots of power these days.
Second,TDP was created so the manufactures of servers manufacturers
would know much power the chip consumes in worst-case maximum-power cases
so they could design power supplies, cooling, etc. That just isn't
useful to datacenter managers.
AMD's marketing only slightly improved the situation by telling
customers of SYSTEMS to look at the processor's ACP (Average CPU
Power).
Two problems:
- Focus on CPU power to avoid talking system power, but system power is
what one needs to know, in average case to estimate electrical bills.
- Focus on average CPU power not server maximums that datacenters need
to design cooling on (see: http://blogs.sun.com/bmseer/entry/watts_a_matter_with_their.
ACP of a CPU ...hmmm, do you know what a pain it is to just measure a CPU. AMD
in their whitepaper , has to isolate the power consumed by the processor
and that consumed by the motherboard -- this requires motherboard modifications and special instrumented server platforms!.
way to much work to get a marketing message, all we want is server power on a variety of memory configuations and full-speed CPUs and actually running at good datacenter utilizations!
WARNING: Everyone loves to talk performance of high-GHZ CPUs and low-power of low-GHz CPUs, so watch for the confusing marketing and much worse "bait&switch." Also watts per core is useless marketing, it is the watt/perf for a system that counts. Also any vendor trying to sell power-efficiency on high-performance
systems should report watt/performance along with their world record performance on that system.
Thursday Sep 13, 2007
The register really needs to start asking tougher questions about power.
A new article "Researchers: AMD less power-hungry than Intel"
by Austin Modine.
This article just takes the results and ignores the really tough
questions facing datacenters and power. I'll help here...
- What is the power draw of large memory configurations 16GB, 32GB, 64GB.
The 2GB to 8GB used in this report are tiny for those making decisions in many datacenters.
- If you have racks and racks of idle servers (even servers at
30% utilisation) you are major problems and should be counting on
new servers to save power!!!
- It is Watt/Performance (like $/perf) not perf/watt, as this highlights
efficiency.
- CPU utilisation needs to be reported, if you are measuring at
less than 60% you are wasting 2 TIMES to 5 TIMES more power per unit of work!
- Everyone needs to realize the variability of power measurements and not report differences as ##.#% - what? one simply doesn't have one-tenth of
1% reproducibility in power measurements of servers.
Focusing on small memory tries to focus on CPU power which is fine for chip vendors, but it is frankly no what customers I talk to care about.
Customers care about full system ("car
not piston performance").
The results show that under certain configurations and load levels, the Intel server was 2% to 12% per cent more power efficient. But in a majority of cases, the AMD server was 9% to 23% per cent more efficient.
These are tiny differences. Not enough testing was done to show if these
results are consistent (power measurement has big run-to-run variations due to
the complexity of most systems.
AMD server was 30% to 53% more power efficient. If accurate,
it's a noteworthy figure, considering many servers spend the most of their time waiting for work.
WHAT?!?!? Datacenters idle? this is insane, maybe in extremely poorly run datacenters.
Actually, if this the case, those people running the datacenter should find other jobs and
not run datacenters.
On the whole, NN&A's tests showed that Intel's power efficiency decreases as memory size increases. Conversely, AMD's power efficiency increases as the memory is upped.
Intel, of course, disputes the results.
"The report doesn't measure our latest Xeons, or quad cores," said Intel rep Nick Knupffer in an email. We have 2 GHz quad cores in the market at 50 watts, 12.5 per core!"
I'll agree with Intel's criticism about latest CPUs and QCs, but even in
the Intel statement they quickly talk about 2GHz QC and not the full
GHz ones. Everyone loves to talk performance of high-GHZ CPUs and low-power of low-GHz CPUs, so watch for the confusing marketing and much worse "bait&switch." Also watts per core is
useless marketing, it is the watt/perf for a system that counts.
NN&A's white paper http://www.worlds-fastest.com/d.pdf/wfw991.pdf
Also take a look at: http://blogs.sun.com/bmseer/entry/saving_the_planet_one_datacenter
If you want to see big power savings (not the small percentages talked about above!) take a look at:
http://blogs.sun.com/ValdisFilks/entry/another_win_for_ecological_computing
...it is far too late, enough writing for today.
Wednesday Aug 22, 2007
Disingenuous Dell vs. HP comparison? Dell has a power comparison with measured data, cool. But they try to compare three 16GB HP Opteron systems versus two Dell Xeon 32GB systems. Quite the easy way to it appear that Dell draws less power, by comparing a total of 64GB for HP and 48GB for Dell. Source:
http://www.dell.com/downloads/global/power/dell2socket_vs_hp4socket_vmware.pdf
- 32GB 4-chip Dual-core 2.8GHz Opteron = 730 watts
- 16GB 2-chip Quad-core 2.66GHz Xeon = 445 watts
Really memory is taking lots more than CPUs, I think memory size
is more important than chip count. Prove me wrong, with data please!
Regardless, we have a few more wattage datapoints.
If Dell compared systems with the same size memories I would not have used the term "disingenuous."
Wednesday Aug 22, 2007
I could hear the cries in the blogosphere as I posted
the entry earlier today... "these graphs are based on what?"
...actually there were based on a real system, but I can't share the data.
I searched and found the following public data(only 8 months old):
http://www.anandtech.com/printarticle.aspx?i=2892
they have graphs that show:
- workload1: two 2.6GHz Opt 2218 8GB DDR2 that go from
260watts(100% util) down to 165 watts (20% util)
- workload2: two 2.6GHz Opt 2218 8GB DDR2 that go from
275watts(100% util) down to 190 watts (20% util)
- workload1: two 3GHz Xeon 5150 8GB FB-DIMMS that go from
325watts(100% util) down to 250 watts (20% util)
- workload2: two 3GHz Xeon 5150 8GB FB-DIMMS that go from
325watts(100% util) down to 270 watts (20% util)
...Not hard to imagine going from 400watts with the latest power
savings to 200 watts at 0% util, is it?
They did make a mistake by graphing perf/watt instead of watt/perf.
looking at their graphs we can make a table of the watt/perf:
watt/performance workload1="DVD Store", 8GB servers
| %Util | 3 GHz Xeon
2chip woodcrest | 2.6GHz Opteron
2chip 2218 |
| 20% | 37 watt/perf | 33 watt/perf |
| 40% | 22 watt/perf | 18 watt/perf |
| 60% | 14 watt/perf | 13 watt/perf |
| 80% | 11 watt/perf | 11 watt/perf |
| 100% | 9 watt/perf | 11 watt/perf |
Not a huge difference between systems, but a huge difference between
the same system running at 20% util vs. 60% util for example. In
either case it is 2.6x more wasteful running at 20% util vs. 60% util.
That means you use 2.6x more watts!!!
This data is actually worse than what I showed.
Get eco smart! Sure, turn on the power saving features. But if you
really want save $ and be a hero you must
put effort into driving up utilisation.
I've been talking about single servers, but this is not about going one by one through all of your servers, any datacenter has RACKS and RACKS of servers, which should make it easier to drive up the utilisation en masse.
Thursday Jun 14, 2007
While this strikes me as very cool, it also bothers me a bit because we can all take
action now, that could make a bigger difference
"Climate Savers Computing Initiative (CSCI), has been joined by most of the biggest names in the computing industry, including Microsoft, IBM, HP, AMD, Dell and Sun, among others. The expressed aim is to make all computers being produced 90% efficient by 2010." ITwire: http://www.itwire.com.au/content/view/12869/1023/
Datacenter managers can save money even before all of this technology,
and it is because they can change their policies.
New Mantra:
"I'm going to run my servers at 10% higher utilisation and turn
off some old servers" (or even a higher % goal)
or...
"I'm going to consolidate and turn off some old servers"
See the math...
http://blogs.sun.com/bmseer/entry/the_total_tyranny_of_low
http://blogs.sun.com/bmseer/entry/total_tyranny_of_low_utilization
Then go home and buy some CFLs, and turn off your lights when not in use... 
Wednesday Mar 28, 2007
It is best to write out the equations (as reminded by
Pavel).
- Performance = operations/time (ops/sec)
- Watt = Energy/time (Joules/sec)
- Watt-hour= kWatt*hours = 1000 (Joules/sec) * hour, basically
we are paying for a certain number of Joules.
So to derive a proper metric on the energy used per operation.
Watt/Performance =
=(joules/sec)/(Ops/sec)
= joules/Op or energy per operation
therefore
Watt/Performance = Energy/operation
Energy is measured in kWatt*hour and that is what we pay money for.
This then is in line with our typical cost metric of $/operation.
To remind everyone the industry has settled on $/performance.
This is no mistake. Like it or not we live in
a cost-constrained world, that is why the ratio was constructed
the way it was with performance or operation in the denominator.
This way one is naturally led to what is the cost per transaction, as
opposed to the thought experiment if I had 'X' dollars what
performance could I achieve.
Tuesday Jan 09, 2007
"Power is now 6% to 7% of datacentre costs compared to 1% a few...years ago,” Ken Harvey CIO of HSBC.
So HSBC is looking at a variety of ways to save money, one of those is
implementing with the Sun Fire T2000.
"Any savings on power consumption will have an immediate financial gain for the bank. HSBC is rolling out Sun T2000 servers, which are said to deliver up to three times the throughput at 30% less power and cooling costs compared to alternative server hardware."
you can read more in the Computer Weekly article 4-Jan-07:
http://www.computerweekly.com/Articles/2007/01/04/220980/how-to-make-money-by-going-green.htm
It is important to note that at high utilisation and with lots of memory the Sun Fire T2000 out-distances the Woodcrest-based systems and others by a lot, see previous blog entries on that in this "Wattage-Power" category.
Expect more news on the Sun Fire T2000 later today...
Monday Dec 18, 2006
As the BM Seer talked about in http://blogs.sun.com/bmseer/entry/wattage_charts_to_think_about
huge saving can be made by increasing utilization on servers. These
gains can be seen in real data centers.
For example take a look at the blog posting by an
Infrastructure Coordinator for a large Dutch IT Services provider
http://virtualize.wordpress.com/2006/12/18/power-savings-through-virtualization/.
This was also mentioned in Sun's Peter Jenkin's blog.
By the way the Solaris virtualization
features can create the same savings. Since the various Solaris virtualization
features are designed to be extremely efficient (as we've shown in the past
for example: http://blogs.sun.com/bmseer/entry/virtually_no_overhead_solaris_zones) you may even save more with the near zero% overhead of Solaris.
Monday Dec 11, 2006
I've seen a variety of figures, anyone have completely vetted data?
Here is what I've been able to find. Note the first table
slices and dices that data differently than the second two tables:
| Stage |
Watts |
| Power our of utility |
1000 |
| Power lost in distribution |
29 |
| Power for HVAC |
169 |
| Power for distribution |
224 |
| Power to servers |
578 |
Another source shows the following for a datacenter that
has lots of power for infrastructure:
| Use |
Watts |
| Power into Datacenter |
1000 |
| Power lost UPS |
90 |
| Power lost in HVAC (air) |
160 |
| Power lost in chiller & pumps |
340 |
| Lighting |
80 |
| Power to servers |
330 |
Another source shows the following for a datacenter that
that uses little power for infrastructure:
| Use |
Watts |
| Power into Datacenter |
1000 |
| Power lost UPS |
50 |
| Power lost in HVAC (air) |
50 |
| Power lost in chiller & pumps |
160 |
| Lighting |
10 |
| Power to servers |
730 |
...what are your figures? Any other data sources that anyone can point me to. If you have data, please add your comments.
Tuesday Dec 05, 2006
Some things to look at when you seen marketing around wattage. You
can avoid errors by really looking at total measured wattage when systems
running and doing real work. I've seen a lot of Intel marketing about
wattage of Woodcrest being 65 watts. But that really doesn't show the
whole picture. I'll break it down a bit...
What GHz at what wattage?:First recognize that Woodcrest
2.66 GHz & 2.33 GHz is 65 watts for chip only, but Woodcrest at 3.0 GHz
is 80 watts. ...and all benchmarks I've seen is on the 80 watt 3.0 GHz
systems.
What about the memory controller?: The CPU isn't everything.
Woodcrest designs have an external memory controller. Opteron designs have
an integrated memory controller. So you need to add another 30 watts (or more)
for the pair of Woodcrest CPUs.
What about the memory technology differences?: The CPU+Memory_controller
isn't everything. Woodcrest designs use FB-DIMMs. Opteron designs use the
more power efficient DDR2. FB-DIMMS draw a lot more power. In fact, as
I've blogged about before, 32GB 2-socket Woodcrest system draws 500 watts!
Measured when the CPU is busy. Sun's Opteron systems is way over 100 watts less.
Every IT department I talk to really wants to cut cost out -- power consumption
is a growing a major factor in IT costs.
...this just in...
Sun is now shipping a wattage meter with the "Try-and-buy" program for
Sun Fire T2000. More details at:
http://blogs.sun.com/cohen/entry/kill_a_watt_--_power
Thursday Nov 02, 2006
Getting the truth out on Woodcrest. BM Seer getting top ranking in
google on woodcrest and watts. A search for "woodcrest wattage" and "Woodcrest watts" gets top two entries. Even a google search for "woodcrest"
gets the BM Seer blog entry in at #27.
Seems lots of people are surprised that
a 32GB 2 Woodcrest server draws a hefty 510 watts - which is an awful lot in 2RU. Why didn't the Woodcrest vendors just publish this at product launch? Global information age means the info will get out. Transparency is what Jonathan, CEO of Sun Microsystems, always talks about.
...added note
Xray's comment is such FUD, 430 watts was measured with almost no disk on a
real workload on only a 16GB woodcrest -- simply put, woodcrest wastes 100watts more than a 32GB T2000, and 2x more watts than a T1000. Woodcrest supporters offer no measured wattage on 16GB or 32GB systems, but try to undermine the facts...
If you have a woodcrest measure the watts and post them, clearly woodcrest vendors don't want you to know.
Woodcrest 16GB 430 watt measured:
Dell 2950
2 x 3GHz Woodcrest Xeon 5160 (4MB L2 cache)
16GB = 8 x 2GB DIMM;
one 73GB 15K rpm SAS (disk idle)
1.333MHz FSB
PERC 5/i, x6 Backplane Integrated Controller Card
QLogic 2462 Dual Channel 4GB Optical FC HBA PCI-E
OS: SuSE - SLES
all bios settings correct
|
|
|
|
Unless the benchmarks require "reasonable siz...
That is not might point. I'm tired of the flow th...