Wednesday Jan 07, 2009
Sun often leads in publishing new benchmarks, now we have to wait for
other vendors to publish, and probably a LOT longer for those same vendors to published measured watt with full-configuration performant benchmark results.
Sun has published the best SPECmail2008 to date on the Sun
Fire X4540 server, demonstrates Sun's leadership position in Mail
Serving. The results highlights the features of the Sun Fire X4540
single system, which includes 48x internal drives [250GB SATA]
in a 4RU packaging, and it was obtained using the highly
available ZFS file system, offering continuous integrity
checking and automatic repair.
The Sun Fire X4540 server, equipped with two quad-core 2.3GHz
Opteron 2356, Sun Java System Messaging Server
6.2 JES4, the ZFS File System and Solaris 10 Operating System, has
achieved the overall record of 3300 SPECmail_MSEnt2008 SMTP and IMAP
users at 15,868 SPECmail2008 Sessions/hour.
This benchmark result clearly demonstrates that a single
Sun's X4540 with integrated storage, together with the Sun Java
System Messaging Server 6.2 with ZFS on Solaris can support a
large, enterprise level IMAP Mail server environment at the highest
level of throughput performance, representing a compact, low cost
and low power solution.
The Sun Fire X4540 server with 32GB of memory and forty-eight
250GB SATA Internal Disks had an average power consumption of
924 Watts, measured during the SPECmail2008 benchmark steady state.
Published Benchmark Results, SPECmail2008 (ordered by performance)
|
System
|
Processors
|
Performance
|
|
Ch
|
Co
|
GHz
|
Type
|
SPECmail_MSEnt2008 Users
|
SPECmail2008 Sessions/hour
|
|
Sun Fire X4540
|
2
|
8
|
2.3
|
2356
|
3300
|
15,868
|
|
Sun Fire X4200 M2
|
2
|
4
|
2.8
|
2220 DC
|
2500
|
12,019
|
Complete benchmark results may be found at the SPEC benchmark
website http://www.spec.org
Note: Number of SPECmail_MSEnt2008 users (bigger is better),
SPECmail2008 Sessions/hour (bigger is better) , Ch/Co: Chips, Cores
Benchmark Description
SPECmail2008 is an industry standard client server benchmark
designed to measure a system's ability to act as a mail server
compliant with the Internet standards Simple Mail Transfer Protocol
(SMTP) and Internet Mail Application Protocol, Version 4 (IMAP4). The
benchmark models business user behavior by simulating a real world
workload experienced by enterprised based email services. The goal of
SPECmail2008 is to enable objective comparisons of mail server
products.
Software on one or more client machines generates a benchmark load
for a System Under Test (SUT) and measures the SUT response times. A
SUT can be a mail server running on a single system or a cluster of
systems.
A SPECmail2008 'run' simulates a 100% load level associated with
the specific number of users, as defined in the configuration file.
The mail server must maintain a specific Quality of Service (QoS) at
the 100% load level to produce a valid benchmark result. If the mail
server does maintain the specified QoS at the 100% load level, the
performance of the mail server is reported as SPECmail_MSEnt2008 SMTP
and IMAP Users at SPECmail2008 Sessions per hour. The
SPECmail_MSEnt2008 users at SPECmail2008 Sessions per Hour metric
reflects the unique workload combination for a SPEC IMAP4 user.
Disclosure Statement:
SPEC, SPECmail reg tm of Standard Performance Evaluation
Corporation. Results as of 1/6/09 on http://www.spec.org. SPECmail2008:
Sun Fire X4540 (8 cores, 2 chip) 3300 SPECmail_MSEnt2008 users at 15,868 SPECmail2008 Sessions/hour. Sun Fire
X4200 M2 (4 cores, 2 chip) 2500 SPECmail_MSEnt2008 users at 12,019
SPECmail2008 Sessions/hour.
Results Summary
|
Performance: |
|
3300 SPECmail_MSEnt2008 users at 15,868 IMAP SPECmail2008
sessions/hour |
|
Reference Date: |
|
December 2, 2008 |
| Systems: |
|
Sun Fire X4540 |
| Total Number Sockets: |
|
2 |
| Total Number Cores: |
|
8 |
| Processor/GHz of Server: |
|
AMD Opteron 2356 DC 2.3GHz |
| Operating System: |
|
Solaris 10 5/08 |
| Mail Server: |
|
Sun Java System Messaging Server 6.2 JES4 |
Friday Jun 20, 2008
miles/gallon is as misleading to consumers! Remember when I said perf/watt is misleading. How do we all avoid these 'math illusions'? Duke University researchers tell us this is simple, just "flip 'em"
Posting a vehicle’s fuel efficiency in “gallons per mile” (GPM) rather than “miles per gallon” (MPG) would help consumers make better decisions about car purchases and environmental impact, researchers from Duke University’s Fuqua School of Business report in the June 20 issue of Science magazine.
Video of Larrick & Soll discussing their research:
click here
Article on Larrick & Soll’s research, which was funded by Duke University.
Check out the above video, you can see that people try to judge by linear improvement in miles/gallon, but this is very misleading. The recommend that we switch to gallons/mile!
Remember back in March 2007, where I said the metric is watt/performance and not perf/watt. http://blogs.sun.com/bmseer/entry/power_efficiency_metrics_clearing_up. Time for SPEC to reconsider their metrics, and only allow default settings to be measured in benchmarks (if power-management is not on by factory default it should NOT be measured in a test - that way customers are best served.
Improving inefficient cars saves a lot of gas, the same valid reasoning shows improving %utilisation IS the big win especially when coupled with efficient servers.
Nothing like a little vindication to start the weekend, OK it's getting late, cya next week 
for a table on savings at differnet miles/gallon see:
http://www.fuqua.duke.edu/news/mpg/table.pdf
Friday May 30, 2008
The SPECpower_ssj2008 benchmark goes out of its way to measure servers at low-utilisation (5 of 11 datapoints are in the wasteful active-idle to 40% range - the worst range for servers). SPECpower_ssj actually shows this. Customers needs to demand that power measured on all benchmarks, now.
Let's look at HP DL580 G5 4-socket (for SPECpower_ssj2008 HP only used the low-GHz 1.86GHz Xeon, tiny 16 GB with special DIMMs, and a hacked non-standard BIOS). Regardless it can still be used to prove my point.
Increasing utilisation even a small amount provides HUGE improvements in watts-per-unit-of-work. Increasing utilisation a tiny 10% improve your watt/work an amazing 33-163% ! If you increase utilisation more you save even more:
| %Util Improvement | Savings in watts-per-unit-of-work |
| 10 | 33-163% |
| 20 | 45-257% |
| 30 | 62-353% |
<| 40 | 84-437% |
| 50 | 114-511% |
| higher% | even more! |
The biggest savings occur when you stop running at low utilisation.
We need to do everything we can to discourage low-utilisation!
The first column, in the table below, is the %utilisation you start at, the rows than show you the %savings if you increase your utilisation. So for example if you were at 20% utilisation (2nd row from the bottom) and increase the %utilisation to 40% (3rd column from the right) you save 140% per unit of work!
| %u | Perf/Power | 100% | 90% | 80% | 70% | 60% | 50% | 40% | 30% | 20% |
| 100% | 929 | | | | | | | | | |
| 90% | 867 | 33% | | | | | | | | |
| 80% | 793 | 45% | 35% | | | | | | | |
| 70% | 712 | 62% | 50% | 37% | | | | | | |
| 60% | 626 | 84% | 71% | 56% | 41% | | | | | |
| 50% | 538 | 114% | 99% | 82% | 64% | 46% | | | | |
| 40% | 451 | 155% | 138% | 117% | 96% | 75% | 54% | | | |
| 30% | 357 | 223% | 200% | 174% | 148% | 121% | 94% | 64% | | |
| 20% | 243 | 374% | 341% | 303% | 264% | 224% | 185% | 140% | 89% | |
| 10% | 129 | 793% | 731% | 659% | 586% | 511% | 437% | 353% | 257% | 163%
| |
IBM bloggers accuse me of many things. LET ME BE CLEAR: These are my personal opinions and NOT the opinions of Sun. This blog is NOT the source for official opinions.
Disclosure statement
SPECpower_ssj2008:HP Proliant DL580 G5 (4-chip QC Xeon L7345 1.86GHz), 546 overall ssj_ops/watt,
359,523 ssj_ops and 387 watt at 100% target load,
325,931 ssj_ops and 376 watt at 90% target load,
291,991 ssj_ops and 368 watt at 80% target load,
255,512 ssj_ops and 359 watt at 70% target load,
217,222 ssj_ops and 347 watt at 60% target load,
180,262 ssj_ops and 335 watt at 50% target load,
145,079 ssj_ops and 322 watt at 40% target load,
110,173 ssj_ops and 309 watt at 30% target load,
71,409 ssj_ops and 294 watt at 20% target load,
36,070 ssj_ops and 280 watt at 10% target load, and Active Idle 271 watts.
SPEC, SPECpower reg tm of Standard Performance Evaluation Corporation.
Results from www.spec.org as of 12/11/07.
In a more realistic configuration the HP DL580 G5, from HP's own power calculators, a HP DL580 G5 with four QC Xeon 2.93GHz Tigerton and 64 GB memory should draw 1,072watts. HP DL580 power consumption from HP Power Calculator system configured with 4 x2.93GHz processors, redundant PSU, 16 x 4GB DIMMs, 8 x 36GB SAS drives,1 x PCI card, 80% utilisation on 9/10/07: http://h30099.www3.hp.com/configurator/powercalcs.asp
Thursday Jun 14, 2007
While this strikes me as very cool, it also bothers me a bit because we can all take
action now, that could make a bigger difference
"Climate Savers Computing Initiative (CSCI), has been joined by most of the biggest names in the computing industry, including Microsoft, IBM, HP, AMD, Dell and Sun, among others. The expressed aim is to make all computers being produced 90% efficient by 2010." ITwire: http://www.itwire.com.au/content/view/12869/1023/
Datacenter managers can save money even before all of this technology,
and it is because they can change their policies.
New Mantra:
"I'm going to run my servers at 10% higher utilisation and turn
off some old servers" (or even a higher % goal)
or...
"I'm going to consolidate and turn off some old servers"
See the math...
http://blogs.sun.com/bmseer/entry/the_total_tyranny_of_low
http://blogs.sun.com/bmseer/entry/total_tyranny_of_low_utilization
Then go home and buy some CFLs, and turn off your lights when not in use...
Wednesday Mar 21, 2007
Yesterday I posted about the power metric you should be using watt/performance, this
fits with $/performance metrics you are familiar with. Performance has to be in
the denominator. Watts cost money so it should be on top. Notice if you have
2.3 watts/performance another system that draws 4.6 watts/performance will end up
costing twice as much.
OK, I told you that I was tired when I wrote the last entry, I got a little sloppy.
The careful reader would notice that my metric is a bit off.
watt/performance is a quick and good heuristic for comparison. But...
To have a proper metric the careful reader would
have noticed that on your electric bill you don't buy watts, one buys kilowatt-hours (or watt-hours).
Then to really perfect a power efficiency metric for datacenters one needs to change the denominator
to highlight the focus on performance or work completed.
If you are buying for performance(ops/sec) or
work-done(ops) then you need to put this in the denominator.
Which bring us to:
Metrics:
Performance: watt-hour/(ops/sec)
Work: watt-hour/ops
... or kWhr/k-ops/sec or kWhr/m-ops/sec to scale it correctly.
|
|
|
|
Even better would be liter/kilometer
I should cocoa! ...but you know these 'mericans.