Wednesday Feb 25, 2009
More support for Solaris:
February 25, 2009 - Sun and HP announced an expanded multi-year partnership agreement that enables HP to distribute and support Sun's Solaris 10 OS. The top five x86/x64 based system vendors now all ship Solaris with their systems. HP is responding to customer demand for expanded OS support on HP ProLiant server and blade platforms. Sun now becomes a strategic HP ProLiant OS distribution partner and Solaris is elevated to the lineup of key operating environments for the ProLiant platform.
Read more at:
http://www.sun.com/aboutsun/media/presskits/2009-0225/index.jsp
Why do I mention this... I know some customers who are going to use this to solve some performance issues.
Wednesday Feb 18, 2009
Even Dell is starting to get it. Server utilisation levels in a data centre are critical to advance overall performance, improving productivity, and reducing costs.
In: http://www.dell.com/downloads/global/power/ps1q09-20090176-esser.pdf
They say:
Operational policies designed to increase server
utilization and advance overall performance and
efficiency can lead to dramatic improvements in data
center productivity without increasing power
consumption.
Yes indeed, I've been blogging that since 2005, remember this little
gem from 2006:
http://blogs.sun.com/bmseer/entry/the_total_tyranny_of_low
But I ran into several customers last week where a major computer vendor (not Dell or Sun) was telling everyone that 10% server utilisation was the most important utilisation to measure - SHAME ON YOU!
Can a group at SPEC or TPC just get on with adding power measurement to all benchmarks as they exist now at benchmark utilisation levels? ...Or just what is going on behind closed doors?
DISCLOSURE: I'm not on any SPEC or TPC committees, nor do I read any confidential updates from Sun employees who do.
Wednesday Feb 18, 2009
At lunch a friend told me about the launch of Galaxy Zoo 2. Galaxy Zoo 2 is a project where you can help by looking at actual images of galaxies taken by the Sloan Digital Sky Survey (SDSS) robotic telescope and then online helping to classify about a quarter of a million galaxies. This is a group effort you classify some, others help...
More than 150,000 people have taken part in Galaxy Zoo so far, producing a wealth of valuable data and sending telescopes on Earth and in space chasing after their discoveries.
The primary goal of Galaxy Zoo 2 is to construct a database of detailed shape information for the largest sample of galaxies ever assembled. Such a database will have substantial legacy value for the international astronomy community. In short, Galaxy Zoo 2 hopes to find out everything there is to know about the appearance of galaxies!
How to help? see:
http://galaxyzoo.org/
Article on Galaxy Zoo: http://news.bbc.co.uk/nolpda/ukfs_news/hi/newsid_7746000/7746298.stm
Friday Feb 13, 2009
Sun SPARC Enterprise T5140 and T5440 Delivers Outstanding Application and Database Performance on SPECjAppServer2004. Yet again, Sun is showing
measured watts on another benchmark. I encourage all other vendors to do the same on all benchmarks. We all need this kind of transparency!
Four Sun SPARC Enterprise T5140 Servers and 1 Sun SPARC Enterprise T5440
server delivered a result of 9500.76 SPECjAppServer2004 JOPS@Standard. ZFS was also used in this benchmark.
One Sun SPARC Enterprise T5140 server in the application tier, consumed on
average 614 Watts of power and the Sun SPARC Enterprise T5440 server in
the database tier, consumed on average 1836 Watts of power during the
execution of this benchmark.
This benchmark used the Oracle WebLogic 10.3 Application Server and
Oracle Database 11g Enterprise Edition. This benchmark result proves that
the Sun SPARC Enterprise T5440 server using the UltraSPARC T2 Plus
processor performs as an outstanding Oracle 11g OLTP database server.
The Sun SPARC Enterprise T5140, T5440 and M3000 servers used to produce
this benchmark result all used the Solaris 10 10/8 Operating Environment.
Four Sun SPARC Enterprise T5140 Servers and 1 Sun SPARC Enterprise T5440
server demonstrated better performance compared to the HP result of
9459.19 SPECjAppServer2004 JOPS@Standard which used 11x HP BL860c
servers and 2x HP Superdomes.
Each Sun SPARC Enterprise T5140 server used 4 instances of Oracle WebLogic
10.3 and the Sun JVM 1.6.0_06 Performance Release in Solaris Containers.
Each Sun SPARC Enterprise T5140 server used ZFS to mirror 2 Solid State
Disks to meet the benchmark durable storage requirements for the
application server logs and JMS Persistence filestore.
The Sun SPARC Enterprise T5440 server in the database tier used Oracle
Automatic Storage Management (ASM) to manage the StorageTek 2540 and 2501
storage arrays for the database files and redo logs.
The Sun result of 9500.76 SPECjAppServer2004 JOPS@Standard using one
Sun SPARC Enterprise T5440 in the database tier used 24 Oracle licenses
for the database. The HP result of 10519.43 SPECjAppServer2004 JOPS@Standard
using one HP Superdome in the database tier used 40 Oracle database licenses.
The Sun T5440 delivered 90% of the performance using 60% of the database
licenses compared to the HP Superdome.
The Sun result of 9500.76 SPECjAppServer2004 JOPS@Standard using one
Sun SPARC Enterprise T5440 in the database tier used 4 Rack Units of
space (H x W x D = 7" x 18" x 25" = 1.8 cu feet). The HP result of
10519.43 SPECjAppServer2004 JOPS@Standard using the HP Superdome database
(a 40 processor partition in 2x A9834A cabinets), have space requirements
of (H x W x D = 72" x 48" x 45" = 90 cu feet). The Sun T5440
occupies 1/50 of the datacenter space at 90% of the performance of an
HP Superdome.
The Sun result of 9500.76 SPECjAppServer2004 JOPS@Standard using one
Sun SPARC Enterprise T5440 in the database tier consumed on average
1836 watts of power during the execution of this benchmark. The HP result
of 10519.43 SPECjAppServer2004 JOPS@Standard using the HP Superdome
database (a 40 processor partition in 2x A9834A cabinets), have power
requirements of 13720 watts(1) or 7.5 TIMES more than Sun's T5440. The Sun T5440 consumes 14% of the
power at 90% of the performance of an HP Superdome.
Performance Comparisons
SPECjAppServer2004 Performance Chart as of 02/04/2009. Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org.
SPECjAppServer2004 JOPS@Standard (bigger is better)
| Submitter |
SPECjAppServer2004
JOPS@Standard |
J2EE Server |
DB Server |
| HP |
10519.43 |
12x HP BL860c
4 cores, 2 chips @ 1.66 GHz Itanium 9100
Oracle OC4J 10.1.3.3.2 |
1x HP Superdome
40 cores, 20 chips @ 1.6 GHz Itanium 9000
Oracle 10g DB 10.2.0.3 |
| Sun |
9500.76 |
4x Sun SPARC Enterprise T5140
16 cores, 2 chips @ 1.2 GHz US-T2 Plus
Oracle WebLogic 10.3 |
1x Sun SPARC Enterprise T5440
32 cores, 4 chips @ 1.4 GHz US-T2 Plus
Oracle 11g DB 11.1.0.7 |
| HP |
9459.19 |
11x HP BL860c
4 cores, 2 chips @ 1.66 GHz Itanium 9100
Oracle OC4J 10.1.3.3.2 |
2x HP Superdome
80 cores, 40 chips @ 1.6 GHz Itanium 9000
Oracle 10g DB 10.2.0.3 with RAC |
| Sun |
8439.366 |
6x Sun SPARC Enterprise T5120
8 cores, 1 chip @ 1.4 GHz US-T2
Sun Java System Application Server |
1x Sun SPARC Enterprise E6900
48 cores, 24 chips @ 1.95 GHz US-IV+
IBM DB2 V9.1 |
Benchmark Description
SPECjAppServer2004 (Java Application Server) is a multi-tier benchmark for
measuring the performance of Java 2 Enterprise Edition (J2EE) technology-based
application servers. SPECjAppServer2004 is an end-to-end application which
exercises all major J2EE technologies implemented by compliant application
servers as follows:
-
The web container, including servelets and JSPs
-
The EJB container
-
EJB2.0 Container Managed Persistence
-
JMS and Message Driven Beans
-
Transaction management
-
Database connectivity
Moreover, SPECjAppServer2004 also heavily exercises all parts of the underlying
infrastructure that make up the application environment, including hardware,
JVM software, database software, JDBC drivers, and the system network.
The primary metric of the SPECjAppServer2004 benchmark is jAppServer Operations
Per Second (JOPS) which is calculated by adding the metrics of the
Dealership Management Application in the Dealer Domain and the Manufacturing
Application in the Manufacturing Domain. There is NO price/performance
metric in this benchmark.
Disclosure Statement:
SPECjAppServer2004: 4x Sun SPARC Enterprise T5140 (8 chips, 64 cores) 9500.76 SPECjAppServer2004 JOPS@Standard.
12x HP BL860c (24 chips, 48 cores) 10519.43 SPECjAppServer2004 JOPS@Standard.
12x HP BL860c (22 chips, 44 cores) 9459.19 SPECjAppServer2004 JOPS@Standard.
6x Sun T5120 (6 chips, 48 cores) 8439.36 SPECjAppServer2004 JOPS@Standard.
SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation.
Results from www.spec.org as of 1/28/09.
1. HP Integrity Superdome using 2x A9834A cabinets. Taking 70% of Typical
Input power of 9800 watts for an 8-cell cabinet and from:
http://h18000.www1.hp.com/products/quickspecs/11717_div/11717_div.HTML
See Also:
SPECjAppServer2004 Results Page
Results Summary
| Published Results |
|
9500.76 SPECjAppServer2004 JOPS@Standard |
| Reference Date: |
|
Feb 4, 2009 |
| Systems: |
|
4x Sun SPARC Enterprise T5140 |
|
|
1x Sun SPARC Enterprise T5440 |
|
|
1x Sun SPARC Enterprise M3000 |
| Total Number Processors: |
|
2, 4, 1 |
| Processor/GHz of Server: |
|
UltraSPARC T2 Plus 1.2 GHz |
|
|
UltraSPARC T2 Plus 1.4 GHz |
|
|
SPARC64VII 2.52 GHz |
| Operating System: |
|
Solaris 10 10/08 |
| Software: |
|
Oracle WebLogic 10.3 Application Server, Standard Edition |
|
|
Oracle Database Enterprise Edition Release 11.1.0.7 |
| JVM: |
|
JDK 1.6.0_06 Performance Release |
note: ...as always you can post comments anonymously, but if you work for a system's vendor you should state who it is.
Thursday Feb 12, 2009
The power of the Sun Storage 6780 Array product
coupled with our 4Gb HBAs has demonstrated industry class leading
SPC-2 benchmarking capabilities: Best-in-class $/performance, Half Price of IBM.
Sun Storage 6780 Array has best-in-class $/performance of $53.61
(RAID5) and $55.25 (RAID6) beating IBM by almost 50%
Sun Storage 6780 Array has best-in-class performance of 4,818.43
(RAID5) and 4,675.50 (RAID6) SPC-2 MBPS.
The Sun Storage 6780 Array has a 1.7X better price/performance
advantage over the same performing IBM system.
The Sun Storage 6780 Array delivers the best
SPC-2 performance of any dual controller system.
The Sun Storage 6780 Array delivers the same performance as
the IBM system, but at nearly half the price.
The Sun Storage 6780 Array delivers the best SPC-2 performance
of any sub-$250K system, the best performing systems are over
$1.6M in SPC-2 total price and are not even 2X in performance.
SPC-2 Performance Chart (in increasing price-performance order)
| System |
SPC-2
MBPS |
$/SPC-2
MBPS |
ASU (GB) |
TSC Price |
Data Protect-
ion Level |
Date |
Result Id |
| Sun SS6780 |
4,818.43 |
$53.61 |
16,383.186 |
$236,790 |
RAID 5 |
2/3/09 |
B00039 |
| IBM DS5300 |
4,818.43 |
$93.80 |
16,383.186 |
$451,986 |
RAID 5 |
9/25/08 |
B00037 |
| Sun SS6780 |
4,675.50 |
$55.25 |
14,042.731 |
$236,790 |
RAID 6 |
2/3/09 |
B00040 |
| IBM DS5300 |
4,675.50 |
$96.67 |
14,042.731 |
$451,986 |
RAID 6 |
9/25/08 |
B00038 |
| Fujitsu E8000 |
3,480.68 |
$238.93 |
4,569.845 |
$831,649 |
Mirroring |
3/8/07 |
B00019 |
SPC-2 MBPS = the Performance Metric
$/SPC-2 MBPS = the Price/Performance Metric
ASU Capacity = the Capacity Metric
Data Protection = Data Protection Metric
TSC Price = Total Cost of Ownership Metric
Results Identifier = A unique identification of the result Metric
Complete SPC-2 benchmark results may be found at
http://www.storageperformance.org.
Benchmark Description
The SPC Benchmark-2™ (SPC-2) is a series of related benchmark
performance tests that simulate the sequential component of demands
placed upon on-line, non-volatile storage in server class computer
systems. SPC-2 provides measurements in support of real world
environments characterized by:
-
Large numbers of concurrent sequential transfers.
-
Demanding data rate requirements, including requirements for real time
processing.
-
Diverse application techniques for sequential processing.
-
Substantial storage capacity requirements.
-
Data persistence requirements to ensure preservation of data without
corruption or loss.
Disclosure Statement:
Sun Storage 6780 Array 4,818.43 SPC-2 MBPS, $/SPC-2 MBPS $53.61, ASU
Capacity 16,383.186GB, Protect RAID 5, Cost $258,329.00, Ident. B00039.
SPC-2, SPC-2 MBPS, $/SPC-2 MBPS are regular trademarks of Storage
Performance Council (SPC). More info www.storageperformance.org
Sun Storage 6780 Array 4,675.50 SPC-2 MBPS, $/SPC-2 MBPS $55.25, ASU
Capacity 14,042.731GB, Protect RAID 6, Cost $258,329.00, Ident. B00040.
SPC-2, SPC-2 MBPS, $/SPC-2 MBPS are regular trademarks of Storage
Performance Council (SPC). More info www.storageperformance.org
Results Summary
| Results |
|
System: |
|
Sun Storage 6780 Array |
|
Sun Storage 6780 Array |
|
Performance: |
|
4,818.43 SPC-2 MBPS |
|
4,675.50 SPC-2 MBPS |
|
Price/Performance: |
|
$53.61 $/SPC-2 MBPS |
|
$55.25 $/SPC-2 MBPS |
|
ASU Capacity: |
|
16,383.186 GB |
|
14,042.731 GB |
|
Data Protection Level: |
|
RAID 5 |
|
RAID 6 |
|
TSC Price: |
|
$258,329.00 |
|
$258,329.00 |
|
Results Identifier: |
|
B00039 |
|
B00040 |
|
Server: |
|
IBM Ssytem x3850 M2 |
|
IBM Ssytem x3850 M2 |
|
Operating System: |
|
Windows Server 2003 SP2 |
|
Windows Server 2003 SP2 |
See Also:
Sun Storage 6780 Array SPC-2 (RAID 5) Executive Summary
(6 pages, acrobat pdf)
Complete Sun Storage 6780 Array SPC-2 (RAID 5) Full Disclosure Report
(acrobat pdf)
Sun Storage 6780 Array SPC-2 (RAID 6) Executive Summary
(6 pages, acrobat pdf)
Complete Sun Storage 6780 Array SPC-2 (RAID 6) Full Disclosure Report
(acrobat pdf)
Storage Performance Council (SPC) Home Page
Ideas International Benchmark page
Tuesday Feb 03, 2009
Just saw this this morning, a good writeup on a SPEC benchmark.
Bryan Cantrill writes:
"The result is such a deformed monstrosity that -- like the index case of some horrific new pathogen -- its only remaining utility lies on the autopsy table: by dissecting SPEC SFS and understanding how it has failed, we can seek to understand deeper truths about benchmarks and their failure modes.
For more read:
"Eulogy for a benchmark: I come to bury SPEC SFS, not to praise it."
http://blogs.sun.com/bmc/entry/eulogy_for_a_benchmark
Basically I think there are some very good SPEC & TPC benchmarks, but I think there is also some partisanship
that needs to be removed to just do the right thing for the industry. See my previous posting for some ideas on good things to do.
I've covered the problems with ageing TPC-C, but maybe I should have used the same language that Bryan used. 
Short-stroking 224 15K RPM drives is the equivalent of fueling a dragster with nitromethane -- it is top performance at a price so high as to be useless off the dragstrip.
Does this apply to the system configurations used in SPECpower. hmmmm, looks like the same colour.
Tuesday Jan 27, 2009
This blog approaches 250,000 visits, first of all thanks I hope you
have all learnt a few things of value from this anonymous Sun employee. It
is not about me. But the stats are quite an honour.
I decided to have an open letter to the benchmark committees, here are my blue sky suggestions:
- Can you please have the exact words you need for a disclosure statement
clearly listed with any submission.
ex: SPEC Disclosure Statement:
Sun SPARC Enterprise X5220 (8 cores, 1 chip) 41847 SPECweb2005. SPEC, SPECweb reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of Dec 4, 2008.
ex: TPC Disclosure Statement:
Sun SPARC Enterprise M9000 118,573.3 QphH@1000GB, $23.38/QphH@1000GB, avail 09/10/08. TPC-H, QphH, $/QphH tm of Transaction Processing Performance Council (TPC). More info www.tpc.org.
That way one could just copy&paste from your form and not try to dig up
your rules. When one needed several different results for for comparisons
it would only take very simple editing.
- Make all vendors publish exact power as measured on the performance bragged about
for all benchmarks. No to make new benchmarks to show watts on current
benchmarks. We all must save energy now. Let the public see the data. This table should
be filled in: "New Table of published power-performance data"
- Only allow power-performance to be reported on systems with default
power-management software. We need power-management on by default for
all results. Also only allow standard BIOS (of course tell Sun "no" as well, as
I know Sun has published results with the same sorts of modifications).
The reason why is we need to move the industry in a way that saves energy without
having to turn "on" or "off" features on a per server basis. That is way to much work
in a complex datacentre. Standard & default should be the direction.
Note: If you are a TPC or SPEC committee member I would appreciate any
comments, these are blue sky ideas. But if you comment you owe it to
all of the readers to state your affiliations and vendor name if you
get are employed by a computer vendor.
Friday Jan 16, 2009
Here us an upated table of power-performance benchmarks, trying to hightlight easy to measure watts on many different standard benchmarks.
I put the number of published benchmarks that show watt-performance for Sun and use
"0" zero for those that have ZERO information on the measured watts used during the benchmark. What is behind this stonewall of red?
It is really funny that some vendors avoid showing easy-to-measure power on performance benchmarks with real configurations. Is it that they have only have a advantage on special configurations?
Power-Performance Benchmarks
| Benchmarks |
Sun |
IBM |
HP |
Dell |
Unisys |
| SPECweb/other-web |
Yes 6/1 |
0 |
0 |
0 |
0 |
| SPECjbb |
Yes: 5 |
0 |
0 |
0 |
0 |
| SPECjAppServer |
Yes: 5 |
0 |
0 |
0 |
0 |
| SPECmail |
Yes: 2 |
0 |
0 |
0 |
0 |
| SPEComp |
Yes: 2 |
0 |
0 |
0 |
0 |
| Lotus Domino iNotes |
Yes: 3 |
0 |
0 |
0 |
0 |
| Oracle's Siebel CRM 8.0 |
Yes: 2 |
0 |
0 |
0 |
0 |
| VMmark |
Yes: 2 |
0 |
0 |
0 |
0 |
| MySQL |
Yes: 1 |
0 |
0 |
0 |
0 |
| ERP (CDA/NDA needed) |
Yes: 3 |
0 |
0 |
0 |
0 |
| SPECpower |
SPECpower
Issues |
Yes |
Yes |
Yes |
0 |
Note customers can get "ERP" power data from Sun under CDA/NDA agreement, as some SW vendors do not allow public display of wattage data measured during the benchmark.
Want more details on Sun's published measured watts, last week I posted "Sun's 2008 summary of measured watts & watt/performance": http://blogs.sun.com/bmseer/entry/sun_s_2008_summary_of
Some other thoughts for those really interested in saving real energy:
- Power-management software should be on by default or NOT ALLOWED IN BENCHMARKS!
- Increase server utilisation to most effectively reduce power consumed! Servers are very very inefficient at low-utilisation!
- look at watt-performance NOT perf/watt when comparing servers!
- replace incandescents with CFL lights.
Disclosure Statement:
SPEC, SPECweb, SPECjbb, SPECjAppServer, SPEComp, and SPECpower benchmark name are registered trademarks of the Standard Performance Evaluation Corporation.
VMware(R) VMmark(R) is a product of VMware, an EMC Company.
Oracle, Siebel, registered trademarks of Oracle Corporation and/or its affiliates. More info www.oracle.com/apps_benchmark/html/white-papers-siebel.html
NotesBench R6iNotes More info: www.notesbench.org
Thursday Jan 15, 2009
eWeek has a new article "Sun VirtualBox a Solid Alternative to VMware, Parallels," where they say:
"Sun Microsystems’ xVM VirtualBox desktop virtualization software is an increasingly powerful, no-cost alternative to VMware Workstation and Parallels Desktop products and should be added to the consideration shortlist of software developers and IT managers."
Sun VirtualBox information on www.sun.com.
Wednesday Jan 14, 2009
The Sun Fire X4450 powered by four 6-core Intel Xeon X7460 processors
at 2.66 GHz, has achieved a score of 19.47 @14 tiles, supporting 84 fully fledged VM instances (14 Tiles). This result puts the SUN X4450 on the leading front of 4-socket/24 cores Intel-based VMware servers.
This results beats IBM x3850-M2 result to date of 19.10 @ 14 tiles
and all the other Intel based submissions in the 24 cores category
with similar architecture while using equal or smaller memory footprint (80GB) then the other systems.
The Sun Fire X4450 server, equipped with four 6C Xeon X7460, provides a highly scalable virtualization platform, that, in combination with the VMware ESX Server 3.5.0 hypervisor, takes full advantage of Intel Virtualization Technology.
To support 14 tiles per server , efficient and reliable Sun StorageTek 2540 Fibre Channel arrays with Sun StorageTek 2501 SAS expansion units were used. This family of arrays offers enterprise-class, reliable RAID-protected functionality in a high-density 2 RU package and is well suited for virtualization environments.
The electrical Power consumption measured for the 4-socket 6C 80GB server during the benchmark runs was on average: 933 Watts (Idle 750 Watts).
The electrical Power consumption for the data storage during
the benchmark runs for each disk array and workload type was on average:
-
1x STK2540 (database workloads): 304 Watts (Idle 296 Watts)
-
1x STK2540 (Tile workloads except database): 306 Watts (Idle 296 Watts)
-
1x STK2501 (Tile workloads except database): 261 Watts (Idle 253 Watts)
Sun is the first vendor (and only) to disclose actual power consumption measured while running VMmark, the industry's most popular virtualization workload.
VMmark Performance Results for 24 Cores, Benchmark Score (bigger is better)
| System |
Socket/
Core/
Thrds |
GHz Type |
GB |
ESX ver |
Spindle |
Mirr |
Tiles |
Score |
Date Pub |
| Sun Fire X4450 |
4/24/24 |
2.66 Xeon X7460 |
80GB |
3.5.0U2 |
192 |
n |
14 |
19.47 |
13-Jan-09 |
| IBM x3850 M2 |
4/24/24 |
2.66 Xeon X7460 |
80GB |
3.5.0U2 |
166 |
n |
14 |
19.10 |
17-Sep-08 |
| Dell PE R900 |
4/24/24 |
2.66 Xeon X7460 |
128GB |
3.5.0U2 |
140 |
n |
14 |
18.96 |
02-Dec-08 |
| HP PL DL580 G5 |
4/24/24 |
2.66 Xeon X7460 |
96GB |
3.5.0U2 |
168 |
n |
14 |
18.56 |
18-Aug-08 |
Complete benchmark results may be found at the VMware benchmark website http://www.vmware.com/products/vmmark/results.html.
Benchmark Description
VMmark is the first benchmark that was designed specifically to
quantify and measure the performance of virtualized environments.
It features a novel tile-based scheme for measuring the
scalability of consolidated workloads and provides a consistent
methodology that captures both the overall scalability and individual
application performance.
The VMmark benchmark is built on VMware's expertise in virtualization
performance and incorporates popular workloads from application
categories most commonly represented in customer data centers.
The purpose of this benchmark is to measure performance
and scalability of a pre-established mix of workloads (a Tile), which
allows comparisons among similar configuration platforms.
A Tile consists of 6 fixed workload applications,
each running in its own Virtual Machine (VM) (6 VMs per Tile)
such as Mail, Java, Web, Database and File Serving
plus a standby server which only purpose is to provide
a spare Virtual Machine that does not do any work and is
not accounted in the score.
VMmark benchmark provides two key performance metrics:
-
The number of tiles supported by a system,
which is an indication of how many systems/applications
can be consolidated on one platform
where the higher the number of tiles supported
the higher the number of consolidated systems.
-
The Score,
which is an overall measure of the amount of work that
is accomplished by all the Tiles in the system and
summarizes the level of service of all the workloads
during a benchmark run.
The score or amount of work is a composition of
Actions/minute(Mail server), New Orders/minute(Java server),
Access/minute(web server), Commits/minute(Database),
MB/second(file server).
Thus, among systems with the same number of tiles,
the system with the higher score is the system that is
capable of producing the greater amount of work.
For detailed description of VMmark, tiles and score definition, please refer to VMmark Features
Disclosure Statement:
VMware(R) VMmark(R) is a product of VMware, an EMC Company. VMmark
utilizes SPECjbb2005(TM) and SPECweb2005(TM), which are available from
the Standard Performance Evaluation Corporation (SPEC).
Results from http://www.vmware.com/products/vmmark/ as of January 13, 2009.
Result Information
| Certified Results |
|
Score: |
|
19.47@14-tiles |
|
Server: |
|
Sun Fire X4450 |
|
Processors: |
|
4-socket 2.66 GHz 6-core Intel Xeon X7460 |
|
Memory: |
|
80 GB |
|
VMware ESX Server: |
|
3.5.0 Update 2 |
|
VMmark: |
|
1.1 |
|
Storage: |
|
9 x STK2540, 7x STK2501, each storage array using 12x146GB 15K RPM disks |
Tuesday Jan 13, 2009
Sun's state-of-the-art datacenter in Santa Clara, California, has attracted more than 2,000 visitors from 350 customers. The big draw: power usage effectiveness, or PUE. The center delivers a PUE of 1.28. That means 78 percent of the power goes to the actual compute equipment (up from as low as 25 percent in the worst case). "That's a huge jump," says Dean Nelson. "We shed a half a megawatt of required power and we shed $400,000 a year in utilities.
Do you want to read how Sun did it? Here is the link to download:
http://wikis.sun.com/display/BluePrints/Energy+Efficient+Datacenters+-+The+Role+of+Modularity+in+Datacenter+Design
"Energy Efficient Datacenters: The Role of Modularity in Datacenter Design"
by Dean Nelson, Michael Ryan, Serena DeVito, Ramesh KV, Petr Vlasaty, Brett Rucker, and Brian Day (June, 2008)
Tuesday Jan 13, 2009
Last week, Time magazine had an article "America's Untapped Energy Resource: Boosting Efficiency". It was all about how we all need to waste less energy for the same amount of work. That is why Sun is a be proponent of measuring watts on servers on standard benchmarks.
Other's try to focus on watts when idle or nearly idle on tiny configurations.
It is really funny that some vendors avoid showing easy to measure power on performance benchmarks at real configurations. Is it that they have a advantage on special configurations and they want to only show that?
I decided to show this graphically, using "0" zero for those that have ZERO information on the measured watts used during the benchmark. What is behind this stonewall of red?
Power-Performance Benchmarks
| Benchmarks |
Sun |
IBM |
HP |
Dell |
Unisys |
| SPECweb |
Yes |
0 |
0 |
0 |
0 |
| SPECjbb |
Yes |
0 |
0 |
0 |
0 |
| SPECjAppServer |
Yes |
0 |
0 |
0 |
0 |
| SPECmail |
Yes |
0 |
0 |
0 |
0 |
| SPEComp |
Yes |
0 |
0 |
0 |
0 |
| Lotus Domino iNotes |
Yes |
0 |
0 |
0 |
0 |
| Oracle's Siebel CRM 8.0 |
Yes |
0 |
0 |
0 |
0 |
| VMmark |
Yes |
0 |
0 |
0 |
0 |
| MySQL |
Yes |
0 |
0 |
0 |
0 |
| ERP |
CDA/NDA |
0 |
0 |
0 |
0 |
| SPECpower |
SPECpower
Issues |
yes |
yes |
yes |
0 |
Note customers can get "ERP" power data from Sun under CDA/NDA agreement, as some SW vendors do not allow public display of wattage data measured during the benchmark.
Want more details on Sun's published measured watts, last week I posted "Sun's 2008 summary of measured watts & watt/performance": http://blogs.sun.com/bmseer/entry/sun_s_2008_summary_of
Some other thoughts for those really interested in saving real energy:
- Power-management software should be on by default or NOT ALLOWED IN BENCHMARKS!
- Increase server utilisation to most effectively reduce power consumed! Servers are very very inefficient at low-utilisation!
- look at watt-performance NOT perf/watt when comparing servers!
- replace incandescents with CFL lights.
Disclosure Statement:
SPEC, SPECweb, SPECjbb, SPECjAppServer, SPEComp, and SPECpower benchmark name are registered trademarks of the Standard Performance Evaluation Corporation.
VMware(R) VMmark(R) is a product of VMware, an EMC Company.
Oracle, Siebel, registered trademarks of Oracle Corporation and/or its affiliates. More info www.oracle.com/apps_benchmark/html/white-papers-siebel.html
NotesBench R6iNotes More info: www.notesbench.org
Friday Jan 09, 2009
George Ou wrote an article called "AMD submits suboptimal SPECpower benchmarks for Intel", pointing out that AMD submitted some results for Intel-based systems.
Of course, call foul on AMD for using eight 2GB dimms instead of four 4GB
DIMMS. AMD did however use Low-Power FB-DIMMs on their Intel submission ("Memory Details: DDR2-667 CL5 LP FB-DIMM;"). But it begs the question why aren't Intel-based vendors using the same standard DIMMs as they use on other benchmarks that don't require power reporting?
OK let's dig deeper, maybe the AMD published Intel result do reveal something. Maybe memory configuration/size is critical to a servers watts? Let's look for some tidbits...
- Why are all SPECpower_ssj on 8GB to 16GB, while other benchmarks using the same CPU family are on 32GB to 64GB?
- Why did the reporters miss that Intel-based systems use a modified (hacked?) BIOS for SPECpower_ssj? Vendors using different BIOS than what is default for customers???
- SPEC data does show servers at low utilisation (anywhere under 50%) have much different power-performance that even 50% utilisation. There has got to be a story there?
- ... other "interesting" configuration choices for SPECpower_ssj
Why don't other vendors post measured watts on a
variety of SPEC benchmarks? This gap of easy-to-measure data is the BIGGEST STORY.
Sun publishes measured watts, as I pointed out in yesterday's posting "Sun's 2008 summary of measured watts & watt/performance": http://blogs.sun.com/bmseer/entry/sun_s_2008_summary_of
Disclosure Statement
SPEC and the SPECpower benchmark name are registered trademarks of the Standard Performance Evaluation Corporation.
Thursday Jan 08, 2009
Sun has shown measured watts with measured performance on UltraSPARC for over four years!:
2008: UltraSPARC T2+ T5240 & UltraSPARC T2+ T5440
2007: UltraSPARC T2
2005: UltraSPARC T1 & T2000 blogs with power-performance
Sun is now showing measured watts with measured performance for Xeon-based and Opteron-based servers as well, Sun is saying we are going see a lot more soon:
2008 Xeon: Virtualization, Web
2008 Opteron: Java, Mail, Web
Real measured watts on a variety of workloads with real-sized memory configurations is critical to truly inform customers. I heard that one vendor was giving out top-secret watt measurements in a 16C datacentre (61F) as a way to lower their watt numbers... but does that make any real sense?
But let's look at the big picture: Datacentre managers need to be look at by "power usage effectiveness" (PUE). To keep a datacentre that cold you need to burn LOTS of watts -- so great way to slash your server numbers for marketing, but really net loss for a real customer.
Sun is looking at a variety of ways to save customers money. Take a look at this ZDnet article "Some like it hot: Why waste dough cooling down a data center?"
Expect to see many more measured results from Sun in 2009... its been years will any other system vendor step up and show the same or will they just do slick marketing and give you dozens of reasons why they can't give you a number you can simply measure? ...or ask you just to measure idle?
Wednesday Jan 07, 2009
The 32-way Sun SPARC Enterprise M9000 with 2.52 GHz SPARC64 VII
processors, 32 processors / 128 cores / 256 threads, achieved 24,650
users on the two-tier SAP Sales and Distribution (SD) standard SAP ERP
6.0 (2005) application benchmark.
The 32-way Sun SPARC Enterprise M9000 gets to 69% of the per-cpu result
of largest configuration 32-way IBM p595 (POWER6 5.0 GHz, 64 cores total). Note the Sun SPARC Enterprise M9000 can be configured as a 64-way system.
SAP-SD 2-Tier Performance Table (in decreasing performance order)
| System |
OS
Database |
Users |
SAP
ERP/ECC
Release |
SAPS |
SAPS/
Proc |
Date |
Sun SPARC Enterprise M9000
64xSPARC64 VII @2.52GHz
1024 GB |
Solaris 10
Oracle 10g |
39,100 |
2005
6.0 |
196,564 |
3,071 |
14-Jul-08 |
IBM Power 595
32xPOWER6 @5.0GHz
64 cores, 512 GB |
AIX 6.1
DB2 9.5 |
35,400 |
2005
6.0 |
177,950 |
5,561 |
08-Apr-08 |
HP Integrity SD64B
64xItanium2 @1.6GHz
128 cores, 512 GB |
HP-UX 11iV3
Oracle 10g |
30,000 |
2005
6.0 |
152,530 |
2,383 |
18-Dec-06 |
Sun SPARC Enterprise M9000
64xSPARC64 VI @2.4GHz
1024 GB |
Solaris 10
Oracle 10g |
25,130 |
2005
6.0 |
129,420 |
2,022 |
11-Jul-08 |
Sun SPARC Enterprise M9000
32xSPARC64 VII @2.52GHz
512 GB |
Solaris 10
Oracle 10g |
24,650 |
2005
6.0 |
123,470 |
3,858 |
17-Dec-08 |
IBM p5 595
64xPOWER5+ @2.3GHz
64 cores, 512 GB |
AIX 5.3
DB2 9 |
23,456 |
2004
5.0 |
117,520 |
1,836 |
25-Jul-06 |
Sun SPARC Enterprise M8000
16xSPARC64 VI @2.4GHz
256 GB |
Solaris 10
Oracle 10g |
7,300 |
2005
6.0 |
36,570 |
2,285 |
17-Apr-07 |
SAP ERP 6.0 (2005) application benchmark is a bit more
heavy-weight than mySAP ERP 2004 (SAP ECC 5.0), which has a performance
impact of ~2-3%.
Complete benchmark results may be found at the SAP benchmark website
http://www.sap.com/benchmark.
Benchmark Description
The SAP Standard Application SD (Sales and Distribution) Benchmark is a two-tier ERP business test that is indicative of full business workloads of complete order processing and invoice processing, and demonstrates the ability to run both the application and database software on a single system. The SAP Standard Application SD Benchmark represents the critical tasks performed in real-world ERP business environments.
SAP is one of the premier world-wide ERP application providers, and maintains systems on the various SAP products.
Disclosure Statement:
Two-tier SAP Sales and Distribution (SD) standard SAP ERP 6.0 (2005) application benchmark
as of 12/17/08:
Sun SPARC Enterprise M9000 (32 processors, 128 cores, 256 threads) 32 x 2.52 GHz SPARC64 VII,
512GB memory, 24,650 SD benchmark users,
Cert#2008075, Oracle 10g, Solaris 10, SAP ECC Release 6.0;
Sun SPARC Enterprise M9000 (64 processors, 256 cores, 512 threads) 64 x 2.52 GHz SPARC64 VII,
1024GB memory, 39,100 SD benchmark users,
Cert#2008042, Oracle 10g, Solaris 10, SAP ECC Release 6.0;
Sun SPARC Enterprise M9000 (64 processors, 128 cores, 256 threads) 64 x 2.4 GHz SPARC64
VI, 1024GB memory, 25,130 SD benchmark users,
Cert#2008040, Oracle 10g, Solaris 10, SAP ECC Release 6.0;
Sun SPARC Enterprise M8000 (16 processors, 32 cores, 64 threads) 16 x 2.4 GHz SPARC64 VI,
256GB memory, 7,300 SD benchmark users, Cert#2007026,
Oracle 10g, Solaris 10, SAP ECC Release 6.0;
IBM Power 595 (32 processors, 64 cores, 128 threads),
35,400 SD benchmark users, 32 x 5.0 GHz POWER6, 512 GB,
DB2 9.5, AIX 6.1, Cert. 2008019, SAP ECC Release 6.0;
IBM System p5 595 (64 processors, 64 cores, 128 threads),
23,456 SD benchmark users, 64 x 2.3 GHz POWER5+, 512 GB,
DB2 9, AIX 5.3, Cert. 2006045, SAP ECC Release 5.0;
HP Integrity SD64B (64 processors, 128 cores, 256 threads),
30,000 SD benchmark users, 64 x 1.6 GHz Dual-Core Intel Itanium 2, 512 GB,
Oracle 10g, HP-UX 11iV3, Cert#2006089, SAP ECC Release 6.0;
SAP, R/3, mySAP reg TM of SAP AG in Germany and other countries.
More info http://www.sap.com/benchmark.
SAP-SD 2-Tier benchmark Summary
| Certified Results |
|
Performance: |
|
24,650 benchmark users |
|
Server: |
|
Sun SPARC Enterprise M9000 |
|
Processors: |
|
32 x 2.52 GHz SPARC64 VII |
|
Memory: |
|
512 GB |
|
Operating system: |
|
Solaris 10 |
|
Database S/W: |
|
Oracle 10g |
|
SAP S/W: |
|
SAP ECC 6.0 |
|
SAP Certification: |
|
#2008075 |
|
Storage: |
|
1 x Internal System Disk
8 x Sun StorageTek(tm) 6140 Arrays |
|
|
|
|
Seems like more desperate calls for help and assis...