BM Seer Unofficial thoughts from an anonymous Sun employee

SPECjAppServer Sun T5140/T5440

Friday Feb 13, 2009

Sun SPARC Enterprise T5140 and T5440 Delivers Outstanding Application and Database Performance on SPECjAppServer2004. Yet again, Sun is showing measured watts on another benchmark. I encourage all other vendors to do the same on all benchmarks. We all need this kind of transparency!

Four Sun SPARC Enterprise T5140 Servers and 1 Sun SPARC Enterprise T5440 server delivered a result of 9500.76 SPECjAppServer2004 JOPS@Standard. ZFS was also used in this benchmark.

One Sun SPARC Enterprise T5140 server in the application tier, consumed on average 614 Watts of power and the Sun SPARC Enterprise T5440 server in the database tier, consumed on average 1836 Watts of power during the execution of this benchmark.

This benchmark used the Oracle WebLogic 10.3 Application Server and Oracle Database 11g Enterprise Edition. This benchmark result proves that the Sun SPARC Enterprise T5440 server using the UltraSPARC T2 Plus processor performs as an outstanding Oracle 11g OLTP database server.

The Sun SPARC Enterprise T5140, T5440 and M3000 servers used to produce this benchmark result all used the Solaris 10 10/8 Operating Environment.

Four Sun SPARC Enterprise T5140 Servers and 1 Sun SPARC Enterprise T5440 server demonstrated better performance compared to the HP result of 9459.19 SPECjAppServer2004 JOPS@Standard which used 11x HP BL860c servers and 2x HP Superdomes.

Each Sun SPARC Enterprise T5140 server used 4 instances of Oracle WebLogic 10.3 and the Sun JVM 1.6.0_06 Performance Release in Solaris Containers. Each Sun SPARC Enterprise T5140 server used ZFS to mirror 2 Solid State Disks to meet the benchmark durable storage requirements for the application server logs and JMS Persistence filestore.

The Sun SPARC Enterprise T5440 server in the database tier used Oracle Automatic Storage Management (ASM) to manage the StorageTek 2540 and 2501 storage arrays for the database files and redo logs.

The Sun result of 9500.76 SPECjAppServer2004 JOPS@Standard using one Sun SPARC Enterprise T5440 in the database tier used 24 Oracle licenses for the database. The HP result of 10519.43 SPECjAppServer2004 JOPS@Standard using one HP Superdome in the database tier used 40 Oracle database licenses. The Sun T5440 delivered 90% of the performance using 60% of the database licenses compared to the HP Superdome.

The Sun result of 9500.76 SPECjAppServer2004 JOPS@Standard using one Sun SPARC Enterprise T5440 in the database tier used 4 Rack Units of space (H x W x D = 7" x 18" x 25" = 1.8 cu feet). The HP result of 10519.43 SPECjAppServer2004 JOPS@Standard using the HP Superdome database (a 40 processor partition in 2x A9834A cabinets), have space requirements of (H x W x D = 72" x 48" x 45" = 90 cu feet). The Sun T5440 occupies 1/50 of the datacenter space at 90% of the performance of an HP Superdome.

The Sun result of 9500.76 SPECjAppServer2004 JOPS@Standard using one Sun SPARC Enterprise T5440 in the database tier consumed on average 1836 watts of power during the execution of this benchmark. The HP result of 10519.43 SPECjAppServer2004 JOPS@Standard using the HP Superdome database (a 40 processor partition in 2x A9834A cabinets), have power requirements of 13720 watts(1) or 7.5 TIMES more than Sun's T5440. The Sun T5440 consumes 14% of the power at 90% of the performance of an HP Superdome.

Performance Comparisons

SPECjAppServer2004 Performance Chart as of 02/04/2009. Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org. SPECjAppServer2004 JOPS@Standard (bigger is better)

Submitter SPECjAppServer2004
JOPS@Standard		 J2EE Server DB Server
HP 10519.43 12x HP BL860c
4 cores, 2 chips @ 1.66 GHz Itanium 9100
Oracle OC4J 10.1.3.3.2		 1x HP Superdome
40 cores, 20 chips @ 1.6 GHz Itanium 9000
Oracle 10g DB 10.2.0.3
Sun 9500.76 4x Sun SPARC Enterprise T5140
16 cores, 2 chips @ 1.2 GHz US-T2 Plus
Oracle WebLogic 10.3		 1x Sun SPARC Enterprise T5440
32 cores, 4 chips @ 1.4 GHz US-T2 Plus
Oracle 11g DB 11.1.0.7
HP 9459.19 11x HP BL860c
4 cores, 2 chips @ 1.66 GHz Itanium 9100
Oracle OC4J 10.1.3.3.2		 2x HP Superdome
80 cores, 40 chips @ 1.6 GHz Itanium 9000
Oracle 10g DB 10.2.0.3 with RAC
Sun 8439.366 6x Sun SPARC Enterprise T5120
8 cores, 1 chip @ 1.4 GHz US-T2
Sun Java System Application Server		 1x Sun SPARC Enterprise E6900
48 cores, 24 chips @ 1.95 GHz US-IV+
IBM DB2 V9.1

Benchmark Description

SPECjAppServer2004 (Java Application Server) is a multi-tier benchmark for measuring the performance of Java 2 Enterprise Edition (J2EE) technology-based application servers. SPECjAppServer2004 is an end-to-end application which exercises all major J2EE technologies implemented by compliant application servers as follows:

  • The web container, including servelets and JSPs
  • The EJB container
  • EJB2.0 Container Managed Persistence
  • JMS and Message Driven Beans
  • Transaction management
  • Database connectivity
Moreover, SPECjAppServer2004 also heavily exercises all parts of the underlying infrastructure that make up the application environment, including hardware, JVM software, database software, JDBC drivers, and the system network.

The primary metric of the SPECjAppServer2004 benchmark is jAppServer Operations Per Second (JOPS) which is calculated by adding the metrics of the Dealership Management Application in the Dealer Domain and the Manufacturing Application in the Manufacturing Domain. There is NO price/performance metric in this benchmark.

Disclosure Statement:

SPECjAppServer2004: 4x Sun SPARC Enterprise T5140 (8 chips, 64 cores) 9500.76 SPECjAppServer2004 JOPS@Standard. 12x HP BL860c (24 chips, 48 cores) 10519.43 SPECjAppServer2004 JOPS@Standard. 12x HP BL860c (22 chips, 44 cores) 9459.19 SPECjAppServer2004 JOPS@Standard. 6x Sun T5120 (6 chips, 48 cores) 8439.36 SPECjAppServer2004 JOPS@Standard. SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 1/28/09.

1. HP Integrity Superdome using 2x A9834A cabinets. Taking 70% of Typical Input power of 9800 watts for an 8-cell cabinet and from: http://h18000.www1.hp.com/products/quickspecs/11717_div/11717_div.HTML

See Also: SPECjAppServer2004 Results Page

Results Summary

Published Results 9500.76 SPECjAppServer2004 JOPS@Standard
Reference Date: Feb 4, 2009
Systems: 4x Sun SPARC Enterprise T5140
1x Sun SPARC Enterprise T5440
1x Sun SPARC Enterprise M3000
Total Number Processors: 2, 4, 1
Processor/GHz of Server: UltraSPARC T2 Plus 1.2 GHz
UltraSPARC T2 Plus 1.4 GHz
SPARC64VII 2.52 GHz
Operating System: Solaris 10 10/08
Software: Oracle WebLogic 10.3 Application Server, Standard Edition
Oracle Database Enterprise Edition Release 11.1.0.7
JVM: JDK 1.6.0_06 Performance Release

note: ...as always you can post comments anonymously, but if you work for a system's vendor you should state who it is.

[7] Comments
Like this post? del.icio.us | furl | slashdot | technorati | digg
Posted at 07:06AM Feb 13, 2009 in Benchmarks
Tags:
Comments:

I thought you would have been RIF'd in the last wave. You bring no value to Sun's customers and if anything you bring a continued sense of ignorant arrogance.

Posted by Jerry Picarre on February 15, 2009 at 07:52 AM PST #

BM Seer wrote:

Wow, that is incredibly mean-spirited comment. I'm showing lots of data and valid comparisons on every benchmark, much more than IBM, HP, etc bloggers. I also bring up questions about other vendors who show very little data about important issues.

Look at all of the wattage data posted in this benchmark writeup! Look at the comparisons system-to-system -- these are things that matter to all customers.

It would be nice to know what company you work for, also I hope you do not get RIF'ed.

Regardless, I do hope in the future you post comments with information not just name-calling and hatred.

Posted by BM Seer on February 17, 2009 at 11:40 AM PST #

Jerry, I looked up your IP address and got inspired to look at a blogger for a computer vendor's corporation headquarters in the same town as you. If you want to talk about "no value"... here goes:

An IBM blogger recently talking about a submission on SPECjAppServer and used 210 words (not counting the disclosure statement). Only 27 words in that blog entry related to the actual benchmark results. 13% percent of all words: no comparisons, no context, no watts, no space, no benchmarks description... wow :(

Yes I'll keep providing value. The above had about 800+ words with all manner of information. I really shouldn't post this late...

SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation.

Posted by BM Seer on February 18, 2009 at 04:46 PM PST #

"Look at all the wattage data" would hold much more water if half of it wasn't guesstimated by means which have already been shown to be error prone.

Given that this is an app server benchmark, how does one know the degree to which the DB server was utilized? Your numbers got me looking at the other published benchmark results, and it would seem there have been several, rather larger results from HP and IBM since the pair of HP results from 2007 mentioned above, and it would appear that the later HP results also used a single Superdome as the DB server.

If I look at IBM's latest published result:

http://www.spec.org/osg/jAppServer2004/results/res2008q4/jAppServer2004-20081216-00126.html

from December of 2008, it appears to use a JVM in the app tier also from December of 2008. The Sun results you are touting here are using a JVM which, based on searching various SPEC Java-oriented results, seems to have been released back in July of 2008 - that is 7 or 8 months older than the publication date of the Sun results touted in your entry. Of course, if there has been little to no change in the JVM in 7 or 8 months that would be one thing, but has the JVM been static for 8 months? I'm not fully conversant with JVM naming conventions, but the "_06" in the version string corresponds to Update 6 right? If I go to java.sun.com today it seems to be encouraging people to download Update 12. That seems to suggest the JVM used in these Sun results is not only 8 months old but 6 updates behind?

Posted by rick jones on February 19, 2009 at 09:44 AM PST #

Rick: we have measured watts on HP platforms including the published on the DL580 G5 they match our estimates.

Sun did actually post watts on a DL580 G5 4-chip 2.93GHz QC Xeon with 64GB = 830watts, see: http://blogs.sun.com/ritu/entry/mysql_benchmark_us_t2_beats
IBM pointing to a Principled technologies paper. They say that an HP DL580 G5 4-chip QC 2.93GHz Xeon 64GB uses an average 942watts. Roughly, the DL580 G5 is in the 800-900watt range, which is a bit less than HP power calculator.

Until you have measured data to prove something different, comments like
"have already been shown to be error prone." is marketing FUD.

Posted by BM Seer on February 19, 2009 at 10:08 AM PST #

'Until you have measured data to prove something different, comments like
"have already been shown to be error prone." is marketing FUD.'

WRT matching estimates, the last comment at:

http://blogs.sun.com/bmseer/entry/vmware_vmmark_sun_fire_x4450#comments

which references:

http://blogs.sun.com/bmseer/entry/vmmark_performance_watt_performance

Where you say that using a power calculator gave 1086 Watts for a system when the actual measurement on one workload (from the link you just gave above) was 830 Watts, which is an instance of a power calculator giving > 30% higher power consumption than measured. And a second workload was 942 Watts, suggesting there is not a constant "fudge factor" one can apply to a power calculator to guesstimate power consumption.

And in the other direction, using the T5220 mentioned in the link in your response, if I go to the T5220's power calculator, plug-in the 1, 1.4GHz CPU; four, 146GB drives, three PCIe cards and 8x8GB DIMMS (I guessed since the blog entry didn't say the number and size of DIMMs) it gives 442 Watts to the measured 480 Watts. If instead I use 16, 4GB DIMMs the power calculator says 526.

Returning to unanswered questions - how _does_ one tell if the DB server in a SPECjAppServer2004 result is actually saturated? That would seem to be important if one is going to use SPECjAppServer2004 results to compare database servers instead of the app servers the benchmark seems to be primarily designed to measure. And why are the published Sun results mentioned here using a JVM that appears to be 6 updates behind?

Posted by rick jones on February 19, 2009 at 08:24 PM PST #

Rick, very simple about the DB server utilisation for large HP Superdome: Rick just ask HP. They should have all of that data.

Sure power calculators aren't perfect, but why don't you plug standard benchmark configurations in for HP, DELL, IBM, ... and tell me their watts. I've done it they are even bigger! Yes, Worst Power-Performance for them.

Posted by BM Seer on February 26, 2009 at 10:22 AM PST #

Post a Comment:
Comments are closed for this entry.