BestPerf: Performance & Best Practices

The Sun SPARC Enterprise T5240 server running the Sun Java Messaging server 7.2 achieved a World Record SPECmail2009 result using Sun Storage 7310 Unified Storage System and ZFS file system.  Sun's OpenStorage platforms enable another world record.

• World record SPECmail2009 benchmark using the Sun SPARC Enterprise T5240 server (two 1.6GHz UltraSPARC T2 Plus), Sun Communications Suite 7, Solaris 10, and the Sun Storage 7310 Unified Storage System achieved 14,500 SPECmail_Ent2009 users at 69,857 Sessions/Hour.

• This SPECmail2009 benchmark result clearly demonstrates that the Sun Messaging Server 7.2, Solaris 10 and ZFS solution can support a large, enterprise level IMAP mail server environment as a low cost 'Sun on Sun' solution, delivering the best performance and maximizing data integrity and availability of Sun Open Storage and ZFS.

• The Sun SPARC Enterprise T5240 server supported 2.4 times more users with 2.4 times better sessions/hour rate than AppleXserv3 solution on the SPECmail2009 benchmark.

• There are no IBM Power6 results on this benchmark.

• The configuration using Sun OpenStorage outperformed all previous results with traditional direct attached storage and significantly higher number of disk devices.

SPECmail2009 Performance Landscape (ordered by performance)

System	Performance		Disks	OS	Messaging Server
	Users	Sessions/ hour
Sun SPARC Enterprise T5240 2 x 1.6GHz UltraSPARC T2 Plus	14,500	69,857	58 NAS	Solaris 10	CommSuite 7.2 Sun JMS 7.2
Sun SPARC Enterprise T5240 2 x 1.6GHz UltraSPARC T2 Plus	12,000	57,758	80 DAS	Solaris 10	CommSuite 5 Sun JMS 6.3
Sun Fire X4275 2 x 2.93GHz Xeon X5570	8,000	38,348	44 NAS	Solaris 10	Sun JMS 6.2
Apple Xserv3,1 2 x 2.93GHz Xeon X5570	6,000	28,887	82 DAS	MacOS 10.6	Dovecot 1.1.14 apple 0.5
Sun SPARC Enterprise T5220 1 x 1.4GHz UltraSPARC T2	3,600	17,316	52 DAS	Solaris 10	Sun JMS 6.2

Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org

Users - SPECmail_Ent2009 Users
Sessions/hour - SPECmail2009 Sessions/hour
NAS - Network Attached Storage
DAS - Direct Attached Storage

Results and Configuration Summary

Hardware Configuration:

Sun SPARC Enterprise T5240

2 x 1.6 GHz UltraSPARC T2 Plus processors
128 GB memory
2 x 146GB, 10K RPM SAS disks, 4 x 32GB SSDs

External Storage:

2 x Sun Storage 7310 Unified Storage System, each with

32 GB of memory
24 x 1 TB 7200 RPM SATA Drives

Software Configuration:

Solaris 10
ZFS
Sun Java Communications Suite 7 Update 2

Sun Java System Messaging Server 7.2
Directory Server 6.3

Benchmark Description

The SPECmail2009 benchmark measures the ability of corporate e-mail systems to meet today's demanding e-mail users over fast corporate local area networks (LAN). The SPECmail2009 benchmark simulates corporate mail server workloads that range from 250 to 10,000 or more users, using industry standard SMTP and IMAP4 protocols. This e-mail server benchmark creates client workloads based on a 40,000 user corporation, and uses folder and message MIME structures that include both traditional office documents and a variety of rich media content. The benchmark also adds support for encrypted network connections using industry standard SSL v3.0 and TLS 1.0 technology. SPECmail2009 replaces all versions of SPECmail2008, first released in August 2008. The results from the two benchmarks are not comparable.

Software on one or more client machines generates a benchmark load for a System Under Test (SUT) and measures the SUT response times. A SUT can be a mail server running on a single system or a cluster of systems.

A SPECmail2009 'run' simulates a 100% load level associated with the specific number of users, as defined in the configuration file. The mail server must maintain a specific Quality of Service (QoS) at the 100% load level to produce a valid benchmark result. If the mail server does maintain the specified QoS at the 100% load level, the performance of the mail server is reported as SPECmail_Ent2009 SMTP and IMAP Users at SPECmail2009 Sessions per hour. The SPECmail_Ent2009 users at SPECmail2009 Sessions per Hour metric reflects the unique workload combination for a SPEC IMAP4 user.

Key Points and Best Practices

• Each Sun Storage 7310 Unified Storage System was configured with one J4400 JBOD array with 22x1TB SATA drives to a mirrored device and 4 shared volumes are built under the mirrored device. Total 8 mirrored volumes from 2 x Sun Storage 7310 are mounted on the system under test (SUT) messaging mail indexes and mail messages file system using NFSV4 protocol. Four SSDs were used as the SUT internal disks. Each SSD is configured as a ZFS file system. Four such ZFS directories are used for the messaging server queue, store metadata, LDAP and queue. SSDs substantially reduced the store metadata and queue latencies.

• Each Sun Storage 7310 Unified Storage System was connected to the SUT via a dual 10-Gigabit Ethernet Fiber XFP card.

• The Sun Storage 7310 Unified Storage System software version is 2009.08.11,1-0.

• The clients used these Java options: java -d64 -Xms4096m -Xmx4096m -XX:+AggressiveHeap

• Substantial performance improvement and scalability was observed with Sun Communications Suite7 update2, Java Messaging Server 7.2 and Directory Server 6.2

• See the SPEC Report for all OS, network and messaging server tunings.

See Also

• SPECmail2009 Benchmark Page

Disclosure Statement

SPEC, SPECmail reg tm of Standard Performance Evaluation Corporation. Results as of 10/22/09 on www.spec.org. SPECmail2009: Sun SPARC Enterprise T5240, SPECmail_Ent2009 14,500 users at 69,857 SPECmail2009 Sessions/hour. Apple Xserv3,1, SPECmail_Ent2009 6,000 users at 28,887 SPECmail2009 Sessions/hour.

The Sun SPARC Enterprise T5440 server with 1.6GHz UltraSPARC T2 Plus with Solaris Containers, Sun Flash Open Storage, and Sun JAVA System Web Server 7.0 Update 5 achieved World Record SPECweb2005.

• Sun has obtained a World Record SPECweb2005 performance result of 100,209 SPECweb2005 on the Sun SPARC Enterprise T5440, running Solaris 10 10/09 Sun JAVA System Web Server 7.0 Update 5, and Java Hotspot™ Server VM.

• This result demonstrates performance leadership of the Sun SPARC Enterprise T5440 server and its scalability, by using Solaris Containers to consolidate multiple web serving environments, and Sun OpenStorage Flash technology to store large datasets for fast data retrieval.

• The Sun SPARC Enterprise T5440 delivers 21% greater SPECweb2005 performance than the HP DL370 G6 with 3.2GHz Xeon W5580 processors.

• The Sun SPARC Enterprise T5440 delivers 40% greater SPECweb2005 performance than the HP DL 585 G5 with four 3.114 GHz Opteron 8393 SE processors.

• The Sun SPARC Enterprise T5440 delivers 2x the SPECweb2005 performance of the HP DL 580 G5 with four 2.66GHz Xeon X7460 processors.

• There are no IBM Power6 results on the SPECweb2005 benchmark.

• This benchmark result clearly demonstrates that the Sun SPARC Enterprise T5440 running Solaris 10 10/09 and Sun Java System Webserver 7.0 Update 5 can support thousands of concurrent web server sessions and is an industry leader in web serving with a Sun solution.
  

Performance Landscape

Server	Processor	SPECweb2005	Banking*	Ecomm*	Support*	Webserver	OS
Sun T5440	4x 1.6 T2 Plus	100,209	176,500	133,000	95,000	Java WebServer	Solaris
HP DL370 G6	2x 3.2 W5580	83,073	117,120	142,080	76,352	Rock	RedHat Linux
HP DL585 G5	4x 3.11 O8393	71,629	117,504	123,072	56,320	Rock	RedHat Linux
HP DL580 G5	4x 2.66 X7460	50,013	97,632	69,600	40,800	Rock	RedHat Linux

* Banking - SPECweb2005-Banking
   Ecomm - SPECweb2005-Ecommerce
   Support - SPECweb2005-Support

Results and Configuration Summary

Hardware Configuration:

  1 Sun SPARC Enterprise T5440 with

• 4 x UltraSPARC T2 Processor 8 core, 64 threads, 1.6 GHz
• 254 GB memory
• 6 x 4Gb PCI Express 8-Port Host Adapter (SG-XPCIE8SAS-E-Z)
• 1 x Sun Storage F5100 Flash Array (TA5100RASA4-80AA)
• 1 x Sun Storage F5100 Flash Array (TA5100RASA4-40AA)

Server Software Configuration:

• Solaris 10 10/09
• JAVA System Web Server 7.0 Update 5
• Java Hotspot™ Server VM

Network configuration:

• 1 x Arista DCS-7124s 24-10GbE port  switch
• 1 x Cisco 2970 series (WS-C2970G-24TS-E) switch for the three 1 GbE networks

Back-end Simulator:

  1 Sun Fire X4270 with

• 2 x 2.93 GHz Intel X5570 Quad core
• 48GB memory
• Solaris 10 10/09
• JSWS 7.0 Update 5
• Java Hotspot™ Server VM

Clients:

  8 Sun Blade™ T6320

• 1 x 1.417 GHz UltraSPARC-T2
• 64 GB memory
• Solaris 10 5/09
• Java Hotspot™ Server VM

  8 Sun Blade™ 6270

• 2 x 2.93 GHz Intel X5570 Quad core
• 36 GB memory
• Solaris 10 5/09
• Java Hotspot™ Server VM

Benchmark Description

SPECweb2005, successor to SPECweb99 and SPECweb99_SSL, is an industry standard benchmark for evaluating Web Server performance developed by SPEC. The benchmark simulates multiple user sessions accessing a Web Server and generating static and dynamic HTTP requests. The major features of SPECweb2005 are:

• Measures simultaneous user sessions
• Dynamic content: currently PHP and JSP implementations
• Page images requested using 2 parallel HTTP connections
• Multiple, standardized workloads: Banking (HTTPS), E-commerce (HTTP and HTTPS), and Support (HTTP)
• Simulates browser caching effects
• File accesses more accurately simulate today's disk access patterns

Key Points and Best Practices

• The server was divided into four Solaris Containers and a single web server instance was executed in each container.
  
• Four processor sets were created (with varying numbers of threads depending on the workload) to run the web server in. This was done to reduce memory access latency using the physical memory closest to the processor.  All interrupts were run on the remaining threads.
  
• Each web server is executed in the FX scheduling class to improve performance by reducing the frequency of context switches.
• Two Sun Storage F5100 Flash Arrays (holding the target file set and logs) were shared by the four containers  for fast data retrieval.   
• Use of Solaris Containers highlights the consolidation of multiple web serving environments on a single server.
• Use of the Sun Ext I/O Expansion unit and Sun Storage F5100 Flash Arrays highlight the expandability of the server.
  

Disclosure Statement

Sun SPARC Enterprise T5440 (8 cores, 1 chip) 100209 SPECweb2005, was submitted to SPEC for review on October 13, 2009.  HP ProLiant DL370 G6 (8 cores, 2 chips) 83,073 SPECweb2005. HP ProLiant DL585 G5 (16 cores, 4 chips) 71,629 SPECweb2005. HP ProLiant DL580 G5 (24 cores, 4 chips) 50,013 SPECweb2005. SPEC, SPECweb reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of Oct 10, 2009.

Significance of Results

A Sun SPARC Enterprise T5240 server equipped with two UltraSPARC T2 Plus processors at 1.6GHz delivered a result of 422782 SPECjbb2005 bops, 26424 SPECjbb2005 bops/JVM. The Sun SPARC Enterprise T5240 consumed an average of 875 Watts of power during the execution of the benchmark.

• The Sun SPARC Enterprise T5240 server running 2x 1.6 GHz UltraSPARC T2 Plus processor delivered 5% better performance than an IBM Power 570 with 4x 4.7 GHz POWER6 processors as measured by the SPECjbb2005 benchmark.

• The Sun SPARC Enterprise T5240 server equipped with two UltraSPARC T2 Plus processors at 1.6GHz demonstrated 10% better performance than the Sun SPARC Enterprise T5240 server equipped with two UltraSPARC T2 Plus processors at 1.4GHz.
• One Sun SPARC Enterprise T5240 (two 1.6GHz UltraSPARC T2 Plus chips, 2RU) has 2.3 times the power/performance than the IBM Power 570 (8RU) that used four 4.7GHz POWER6 chips.
• The Sun SPARC Enterprise T5240 used OpenSolaris 2009.06 and the Sun JDK 1.6.0_14 Performance Release to obtain this result.

Performance Landscape

SPECjbb2005 Performance Chart (ordered by performance), select results presented.

bops : SPECjbb2005 Business Operations per Second (bigger is better)

System	Processors					Performance
	Chips	Cores	Threads	GHz	Type	bops	bops/JVM
Sun SPARC Enterprise T5240	2	16	128	1.6	UltraSPARC T2 Plus	422782	26424
IBM Power 570	4	8	16	4.7	POWER6	402923	100731
Sun SPARC Enterprise T5240	2	16	128	1.4	UltraSPARC T2 Plus	384934	24058

Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org.

Results and Configuration Summary

Hardware Configuration:

Sun SPARC Enterprise T5240

2 x 1.6 GHz UltraSPARC T2 Plus processors
64 GB

Software Configuration:

OpenSolaris 2009.06
Java HotSpot(TM) 32-Bit Server, Version 1.6.0_14 Performance Release

Benchmark Description

SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).

Key Points and Best Practices

• Each JVM executed in the FX scheduling class to improve performance by reducing the frequency of context switches.
• Each JVM was bound to a separate processor containing 1 core to reduce memory access latency using the physical memory closest to the processor.

See Also

• SPECjbb2005 Benchmark Reports

Disclosure Statement

SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 8/25/2009 on http://www.spec.org.
Sun SPARC T5240 (2 chips, 16 cores) 422782 SPECjbb2005 bops, 26424 SPECjbb2005 bops/JVM;Sun SPARC T5240 (2 chips, 16 cores) 384934 SPECjbb2005 bops, 24058 SPECjbb2005 bops/JVM; IBM Power 570 (4 chips, 8 cores) 402923 SPECjbb2005 bops, 100731 SPECjbb2005 bops/JVM.

Sun watts were measured on the system during the test.

IBM p 570 4P (2 building blocks) power specifications calculated as 80% of maximum input power reported 7/8/09 in 'Facts and Features Report': ftp://ftp.software.ibm.com/common/ssi/pm/br/n/psb01628usen/PSB01628USEN.PDF

Significance of Results

A Sun SPARC Enterprise T5220 server equipped with one UltraSPARC T2 processor at 1.6GHz delivered a World Record single-chip result of 231464 SPECjbb2005 bops, 28933 SPECjbb2005 bops/JVM. The Sun SPARC Enterprise T5220 consumed an average of 520 Watts of power during the execution of this benchmark.

• The Sun SPARC Enterprise T5220 server (one 1.6 GHz UltraSPARC T2 chip) demonstrated 3% better performance over the Fujitsu TX100 result of 223691 SPECjbb2005 bops which used one 3.16 GHz Xeon X3380 processor.
• The Sun SPARC Enterprise T5220 (one 1.6 GHz UltraSPARC T2 chip) demonstrated 8% better performance over the IBM x3200 result of 214578 SPECjbb2005 bops which used one 3.16 GHz Xeon X3380 processor.
• The Sun SPARC Enterprise T5220 server (one 1.6 GHz UltraSPARC T2 chip) demonstrated 10% better performance over the Fujitsu RX100 result of 211144 SPECjbb2005 bops which used one 3.16 GHz Xeon X3380 processor.
• The Sun SPARC Enterprise T5220 server (one 1.6 GHz UltraSPARC T2 chip) demonstrated 19% better performance over the IBM X3350 result of 194256 SPECjbb2005 bops which used one 3 GHz Xeon X3370 processor.
• The Sun SPARC Enterprise T5220 server (one 1.6 GHz UltraSPARC T2 chip) demonstrated 2.6X the performance over the IBM p570 result of 88089 SPECjbb2005 bops which used one 4.7 GHz POWER6 processor.
• One Sun SPARC Enterprise T5220 (one 1.6GHz UltraSPARC T2 Plus chip, 2RU) has 2.1 the power/performance than the IBM Power 570 (4RU) that used two 4.7GHz POWER6 chips.
• The Sun SPARC Enterprise T5220 used OpenSolaris 2009.06 and the Sun JDK 1.6.0_14 Performance Release to obtain this result.

Performance Landscape

SPECjbb2005 Performance Chart (ordered by performance)

bops : SPECjbb2005 Business Operations per Second (bigger is better)

System	Processors					Performance
	Chips	Cores	Threads	GHz	Type	bops	bops/JVM
Sun SPARC Enterprise T5220	1	8	64	1.6	UltraSPARC T2	231464	28933
Sun Blade T6320	1	8	64	1.6	UltraSPARC T2	229576	28697
Fujitsu TX100	1	4	4	3.16	Intel Xeon	223691	111846
IBM x3200 M2	1	4	4	3.16	Intel Xeon	214578	107289
Fujitsu RX100	1	4	4	3.16	Intel Xeon	211144	105572
IBM Power 570	2	4	8	4.7	POWER6	205917	102959
IBM x3350	1	4	4	3.0	Intel Xeon	194256	97128
Sun SPARC Enterprise T5220	1	8	64	1.4	UltraSPARC T2	192055	24007
IBM Power 570	1	2	4	4.7	POWER6	88089	88089

Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org.

Results and Configuration Summary

Hardware Configuration:

Sun SPARC Enterprise T5220

1x 1.6 GHz UltraSPARC T2 processor
64 GB

Software Configuration:

OpenSolaris 2009.06
Java HotSpot(TM) 32-Bit Server, Version 1.6.0_14 Performance Release

Benchmark Description

SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).

Key Points and Best Practices

• Each JVM executed in the FX scheduling class to improve performance by reducing the frequency of context switches.
• Each JVM was bound to a separate processor containing 1 core to reduce memory access latency using the physical memory closest to the processor.

See Also

• SPECjbb2005 Benchmark Reports

Disclosure Statement

SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 8/25/2009 on http://www.spec.org.
Sun SPARC T5220 231464 SPECjbb2005 bops, 28933 SPECjbb2005 bops/JVM Submitted to SPEC for review; IBM p 570 88089 SPECjbb2005 bops, 88089 SPECjbb2005 bops/JVM; Fujitsu TX100 223691 SPECjbb2005 bops, 111846 SPECjbb2005 bops/JVM; IBM x3350 194256 SPECjbb2005 bops, 97128 SPECjbb2005 bops/JVM; Sun SPARC Enterprise T5120 192055 SPECjbb2005 bops, 24007 SPECjbb2005 bops/JVM.

Sun watts were measured on the system during the test.

IBM p 570 2P (1 building blocks) power specifications calculated as 80% of maximum input power reported 7/8/09 in "Facts and Features Report": ftp://ftp.software.ibm.com/common/ssi/pm/br/n/psb01628usen/PSB01628USEN.PDF

Significance of Results

The Sun SPARC Enterprise T5240 server running the Sun Java Messaging server 6.3 achieved World Record SPECmail2009 results using ZFS.

• A Sun SPARC Enterprise T5240 server powered by two 1.6 GHz UltraSPARC T2 Plus processors running the Sun Java Communications Suite 5 software along with the Solaris 10 Operating System and using six Sun StorageTek 2540 arrays achieved a new World Record 12000 SPECmail_Ent2009 IMAP4 users at 57,758 Sessions/hour for SPECmail2009.
• The Sun SPARC Enterprise T5240 server achieve twice the number of users and sessions/hour rate than the Apple Xserv3,1 solution equipped with Intel Nehalem processors.
• The Sun result was obtained using ~10% fewer disk spindles with the Sun StorageTek 2540 RAID controller direct attach storage solution versus Apple's direct attached storage.
• This benchmark result demonstrates that the Sun SPARC Enterprise T5240 server together with Sun Java Communication Suite 5 component Sun Java System Messaging Server 6.3, Solaris 10 and ZFS on Sun StorageTek 2540 arrays supports a large, enterprise level IMAP mail server environment. This solution is reliable, low cost, and low power, delivering the best performance and maximizing the data integrity with Sun's ZFS file systems.

Performance Landscape

SPECmail2009 (ordered by performance)

System	Processors			Performance
	Type	GHz	Ch, Co, Th	SPECmail_Ent2009 Users	SPECmail2009 Sessions/hour
Sun SPARC Enterprise T5240	UltraSPARC T2 Plus	1.6	2, 16, 128	12,000	57,758
Sun Fire X4275	Xeon X5570	2.93	2, 8, 16	8,000	38,348
Apple Xserv3,1	Xeon X5570	2.93	2, 8, 16	6,000	28,887
Sun SPARC Enterprise T5220	UltraSPARC T2	1.4	1, 8, 64	3,600	17,316

Notes:

Number of SPECmail_Ent2009 users (bigger is better)
SPECmail2009 Sessions/hour (bigger is better)
Ch, Co, Th: Chips, Cores, Threads

Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org

Results and Configuration Summary

Hardware Configuration:

Sun SPARC Enterprise T5240

2 x 1.6 GHz UltraSPARC T2 Plus processors
128 GB
8 x 146GB, 10K RPM SAS disks

6 x Sun StorageTek 2540 Arrays,

4 arrays with 12 x 146GB 15K RPM SAS disks
2 arrays with 12 x 73GB 15K RPM SAS disks

2 x Sun Fire X4600 benchmark manager, load generator and mail sink

8 x AMD Opteron 8356 2.7 GHz QC processors
64 GB
2 x 73GB 10K RPM SAS disks

Sun Fire X4240 load generator

2 x AMD Opteron 2384 2.7 GHz DC processors
16 GB
2 x 73GB 10K RPM SAS disks

Software Configuration:

Solaris 10
ZFS
Sun Java Communication Suite 5
Sun Java System Messaging Server 6.3

Benchmark Description

The SPECmail2009 benchmark measures the ability of corporate e-mail systems to meet today's demanding e-mail users over fast corporate local area networks (LAN). The SPECmail2009 benchmark simulates corporate mail server workloads that range from 250 to 10,000 or more users, using industry standard SMTP and IMAP4 protocols. This e-mail server benchmark creates client workloads based on a 40,000 user corporation, and uses folder and message MIME structures that include both traditional office documents and a variety of rich media content. The benchmark also adds support for encrypted network connections using industry standard SSL v3.0 and TLS 1.0 technology. SPECmail2009 replaces all versions of SPECmail2008, first released in August 2008. The results from the two benchmarks are not comparable.

Software on one or more client machines generates a benchmark load for a System Under Test (SUT) and measures the SUT response times. A SUT can be a mail server running on a single system or a cluster of systems.

A SPECmail2009 'run' simulates a 100% load level associated with the specific number of users, as defined in the configuration file. The mail server must maintain a specific Quality of Service (QoS) at the 100% load level to produce a valid benchmark result. If the mail server does maintain the specified QoS at the 100% load level, the performance of the mail server is reported as SPECmail_Ent2009 SMTP and IMAP Users at SPECmail2009 Sessions per hour. The SPECmail_Ent2009 users at SPECmail2009 Sessions per Hour metric reflects the unique workload combination for a SPEC IMAP4 user.

Key Points and Best Practices

• Each Sun StorageTek 2540 array was configured with 6 hardware RAID1 volumes. A total of 36 RAID1 volumes were configured with 24 of size 146GB and 12 of size 73GB. Four ZPOOLs of (6x146GB RAID1 volumes) were mounted as the four primary message stores and ZFS file systems. Four ZPOOLs of (8x73GB RAID1 volumes) were mounted as the four primary message indexes. The hardware RAID1 volumes were created with 64K stripe size without read ahead turned on. The 7x146GB internal drives were used to create four ZPOOLs and ZFS file systems for the LDAP, store metadata, queue and the mailserver log.

• The clients used these Java options: java -d64 -Xms4096m -Xmx4096m -XX:+AggressiveHeap

• See the SPEC Report for all OS, network and messaging server tunings.

See Also

• SPEC SPECmail2009 Benchmark Page

Disclosure Statement

SPEC, SPECmail reg tm of Standard Performance Evaluation Corporation. Results as of 08/07/2009 on www.spec.org. SPECmail2009: Sun SPARC Enterprise T5240 (16 cores, 2 chips) SPECmail_Ent2009 12000 users at 57,758 SPECmail2009 Sessions/hour. Apple Xserv3,1 (8 cores, 2 chips) SPECmail_Ent2009 6000 users at 28,887 SPECmail2009 Sessions/hour.

Significance of Results

The Sun Blade T6320 server module equipped with one UltraSPARC T2 processor running at 1.6 GHz delivered a World Record single-chip result while running the SPECjbb2005 benchmark.

• The Sun Blade T6320 server module powered by one 1.6 GHz UltraSPARC T2 processor delivered a result of 229576 SPECjbb2005 bops, 28697 SPECjbb2005 bops/JVM when running the SPECjbb2005 benchmark.
• The Sun Blade T6320 server module (with one 1.6 GHz UltraSPARC T2 processor) demonstrated 2.6X better performance than the IBM System p 570 with one 4.7 GHz POWER6 processor.
• The Sun Blade T6320 server module (with one 1.6 GHz UltraSPARC T2 processor) demonstrated 3% better performance than the Fujitsu TX100 result which used one 3.16 GHz Intel Xeon X3380 processor.
• The Sun Blade T6320 server module (with one 1.6 GHz UltraSPARC T2 processor) demonstrated 7% better performance than the IBM x3200 result which used one 3.16 GHz Xeon X3380 processor.
• The Sun Blade T6320 server module running the 1.6 GHz UltraSPARC T2 processor delivered 20% better performance than a Sun SPARC Enterprise T5120 with the 1.4 GHz UltraSPARC T2 processor.
• The Sun Blade T6320 used the OpenSolaris 2009.06 operation system and the Java HotSpot(TM) 32-Bit Server, Version 1.6.0_14 Performance Release JVM to obtain this leading result.

Performance Landscape

SPECjbb2005 Performance Chart (ordered by performance)

bops: SPECjbb2005 Business Operations per Second (bigger is better)

System	Processors					Performance
	Chips	Cores	Threads	GHz	Type	SPECjbb2005 bops	SPECjbb2005 bops/JVM
Sun Blade T6320	1	8	64	1.6	UltraSPARC T2	229576	28697
Fujitsu TX100	1	4	4	3.16	Intel Xeon	223691	111846
IBM x3200 M2	1	4	4	3.16	Intel Xeon	214578	107289
Fujitsu RX100	1	4	4	3.16	Intel Xeon	211144	105572
IBM x3350	1	4	4	3.0	Intel Xeon	194256	97128
Sun SE T5120	1	8	64	1.4	UltraSPARC T2	192055	24007
IBM p 570	1	2	4	4.7	POWER6	88089	88089

Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org.

Results and Configuration Summary

Hardware Configuration:

Sun Blade T6320

1 x 1.6 GHz UltraSPARC T2 processor
64 GB

Software Configuration:

OpenSolaris 2009.06
Java HotSpot(TM) 32-Bit Server, Version 1.6.0_14 Performance Release

Benchmark Description

SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).

Key Points and Best Practices

• Enhancements to the JVM had a major impact on performance.
• Each JVM executed in the FX scheduling class to improve performance by reducing the frequency of context switches.
• Each JVM bound to a separate processor containing 1 core to reduce memory access latency using the physical memory closest to the processor.
  

See Also

• SPECjbb2005 Benchmark Reports

Disclosure Statement

SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results as of 7/17/2009 on http://www.spec.org. SPECjbb2005, Sun Blade T6320 229576 SPECjbb2005 bops, 28697 SPECjbb2005 bops/JVM; IBM p 570 88089 SPECjbb2005 bops, 88089 SPECjbb2005 bops/JVM; Fujitsu TX100 223691 SPECjbb2005 bops, 111846 SPECjbb2005 bops/JVM; IBM x3350 194256 SPECjbb2005 bops, 97128 SPECjbb2005 bops/JVM; Sun SPARC Enterprise T5120 192055 SPECjbb2005 bops, 24007 SPECjbb2005 bops/JVM.

A Sun SPARC Enterprise T5440 server equipped with four UltraSPARC T2 Plus processors running at 1.6GHz, delivered a result of 841380 SPECjbb2005 bops, 26293 SPECjbb2005 bops/JVM when running the SPECjbb2005 benchmark.

• One Sun SPARC Enterprise T5440 (four 1.6GHz UltraSPARC T2 Plus chips, 4RU) demonstrated 5% better performance than the IBM Power 570 (16RU) result of 798752 SPECjbb2005 bops using eight 4.7 GHz POWER6 chips. The IBM system requires twice the number of processor chips and the system requires 4 times more space than the T5440.
• One Sun SPARC Enterprise T5440 (four 1.6GHz UltraSPARC T2 Plus chips, 4RU) has 2.3 times better power/performance than the IBM Power 570 (16RU) that used eight 4.7 GHz POWER6 chips.
• Sun's 1.6GHz UltraSPARC T2 Plus Processor is over 2.3x performance of 4.7 GHz IBM POWER6 processor.
• One Sun SPARC Enterprise T5440 (four 1.6GHz UltraSPARC T2 Plus chips) demonstrated 21% better performance when compared to the Sun SPARC Enterprise T5440 result of 692736 SPECjbb2005 bops using four 1.4GHz UltraSPARC T2 Plus chips.
• The Sun SPARC Enterprise T5440 used OpenSolaris 2009.06 and the Sun JDK 1.6.0_14 Performance Release to obtain this result.

Performance Landscape

SPECjbb2005 Performance Chart (ordered by performance)

bops : SPECjbb2005 Business Operations per Second (bigger is better)

System	Processors					Performance
	Chips	Cores	Threads	GHz	Type	bops	bops/JVM
HP DL585 G6	4	24	24	2.8	AMD Opteron	937207	234302
Sun SPARC Enterprise T5440	4	32	256	1.6	US-T2 Plus	841380	26293
IBM Power 570	8	16	32	4.7	POWER6	798752	99844
Sun SPARC Enterprise T5440	4	32	256	1.4	US-T2 Plus	692736	21648

Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org.

Results and Configuration Summary

Hardware Configuration:

Sun SPARC Enterprise T5440

4 x 1.6 GHz UltraSPARC T2 Plus processors
256 GB

Software Configuration:

OpenSolaris 2009.06
Java HotSpot(TM) 32-Bit Server, Version 1.6.0_14 Performance Release

Benchmark Description

SPECjbb2005 (Java Business Benchmark) measures the performance of a Java implemented application tier (server-side Java). The benchmark is based on the order processing in a wholesale supplier application. The performance of the user tier and the database tier are not measured in this test. The metrics given are number of SPECjbb2005 bops (Business Operations per Second) and SPECjbb2005 bops/JVM (bops per JVM instance).

Key Points and Best Practices

• Enhancements to the JVM had a major impact on performance.
• Each JVM executed in the FX scheduling class to improve performance by reducing the frequency of context switches.
• Each JVM bound to a separate processor containing 1 core to reduce memory access latency using the physical memory closest to the processor.

See Also

• SPECjbb2005 Benchmark Reports

Disclosure Statement:

SPECjbb2005 Sun SPARC Enterprise T5440 (4 chips, 32 cores) 841380 SPECjbb2005 bops, 26293 SPECjbb2005 bops/JVM. Results submitted to SPEC. HP DL585 G5 (4 chips, 24 cores) 937207 SPECjbb2005 bops, 234302 SPECjbb2005 bops/JVM. IBM Power 570 (8 chips, 16 cores) 798752 SPECjbb2005 bops, 99844 SPECjbb2005 bops/JVM. Sun SPARC Enterprise T5440 (4 chips, 32 cores) 692736 SPECjbb2005 bops, 21648 SPECjbb2005 bops/JVM. SPEC, SPECjbb reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 7/20/09

Sun watts were measured on the system during the test.

IBM p 570 8P (4 building blocks) power specifications calculated as 80% of maximum input power reported 7/8/09 in “Facts and Features Report”: ftp://ftp.software.ibm.com/common/ssi/pm/br/n/psb01628usen/PSB01628USEN.PDF

Significance of Results

Sun has a new SPECmail2009 result on a Sun Fire X4275 server and Sun Storage 7110 Unified Storage System running Sun Java Messaging server 6.2.  OpenStorage and ZFS were a key part of the new World Record SPECmail2009.

• The Sun Fire X4275 server, equipped by 2, 2.93GHz Intel Xeon QC X5570 processors, running the Sun Java System Messaging Server 6.2 on Solaris 10 achieved a new World Record 8000 SPECmail_Ent2009 IMAP4 users at 38,348 Sessions/hour. 

• The Sun result was obtained using  about half disk spindles, with the Sun Storage 7110 Unified Storage System, than  Apple's direct attached storage solution.  The Sun submission is the first result using a NAS solution, specifically two Sun Storage 7110 Unified Storage System.
• This benchmark result clearly demonstrates that the Sun Fire X4275 server together with Sun Java System Messaging Server 6.2, Solaris 10 on Sun Storage 7110 Unified Storage Systems can support a large, enterprise level IMAP mail server environment as a reliable low cost solution, delivering best performance and maximizing data integrity with ZFS.

SPECmail 2009 Performance Landscape(ordered by performance)

System	Processors			Performance
	Type	GHz	Ch, Co, Th	SPECmail_Ent2009 Users	SPECmail2009 Sessions/hour
Sun Fire X4275	Intel X5570	2.93	2, 8, 16	8000	38,348
Apple Xserv3,1	Intel X5570	2.93	2, 8, 16	6000	28,887
Sun SPARC Enterprise T5220	UltraSPARC T2	1.4	1, 8, 64	3600	17,316

Notes:

Number of SPECmail_Ent2009 users (bigger is better)
SPECmail2009 Sessions/hour (bigger is better)
Ch, Co, Th: Chips, Cores, Threads

Complete benchmark results may be found at the SPEC benchmark website http://www.spec.org

Results and Configuration Summary

Hardware Configuration:

Sun Fire X4275

2 x 2.93 GHz QC Intel Xeon X5570 processors
72 GB
12 x 300GB, 10000 RPM SAS disk


2 x Sun Storage 7110 Unified Storage System, each with

16 x 146GB SAS 10K RPM

Software Configuration:

O/S: Solaris 10
ZFS
Mail Server: Sun Java System Messaging Server 6.2

Benchmark Description

The SPECmail2009 benchmark measures the ability of corporate e-mail systems to meet today's demanding e-mail users over fast corporate local area networks (LAN). The SPECmail2009 benchmark simulates corporate mail server workloads that range from 250 to 10,000 or more users, using industry standard SMTP and IMAP4 protocols. This e-mail server benchmark creates client workloads based on a 40,000 user corporation, and uses folder and message MIME structures that include both traditional office documents and a variety of rich media content. The benchmark also adds support for encrypted network connections using industry standard SSL v3.0 and TLS 1.0 technology. SPECmail2009 replaces all versions of SPECmail2008, first released in August 2008. The results from the two benchmarks are not comparable.

Software on one or more client machines generates a benchmark load for a System Under Test (SUT) and measures the SUT response times. A SUT can be a mail server running on a single system or a cluster of systems.

A SPECmail2009 'run' simulates a 100% load level associated with the specific number of users, as defined in the configuration file. The mail server must maintain a specific Quality of Service (QoS) at the 100% load level to produce a valid benchmark result. If the mail server does maintain the specified QoS at the 100% load level, the performance of the mail server is reported as SPECmail_Ent2009 SMTP and IMAP Users at SPECmail2009 Sessions per hour. The SPECmail_Ent2009 users at SPECmail2009 Sessions per Hour metric reflects the unique workload combination for a SPEC IMAP4 user.

Key Points and Best Practices

• Each XTA7110 was configured as 1xRAID1 (14x143GB) volume with 2 NFS Shared LUNs, accessed by the SUT via NFSV4. The mailstore volumes were mounted with the nfs mount options: nointr,hard, xattr. There were a total of 4GB/sec Network connections between the SUT and the 7110 Unified Storage systems using 2 NorthStar dual 1GB/sec NICs.
  
• The clients used these Java options: java -d64 -Xms4096m -Xmx4096m -XX:+AggressiveHeap
• See the SPEC Report for all OS, network and messaging server tunings.

See Also

• SPECmail2009 Benchmark Page

Disclosure Statement

SPEC, SPECmail reg tm of Standard Performance Evaluation Corporation. Results as of 07/06/2009 on www.spec.org. SPECmail2009: Sun Fire X4275 (8 cores, 2 chips) SPECmail_Ent2009 8000 users at 38,348 SPECmail2009 Sessions/hour. Apple Xserv3,1 (8 cores, 2 chips) SPECmail_Ent2009 6000 users at 28,887 SPECmail2009 Sessions/hour.

On Madhu Konda's Weblog he writes:

Currently, our performance team takes care of "Middleware" product performance including Glassfish, so check out our blogs from my team members.


http://java.sun.com/docs/performance

http://java.sun.com/performance/reference/whitepapers/6_performance.html

http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html

http://java.sun.com/performance/reference/whitepapers/tuning.html

http://java.sun.com/docs/hotspot/gc1.4.2/faq.htm

one more note:  If you are on JDK 6, I highly recommend that you pick up jdk 6.0_14(update 14) since it has a improvements when using 64-bit large heaps by turning on Compressed OOPs (-XX:+UseCompressedOops when using -d64 switch).

For more info, please see Madhu Konda's Weblog.

Sun recently entered the SPECpower fray with the publication of three results on the SPECpower_ssj2008 benchmark.  Strangely, the three publications documented results on the same hardware platform (Sun Netra X4250) running identical software stacks, but the results were markedly different.  What exactly were we trying to get at?

 Benchmark Configurations

Sun produces robust industrial-grade servers with a range of redundancy features we believe benefit our customers.   These features increase reliability, at the cost of additional power consumption. For example, redundant power supplies and redundant fans allow servers to tolerate faults, and hot-swap capabilities further minimize downtime.

The benchmark run and reporting rules require the incorporation within the tested configuration of all components implied by the model name.  Within these limitations, the first publication was intended to be the best result (that is, the lowest power consumption per unit of performance) achievable on the Sun Netra X4250 platform, by minimizing the configured hardware to the greatest extent possible.

Common Components

All tested configurations had the following components in common:

• System:  Sun Netra X4250
• Processor: 2 x Intel L5408 QC @ 2.13GHz
• 2 x 658 watt redundant AC power supplies
• redundant fans
• standard I/O expansion mezzanine
• standard Telco dry contact alarm

And the same software stack:

• OS: Windows Server 2003 R2 Enterprise X64 Edition SP2
• Drivers: platform-specific drivers from Sun Netra X4250 Tools and Drivers DVD Version 2.1N
• JVM: Java HotSpot 32-Bit Server VM on Windows, version 1.6.0_14
  

Tiny Configuration

In addition to the common hardware components, the tiny configuration was limited to:

• 8 GB of Memory (4 x 2048 MB as PC2-5300F 2Rx8)
• 1 x Sun 146 GB 10K RPM SAS internal drive

This is called the tiny configuration because it seems unlikely that most customers would configure an 8-core server with only one disk and only 1 GB available per core. Nevertheless, from a benchmark point of view, this configuration gave the best result.

Typical Configuration

The other two results were both produced on a configuration we considered much more typical of configurations that are actually ordered by customers.  In addition to the common hardware, these typical configuration included:

• 32 GB of Memory (8 x 4096 MB as PC2-5300F)
• 4 x Sun 146 GB 10K RPM SAS internal drives
• 1 x Sun x8 PCIe Quad Gigabit Ethernet option card (X4447A-Z)

Nothing special was done with the additional components.  The added memory increased the performance component of the benchmark. The other components were installed and configured but allowed to sit idle, so consumed less power than they would have under load.

One Other Thing: Tuning for Performance

So one thing we're getting at is the difference in power consumption between a small configuration optimized for a power-performance benchmark and a typical configuration optimized for customer workloads.  Hardware (power consumption) is only half of the benchmark--the other half being the performance achieved by the System Under Test (SUT).

Tuning Choices 

In all three publications the identical tunings were applied at the software level: identical java command-line arguments and JVM-to-processor affinity.  We also applied, in the case of the better results, the common (but usually non-default) BIOS-level optimization of disabling hardware prefetcher and adjacent cache line prefetch.  These optimizations are commonly applied to produce optimized SPECpower_ssj2008 results but it is unlikely that many production applications would benefit from these settings.  To demonstrate the effect of this tuning, the final result was generated with standard BIOS settings.

 And just so we couldn't be accused of sand-bagging the results, the number of JVMs was increased in the typical configurations to take advantage of the additional memory populated over and above the tiny configuration.  Additional performance was achieved but sadly it doesn't compensate for the higher power consumption of all that memory.

So in summary we tuned:

• Tiny Configuration: non-default BIOS settings
• Typical Configuration 1: non-default BIOS settings; additional JVMs to utilize added memory
• Typical Configuration 2: default BIOS settings; additional JVMs to utilize added memory

At the OS level, all tunings  were identical.

Results 

The results are summarized in this table:

System (Click system for SPEC full disclosure)	Processors		Performance
	Model	GHz	Metric overall ssj_ops/watt	Peak Performance ssj_ops	Peak Power watts	Idle Power watts
Sun Netra X4250 (8GB non-default BIOS)	L5408	2.13	600	244832	226	174
Sun Netra X425 (32GB non-default BIOS)	L5408	2.13	478	251555	294	226
Sun Netra X4250 (32GB default BIOS)	L5408	2.13	437	229828	296	225

Conclusions

• The measurement and reporting methods of the benchmark encourage small memory configurations.  Comparing the first and second result, adding additional memory yielded minimal performance improvement (from 244832 to 251555) but a large increase in power consumption, 68 watts at peak.

• In our opinion, unrealistically small configurations yield the best results on this benchmark.  On the more typical system, the benchmark overall metric decreased from 600 overall ssj_ops per watt to 478 overall ssj_ops per watt, despite our best effort to utilize the additional configured memory.

• On typical configurations, reverting to default BIOS settings resulted in a significant decrease in performance (from 25155 to 229828) with no corresponding decrease in power consumption (essentially identical for both results).

Configurations typical of customer systems (with adequate memory, internal disks, and option cards) consume more power than configurations which are commonly benchmarked, while providing no corresponding improvement in SPECpower_ssj2008 benchmark performance. The result is a lower overall power-performance metric on typical configurations and a lack of published benchmark results on robust systems with the capacities and redundancies that enterprise customers desire.

Fair Use Disclosure

SPEC, SPECpower, and SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation.  All results from the SPEC website (www.spec.com)  as of June 5, 2009.  For a complete set of accepted results refer to that site.

A sample of the various Sun and partner technologies to be discussed:
OpenSolaris, Solaris, Linux, Windows, vmware, gcc, Java, Glassfish, MySQL, Java, Sun-Studio, ZFS, dtrace, perflib, Oracle, DB2, Sybase, OpenStorage, CMT, SPARC64, X64, X86, Intel, AMD