BestPerf: Performance & Best Practices

Significance of Results

A Sun Blade 6048 chassis with 48 Sun Blade X6275 server modules ran benchmarks using the NAMD molecular dynamics applications software. NAMD is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. NAMD is driven by major trends in computing and structural biology and received a 2002 Gordon Bell Award.

• The cluster of 32 Sun Blade X6275 server modules was 9.2x faster than the 512 processor configuration of the IBM BlueGene/L.

• The cluster of 48 Sun Blade X6275 server modules exhibited excellent scalability for NAMD molecular dynamics simulation, up to 37.8x speedup for 48 blades relative to 1 blade.

• For largest molecule considered, the cluster of 48 Sun Blade X6275 server modules achieved a throughput of 0.028 seconds per simulation step.

Molecular dynamics simulation is important to biological and materials science research. Molecular dynamics is used to determine the low energy conformations or shapes of a molecule. These conformations are presumed to be the biologically active conformations.

Performance Landscape

The NAMD Performance Benchmarks web page plots the performance of NAMD when the ApoA1 benchmark is executed on a variety of clusters. The performance is expressed in terms of the time in seconds required to execute one step of the molecular dynamics simulation, multiplied by the number of "processors" on which NAMD executes in parallel. The following table compares the performance of the Sun Blade X6275 cluster to several of the clusters for which performance is reported on the web page. In this table, the performance is expressed in terms of the time in seconds required to execute one step of the molecular dynamics simulation. A smaller number implies better performance.

Cluster Name and Interconnect	Throughput for 128 Cores (seconds per step)	Throughput for 256 Cores (seconds per step)	Throughput for 512 Cores (seconds per step)
Sun Blade X6275 InfiniBand	0.014	0.0073	0.0048
Cambridge Xeon/3.0 InfiniPath	0.016	0.0088	0.0056
NCSA Xeon/2.33 InfiniBand	0.019	0.010	0.008
AMD Opteron/2.2 InfiniPath	0.025	0.015	0.008
IBM HPCx PWR4/1.7 Federation	0.039	0.021	0.013
SDSC IBM BlueGene/L MPI	0.108	0.061	0.044

The following tables report results for NAMD molecular dynamics using a cluster of Sun Blade X6275 server modules. The performance of the cluster is expressed in terms of the time in seconds that is required to execute one step of the molecular dynamics simulation. A smaller number implies better performance.

Blades	Cores	STMV molecule (1)			f1 ATPase molecule (2)			ApoA1 molecule (3)
		Thruput (secs/ step)	spdup	effi'cy	Thruput (secs/ step)	spdup	effi'cy	Thruput (secs/ step)	spdup	effi'cy
48	768	0.0277	37.8	79%	0.0075	35.2	73%	0.0039	22.2	46%
36	576	0.0324	32.3	90%	0.0096	27.4	76%	0.0045	19.3	54%
32	512	0.0368	28.4	89%	0.0104	25.3	79%	0.0048	18.1	57%
24	384	0.0481	21.8	91%	0.0136	19.3	80%	0.0066	13.2	55%
16	256	0.0715	14.6	91%	0.0204	12.9	81%	0.0073	11.9	74%
12	192	0.0875	12.0	100%	0.0271	9.7	81%	0.0096	9.1	76%
8	128	0.1292	8.1	101%	0.0337	7.8	98%	0.0139	6.3	79%
4	64	0.2726	3.8	95%	0.0666	4.0	100%	0.0224	3.9	98%
1	16	1.0466	1.0	100%	0.2631	1.0	100%	0.0872	1.0	100%

spdup - speedup versus 1 blade result
effi'cy - speedup efficiency versus 1 blade result

(1) Satellite Tobacco Mosaic Virus (STMV) molecule, 1,066,628 atoms, 12 Angstrom cutoff, Langevin dynamics, 500 time steps
(2) f1 ATPase molecule, 327,506 atoms, 11 Angstrom cutoff, particle mesh Ewald dynamics, 500 time steps
(3) ApoA1 molecule, 92,224 atoms, 12 Angstrom cutoff, particle mesh Ewald dynamics, 500 time steps

Results and Configuration Summary

Hardware Configuration

48 x Sun Blade X6275, each with
2 x (2 x 2.93 GHz Intel QC Xeon X5570 (Nehalem) processors)
2 x (24 GB memory)
Hyper-Threading (HT) off, Turbo Mode on

Software Configuration

SUSE Linux Enterprise Server 10 SP2 kernel version 2.6.16.60-0.31_lustre.1.8.0.1-smp
OpenMPI 1.3.2
gcc 4.1.2 (1/15/2007), gfortran 4.1.2 (1/15/2007)

Benchmark Description

Molecular dynamics simulation is widely used in biological and materials science research. NAMD is a public-domain molecular dynamics software application for which a variety of molecular input directories are available. Three of these directories define:

• the Satellite Tobacco Mosaic Virus (STMV) that comprises 1,066,628 atoms
• the f1 ATPase enzyme that comprises 327,506 atoms
• the ApoA1 enzyme that comprises 92,224 atoms

Each input directory also specifies the type of molecular dynamics simulation to be performed, for example, Langevin dynamics with a 12 Angstrom cutoff for 500 time steps, or particle mesh Ewald dynamics with an 11 Angstrom cutoff for 500 time steps.

Key Points and Best Practices

Models with large numbers of atoms scale better than models with small numbers of atoms.

The Intel QC X5570 processors include a turbo boost feature coupled with a speed-step option in the CPU section of the Advanced BIOS settings. Under specific circumstances, this can provide cpu overclocking which increases the processor frequency from 2.93GHz to 3.33GHz. This feature was was enabled when generating the results reported here.

See Also

• NAMD - Scalable Molecular Dynamics
• NAMD Performance Benchmarks

Disclosure Statement

NAMD, see http://www.ks.uiuc.edu/Research/namd/performance.html for more information, results as of 11/17/2009.

TPC-C Sun SPARC Enterprise T5440 with Oracle RAC World Record Database Result

Sun and Oracle demonstrate the World's fastest database performance. Sun Microsystems using 12 Sun SPARC Enterprise T5440 servers, 60 Sun Storage F5100 Flash arrays and Oracle 11g Enterprise Edition with Real Application Clusters and Partitioning delivered a world-record TPC-C benchmark result.

• The 12-node Sun SPARC Enterprise T5440 server cluster result delivered a world record TPC-C benchmark result of 7,646,486.7 tpmC and $2.36 $/tpmC (USD) using Oracle 11g R1 on a configuration available 12/14/09.

• The 12-node Sun SPARC Enterprise T5440 server cluster beats the performance of the IBM Power 595 (5GHz) with IBM DB2 9.5 database by 26% and has 16% better price/performance on the TPC-C benchmark.
  

• The complete Oracle/Sun solution used 10.7x better computational density than the IBM configuration (computational density = performance/rack).

• The complete Oracle/Sun solution used 8 times fewer racks than the IBM configuration.

• The complete Oracle/Sun solution has 5.9x better power/performance than the IBM configuration.

• The 12-node Sun SPARC Enterprise T5440 server cluster beats the performance of the HP Superdome (1.6GHz Itanium2) by 87% and has 19% better price/performance on the TPC-C benchmark.

• The Oracle/Sun solution utilized Sun FlashFire technology to deliver this result. The Sun Storage F5100 flash array was used for database storage.

• Oracle 11g Enterprise Edition with Real Application Clusters and Partitioning scales and effectively uses all of the nodes in this configuration to produce the world record performance.

• This result showed Sun and Oracle's integrated hardware and software stacks provide industry-leading performance.

More information on this benchmark will be posted in the next several days.

Performance Landscape

TPC-C results (sorted by tpmC, bigger is better)

System	tpmC	Price/tpmC	Avail	Database	Cluster	Racks	w/KtpmC
12 x Sun SPARC Enterprise T5440	7,646,487	2.36 USD	12/14/09	Oracle 11g RAC	Y	9	9.6
IBM Power 595	6,085,166	2.81 USD	12/10/08	IBM DB2 9.5	N	76	56.4
HP Integrity Superdome	4,092,799	2.93 USD	08/06/07	Oracle 10g R2	N	46	to be added

Avail - Availability date
w/KtmpC - Watts per 1000 tpmC
Racks - clients, servers, storage, infrastructure

Sun and IBM TPC-C Response times

System	tpmC	Response Time New Order 90th%	Response Time New Order Average
12 x Sun SPARC Enterprise T5440	7,646,487	0.170	0.168
IBM Power 595	6,085,166	1.69	1.22
Response Time Ratio - Sun Better		9.9x	7.3x

Sun uses 7x comparison to highlight the differences in response times between Sun's solution and IBM.  Although notice that Sun is 10x faster on New Order transactions that finish in the 90% percentile.

It is also interesting to note that none of Sun's response times, avg or 90th percentile, for any transaction is over 0.25 seconds. While IBM does not have even one interactive transaction, not even the menu, below 0.50 seconds. Graphs of Sun's and IBM's response times for New-Order can be found in the full disclosure reports on TPC's website TPC-C Official Result Page.

Results and Configuration Summary

Hardware Configuration:

9 racks used to hold

Servers:

12 x Sun SPARC Enterprise T5440
4 x 1.6 GHz UltraSPARC T2 Plus
512 GB memory
10 GbE network for cluster

Storage:

60 x Sun Storage F5100 Flash Array
61 x Sun Fire X4275, Comstar SAS target emulation
24 x Sun StorageTek 6140 (16 x 300 GB SAS 15K RPM)
6 x Sun Storage J4400
3 x 80-port Brocade FC switches

Clients:

24 x Sun Fire X4170, each with
2 x 2.53 GHz X5540
48 GB memory

Software Configuration:

Solaris 10 10/09
OpenSolaris 6/09 (COMSTAR) for Sun Fire X4275
Oracle 11g Enterprise Edition with Real Application Clusters and Partitioning
Tuxedo CFS-R Tier 1
Sun Web Server 7.0 Update 5

Benchmark Description

TPC-C is an OLTP system benchmark. It simulates a complete environment where a population of terminal operators executes transactions against a database. The benchmark is centered around the principal activities (transactions) of an order-entry environment. These transactions include entering and delivering orders, recording payments, checking the status of orders, and monitoring the level of stock at the warehouses.

See Also

• TPC-C Official Result Page
  

• www.tpc.org Website

Disclosure Statement

TPC Benchmark C, tpmC, and TPC-C are trademarks of the Transaction Performance Processing Council (TPC). 12-node Sun SPARC Enterprise T5440 Cluster (1.6GHz UltraSPARC T2 Plus, 4 processor) with Oracle 11g Enterprise Edition with Real Application Clusters and Partitioning, 7,646,486.7 tpmC, $2.36/tpmC. Available 12/14/09. IBM Power 595 (5GHz Power6, 32 chips, 64 cores, 128 threads) with IBM DB2 9.5, 6,085,166 tpmC, $2.81/tpmC, available 12/10/08. HP Integrity Superdome(1.6GHz Itanium2, 64 processors, 128 cores, 256 threads) with Oracle 10g Enterprise Edition, 4,092,799 tpmC, $2.93/tpmC. Available 8/06/07. Source: www.tpc.org, results as of 11/5/09.

The Oracle BI EE workload was run on two Sun SPARC Enterprise T5440 servers and acheived world record performance.

• Two Sun SPARC Enterprise T5440 servers with four 1.6 GHz UltraSPARC T2 Plus processors delivered the best performance of 50K concurrent users on the Oracle BI EE 10.1.3.4 benchmark with Oracle 11g database running on free and open Solaris 10.

• The two node Sun SPARC Enterprise T5440 servers with Oracle BI EE running on Solaris 10 using 8 Solaris Containers shows 1.8x scaling over Sun's previous one node SPARC Enterprise T5440 server result with 4 Solaris Containers.

• The two node SPARC Enterprise T5440 servers demonstrated the performance and scalability of the UltraSPARC T2 Plus processor demonstrating 50K users can be serviced with 0.2776 sec response time.

• The Sun SPARC Enterprise T5220 server was used as an NFS server with 4 internal SSDs and the ZFS file system which showed significant I/O performance improvement over traditional disk for Business Intelligence Web Catalog activity.

• IBM has not published any POWER6 processor based results on this important benchmark.

Performance Landscape

System	Processors			Users
	Chips	GHz	Type
2 x Sun SPARC Enterprise T5440	8	1.6	UltraSPARC T2 Plus	50,000
1 x Sun SPARC Enterprise T5440	4	1.6	UltraSPARC T2 Plus	28,000
5 x Sun Fire T2000	1	1.2	UltraSPARC T1	10,000

Results and Configuration Summary

Hardware Configuration:

2 x Sun SPARC Enterprise T5440 (1.6GHz/128GB)
1 x Sun SPARC Enterprise T5220 (1.2GHz/64GB) and 4 SSDs (used as NFS server)

Software Configuration:

Solaris10 05/09
Oracle BI EE 10.1.3.4
Oracle 11gR1

Benchmark Description

The objective of this benchmark is to highlight how Oracle BI EE can support pervasive deployments in large enterprises, using minimal hardware, by simulating an organization that needs to support more than 25,000 active concurrent users, each operating in mixed mode: ad-hoc reporting, application development, and report viewing.

The user population was divided into a mix of administrative users and business users. A maximum of 28,000 concurrent users were actively interacting and working in the system during the steady-state period. The tests executed 580 transactions per second, with think times of 60 seconds per user, between requests. In the test scenario 95% of the workload consisted of business users viewing reports and navigating within dashboards. The remaining 5% of the concurrent users, categorized as administrative users, were doing application development.

The benchmark scenario used a typical business user sequence of dashboard navigation, report viewing, and drill down. For example, a Service Manager logs into the system and navigates to his own set of dashboards viz. .Service Manager.. The user then selects the .Service Effectiveness. dashboard, which shows him four distinct reports, .Service Request Trend., .First Time Fix Rate., .Activity Problem Areas., and .Cost Per completed Service Call . 2002 till 2005. . The user then proceeds to view the .Customer Satisfaction. dashboard, which also contains a set of 4 related reports. He then proceeds to drill-down on some of the reports to see the detail data. Then the user proceeds to more dashboards, for example .Customer Satisfaction. and .Service Request Overview.. After navigating through these dashboards, he logs out of the application

This benchmark did not use a synthetic database schema. The benchmark tests were run on a full production version of the Oracle Business Intelligence Applications with a fully populated underlying database schema. The business processes in the test scenario closely represents a true customer scenario.

See Also

• Dual T5440 White Paper
• Single T5440 White Paper
• Giri Mandalika Blog
• Oracle Business Intelligence Website, BUSINESS INTELLIGENCE has other results

Disclosure Statement

Oracle BI EE benchmark results 10/13/2009, see

Significance of Results

Clusters of Sun Blade X6275 and X6270 server modules were used to run benchmarks using the CP2K ab-initio dynamics applications software.

• For the X6270 cluster with Dual Data Rate (DDR) InfiniBand the rate of increase of scalability slows dramatically at 16 nodes, whereas for the X6275 cluster with QDR InfiniBand the scalability continues to 72 nodes.

• For 64 nodes, the speed of the Sun Blade X6275 cluster with QDR InfiniBand was 2.7X that of a Sun Blade X6270 cluster with DDR InfiniBand.

Ab-initio dynamics simulation is important to materials science research.  Dynamics simulation is used to determine the trajectories of atoms or molecules over time.

Performance Landscape

The CP2K Bulk Water Benchmarks web page plots the performance of CP2K ab-initio dynamics benchmarks that have from 32 to 512 water molecules for a cluster that comprises two 2.66GHz Xeon E5430 quad core CPUs per node and that uses Dual Data Rate InfiniBand.

The following table reports the execution time for the 512 water molecule benchmark when executed on the Sun Blade X6275 cluster having Quad Data Rate InfiniBand and on the Sun Blade X6270 cluster having Dual Data Rate InfiniBand. Each node of either Sun Blade cluster comprises two 2.93GHz Intel Xeon X5570 quad core CPUs. In the following table, the performance is expressed in terms of the "wall clock" time in seconds required to execute ten steps of the ab-initio dynamics simulation for 512 water molecules. A smaller number implies better performance.

Number of Nodes	X6275 QDR InfiniBand (seconds for 10 steps)	X6270 DDR InfiniBand (seconds for 10 steps)
96		1184.36
72	564.16
64	598.41	1591.35
32	706.82	1436.49
24	950.02	1752.20
16	1227.73	2119.50
12	1440.16	1739.26
8	1876.95	2120.73
4	3408.39	3705.44

Results and Configuration Summary

Hardware Configuration:

Sun Blade[tm] 6048 Modular System with 3 shelves, each shelf with
12 x Sun Blade X6275, each blade with

2 x (2 x 2.93 GHz Intel QC Xeon X5570 processors)
2 x (24 GB memory)
Hyper-Threading (HT) off, Turbo Mode on

QDR InfiniBand

96 x Sun Blade X6270, each blade with

2 x 2.93 GHz Intel QC Xeon X5570 processors)
1 x (24 GB memory)
Hyper-Threading (HT) off, Turbo Mode off

DDR InfiniBand

Software Configuration:

SUSE Linux Enterprise Server 10 SP2 kernel version 2.6.16.60-0.31_lustre.1.8.0.1-smp
OpenMPI 1.3.2
Sun Studio 12 f90 compiler, ScaLAPACK, BLACS and Performance Libraries
FFTW (Fastest Fourier Transform in the West) 3.2.1

Benchmark Description

CP2K is a parallel ab-initio dynamics code that is designed to perform atomistic and molecular simulations of solid state, liquid, molecular and biological systems. It provides a general framework for different methods such as e.g. density functional theory (DFT) using a mixed Gaussian and plane waves approach (GPW), and classical pair and many-body potentials.

Ab-initio dynamics simulation is widely used in materials science research. CP2K is a public-domain ab-initio dynamics software application.

Key Points and Best Practices

• QDR InfiniBand scales better than DDR InfiniBand.

• The Intel QC X5570 processors include a turbo boost feature coupled with a speed-step option in the CPU section of the Advanced BIOS settings. Under specific circumstances, this can provide cpu overclocking which increases the processor frequency from 2.93GHz to 3.2GHz. This feature was was enabled for the X6275 and disabled for the X6270 when generating the results reported here.

See Also

• CP2K Developers Home Page

Disclosure Statement

CP2K, see http://cp2k.berlios.de/ for more information, results as of 10/13/2009.

Significance of Results

• Four Sun Blade X6270 (2 processors, 8 cores, 16 threads), running SAP ERP application Release 6.0 Enhancement Pack 4 (Unicode) with Oracle Database on top of Solaris 10 OS delivered the highest eight-processor result on the two-tier SAP SD-Parallel Standard Application Benchmark, as of Oct 12th, 2009.

• Four Sun Blade X6270 servers with Intel Xeon X5570 processors achieved 1.9x performance improvement from two Sun Blade X6270 with the same processors.

• Two Sun Blade X6270 (2 processors, 8 cores, 16 threads), running SAP ERP application Release 6.0 Enhancement Pack 4 (Unicode) with Oracle Database on top of Solaris 10 OS delivered the highest four-processor result on the two-tier SAP SD-Parallel Standard Application Benchmark, as of Oct 12th, 2009.

• Two Sun Blade X6270 servers with Intel Xeon X5570 processors achieved 1.9x performance imporvement over a single 2-processor Sun Blade X6270 system.

• A one node Sun Blade X6270 server with Intel Xeon X5570 processors running Oracle RAC delivers the same result as a Sun Fire X4270 server with Intel Xeon X5570 processors running Oracle with no performance difference between Oracle 10g and Oracle 10g RAC.

• This benchmark highlights the near-linear scaling of Oracle 10g Real Application Cluster runs on Sun Microsystems hardware in a SAP environment.

• In January 2009, a new version, the Two-tier SAP ERP 6.0 Enhancement Pack 4 (Unicode) Standard Sales and Distribution (SD) Benchmark, was released. This new release has higher cpu requirements and so yields from 25-50% fewer users compared to the previous Two-tier SAP ERP 6.0 (non-unicode) Standard Sales and Distribution (SD) Benchmark. 10-30% of this is due to the extra overhead from the processing of the larger character strings due to Unicode encoding. See this SAP Note for more details.

• Unicode is a computing standard that allows for the representation and manipulation of text expressed in most of the world's writing systems. Before the Unicode requirement, this benchmark used ASCII characters meaning each was just 1 byte. The new version of the benchmark requires Unicode characters and the Application layer (where ~90% of the cycles in this benchmark are spent) uses a new encoding, UTF-16, which uses 2 bytes to encode most characters (including all ASCII characters) and 4 bytes for some others. This requires computers to do more computation and use more bandwidth and storage for most character strings. Refer to the above SAP Note for more details.

Performance Landscape

SAP SD-Parallel 2-Tier Performance Table (in decreasing performance order).

SAP ERP 6.0 Enhancement Pack 4 (Unicode) Results
(New version of the benchmark as of January 2009)

System	OS Database	Users	SAP ERP/ECC Release	SAPS	Date
Four Sun Blade X6270 2xIntel Xeon X5570 @2.93GHz 48 GB	Solaris 10 Oracle 10g Real Application Clusters	13,718	2009 6.0 EP4 (Unicode)	75,762	12-Oct-09
Two Sun Blade X6270 2xIntel Xeon X5570 @2.93GHz 48 GB	Solaris 10 Oracle 10g Real Application Clusters	7,220	2009 6.0 EP4 (Unicode)	39,420	12-Oct-09
One Sun Blade X6270 2xIntel Xeon X5570 @2.93GHz 48 GB	Solaris 10 Oracle 10g Real Application Clusters	3,800	2009 6.0 EP4 (Unicode)	20,750	12-Oct-09
Sun Fire X4270 2xIntel Xeon X5570 @2.93GHz 48 GB	Solaris 10 Oracle 10g	3,800	2009 6.0 EP4 (Unicode)	21,000	21-Aug-09

Complete benchmark results may be found at the SAP benchmark website http://www.sap.com/benchmark.

Results and Configuration Summary

Four Sun Blade X6270 Servers, each with two Intel Xeon X5570 2.93 GHz(2 processors, 8 cores, 16 threads)

Number of SAP SD benchmark users:			13,718
Average dialog response time:			0.86 seconds
Throughput:
	Dialog steps/hour:		4,545,729
	SAPS:		75,762
SAP Certification:			2009041

Two Sun Blade X6270 Servers, each with two Intel Xeon X5570 2.93 GHz(2 processors, 8 cores, 16 threads)

Number of SAP SD benchmark users:			7,220
Average dialog response time:			0.99 seconds
Throughput:
	Dialog steps/hour:		2,365,000
	SAPS:		39,420
SAP Certification:			2009040

One Sun Blade X6270 Servers, with two Intel Xeon X5570 2.93 GHz(2 processors, 8 cores, 16 threads)

Number of SAP SD benchmark users:			3,800
Average dialog response time:			0.99 seconds
Throughput:
	Dialog steps/hour:		1,245,000
	SAPS:		20,750
SAP Certification:			2009039

Software:

Oracle 10g Real Application Clusters
Solaris 10 OS

Benchmark Description

The SAP Standard Application Sales and Distribution - Parallel (SD-Parallel) Benchmark is a two-tier ERP business test that is indicative of full business workloads of complete order processing and invoice processing, and demonstrates the ability to run both the application and database software on a single system. The SAP Standard Application SD Benchmark represents the critical tasks performed in real-world ERP business environments.
SD Versus SD-Parallel
The SD-Parallel Benchmark consists of the same transactions and user interaction steps as the SD Benchmark. This means that the SD-Parallel Benchmark runs the same business processes as the SD Benchmark. The difference between the benchmarks is the technical data distribution. An Additional Rule for Parallel and Distributed Databases
The additional rule is: Equally distribute the benchmark users across all database nodes for the used benchmark clients (round-robin-method). Following this rule, all database nodes work on data of all clients. This avoids unrealistic configurations such as having only one client per database node.
The SAP Benchmark Council agreed to give the parallel benchmark a different name so that the difference can be easily recognized by any interested parties - customers, prospects, and analysts. The naming convention is SD-Parallel for Sales & Distribution - Parallel.
SAP is one of the premier world-wide ERP application providers, and maintains a suite of benchmark tests to demonstrate the performance of competitive systems on the various SAP products.

Disclosure Statement

SAP SD benchmark based on SAP enhancement package 4 for SAP ERP 6.0 (Unicode) application benchmark as of Oct 12th, 2009: Four Sun Blade X6270 (each 2 processors, 8 cores, 16 threads) 13,718 SAP SD Users, 2x 2.93 GHz Intel Xeon x5570, each 48 GB memory, running two-tier SAP Sales and Distribution Parallel (SD-Parallel) standard SAP SD benchmark with Oracle 10g Real Application Clusters and Solaris 10, Cert# 2009041. Two Sun Blade X6270 (each 2 processors, 8 cores, 16 threads) 7,220 SAP SD Users, 2x 2.93 GHz Intel Xeon x5570, each 48 GB memory, running two-tier SAP Sales and Distribution Parallel (SD-Parallel) standard SAP SD benchmark with Oracle 10g Real Application Clusters and Solaris 10, Cert# 2009040. Sun Blade X6270 (2 processors, 8 cores, 16 threads) 3,800 SAP SD Users, 2x 2.93 GHz Intel Xeon x5570, 48 GB memory, running two-tier SAP Sales and Distribution Parallel (SD-Parallel) standard SAP SD benchmark with Oracle 10g Real Application Clusters and Solaris 10, Cert# 2009039. Sun Fire X4270 (2 processors, 8 cores, 16 threads) 3,800 SAP SD Users, 2x 2.93 GHz Intel Xeon x5570, 48 GB memory, running two-tier SAP Sales and Distribution (SD) standard SAP SD benchmark with Oracle 10g and Solaris 10, Cert# 2009033.

SAP, R/3, reg TM of SAP AG in Germany and other countries. More info www.sap.com/benchmark

TPC-C Sun SPARC Enterprise T5440 with Oracle RAC World Record Database Result

Sun and Oracle demonstrate the World's fastest database performance. Sun Microsystems using 12 Sun SPARC Enterprise T5440 servers, 60 Sun Storage F5100 Flash arrays and Oracle 11g Enterprise Edition with Real Application Clusters and Partitioning delivered a world-record TPC-C benchmark result.

• The 12-node Sun SPARC Enterprise T5440 server cluster result delivered a world record TPC-C benchmark result of 7,646,486.7 tpmC and $2.36 $/tpmC (USD) using Oracle 11g R1 on a configuration available 12/14/09.

• The 12-node Sun SPARC Enterprise T5440 server cluster beats the performance of the IBM Power 595 (5GHz) with IBM DB2 9.5 database by 26% and has 16% better price/performance on the TPC-C benchmark.

• The complete Oracle/Sun solution used 10.7x better computational density than the IBM configuration (computational density = performance/rack).

• The complete Oracle/Sun solution used 8 times fewer racks than the IBM configuration.

• The complete Oracle/Sun solution has 5.9x better power/performance than the IBM configuration.

• The 12-node Sun SPARC Enterprise T5440 server cluster beats the performance of the HP Superdome (1.6GHz Itanium2) by 87% and has 19% better price/performance on the TPC-C benchmark.

• The Oracle/Sun solution utilized Sun FlashFire technology to deliver this result. The Sun Storage F5100 flash array was used for database storage.

• Oracle 11g Enterprise Edition with Real Application Clusters and Partitioning scales and effectively uses all of the nodes in this configuration to produce the world record performance.

• This result showed Sun and Oracle's integrated hardware and software stacks provide industry-leading performance.

More information on this benchmark will be posted in the next several days.

Performance Landscape

TPC-C results (sorted by tpmC, bigger is better)

System	tpmC	Price/tpmC	Avail	Database	Cluster	Racks	w/KtpmC
12 x Sun SPARC Enterprise T5440	7,646,487	2.36 USD	12/14/09	Oracle 11g RAC	Y	9	9.6
IBM Power 595	6,085,166	2.81 USD	12/10/08	IBM DB2 9.5	N	76	56.4
Bull Escala PL6460R	6,085,166	2.81 USD	12/15/08	IBM DB2 9.5	N	71	56.4
HP Integrity Superdome	4,092,799	2.93 USD	08/06/07	Oracle 10g R2	N	46	to be added

Avail - Availability date
w/KtmpC - Watts per 1000 tpmC
Racks - clients, servers, storage, infrastructure

Results and Configuration Summary

Hardware Configuration:

9 racks used to hold

Servers:

12 x Sun SPARC Enterprise T5440
4 x 1.6 GHz UltraSPARC T2 Plus
512 GB memory
10 GbE network for cluster

Storage:

60 x Sun Storage F5100 Flash Array
61 x Sun Fire X4275, Comstar SAS target emulation
24 x Sun StorageTek 6140 (16 x 300 GB SAS 15K RPM)
6 x Sun Storage J4400
3 x 80-port Brocade FC switches

Clients:

24 x Sun Fire X4170, each with
2 x 2.53 GHz X5540
48 GB memory

Software Configuration:

Solaris 10 10/09
OpenSolaris 6/09 (COMSTAR) for Sun Fire X4275
Oracle 11g Enterprise Edition with Real Application Clusters and Partitioning
Tuxedo CFS-R Tier 1
Sun Web Server 7.0 Update 5

Benchmark Description

TPC-C is an OLTP system benchmark. It simulates a complete environment where a population of terminal operators executes transactions against a database. The benchmark is centered around the principal activities (transactions) of an order-entry environment. These transactions include entering and delivering orders, recording payments, checking the status of orders, and monitoring the level of stock at the warehouses.

POSTSCRIPT: Here are some comments on IBM's grasping-at-straws-perf/core attacks on the TPC-C result:
c0t0d0s0 blog: "IBM's Reaction to Sun&Oracle TPC-C

See Also

• TPC-C Official Result Page
  

• www.tpc.org Website

Disclosure Statement

TPC Benchmark C, tpmC, and TPC-C are trademarks of the Transaction Performance Processing Council (TPC). 12-node Sun SPARC Enterprise T5440 Cluster (1.6GHz UltraSPARC T2 Plus, 4 processor) with Oracle 11g Enterprise Edition with Real Application Clusters and Partitioning, 7,646,486.7 tpmC, $2.36/tpmC. Available 12/14/09. IBM Power 595 (5GHz Power6, 32 chips, 64 cores, 128 threads) with IBM DB2 9.5, 6,085,166 tpmC, $2.81/tpmC, available 12/10/08. HP Integrity Superdome(1.6GHz Itanium2, 64 processors, 128 cores, 256 threads) with Oracle 10g Enterprise Edition, 4,092,799 tpmC, $2.93/tpmC. Available 8/06/07. Source: www.tpc.org, results as of 10/11/09.

Significance of Results

Results are presented for the Weather Research and Forecasting (WRF) code running on twelve Sun Blade X6275 server modules, housed in the Sun Blade 6048 chassis, using the 2.5 km CONUS benchmark dataset.

• The Sun Blade X6275 cluster was able to achieve 373 GFLOP/s on the CONUS 2.5-KM Dataset.
• The results demonstrate an 91% speedup efficiency, or 11x speedup, from 1 to 12 blades.
• The current results results were run with turbo on.

Performance Landscape

Performance is expressed in terms "simulation speedup" which is the ratio of the simulated time step per iteration to the average wall clock time required to compute it. A larger number implies better performance.

The current results were run with turbo mode on.

WRF 3.0.1.1: Weather Research and Forecasting CONUS 2.5-KM Dataset
# Blade	# Node	# Proc	# Core	Performance (Simulation Speedup)		Computation Rate GFLOP/sec		Speedup/Efficiency (vs. 1 blade)		Turbo On Relative Perf
				Turbo On	Turbo Off	Turbo On	Turbo Off	Turbo On	Turbo Off
12	24	48	192	13.58	12.93	373.0	355.1	11.0 / 91%	10.4 / 87%	+6%
8	16	32	128	9.27		254.6		7.5 / 93%
6	12	24	96	7.03	6.60	193.1	181.3	5.7 / 94%	5.3 / 89%	+7%
4	8	16	64	4.74		130.2		3.8 / 96%
2	4	8	32	2.44		67.0		2.0 / 98%
1	2	4	16	1.24	1.24	34.1	34.1	1.0 / 100%	1.0 / 100%	+0%

Results and Configuration Summary

Hardware Configuration:

Sun Blade 6048 Modular System
12 x Sun Blade X6275 Server Modules, each with
4 x 2.93 GHz Intel QC X5570 processors
24 GB (6 x 4GB)
QDR InfiniBand
HT disabled in BIOS
Turbo mode enabled in BIOS

Software Configuration:

OS: SUSE Linux Enterprise Server 10 SP 2
Compiler: PGI 7.2-5
MPI Library: Scali MPI v5.6.4
Benchmark: WRF 3.0.1.1
Support Library: netCDF 3.6.3

Benchmark Description

The Weather Research and Forecasting (WRF) Model is a next-generation mesoscale numerical weather prediction system designed to serve both operational forecasting and atmospheric research needs. WRF is designed to be a flexible, state-of-the-art atmospheric simulation system that is portable and efficient on available parallel computing platforms. It features multiple dynamical cores, a 3-dimensional variational (3DVAR) data assimilation system, and a software architecture allowing for computational parallelism and system extensibility.

Dataset used:

Single domain, large size 2.5KM Continental US (CONUS-2.5K)

• 1501x1201x35 cell volume
• 6hr, 2.5km resolution dataset from June 4, 2005
• Benchmark is the final 3hr simulation for hrs 3-6 starting from a provided restart file; the benchmark may also be performed (but seldom reported) for the full 6hrs starting from a cold start
• Iterations output at every 15 sec of simulation time, with the computation cost of each time step ~412 GFLOP
http://www.mmm.ucar.edu/wrf/WG2/benchv3/2_5KM_Results_20090518.htm

Key Points and Best Practices

• Processes were bound to processors in round-robin fashion.
• Model simulation time is 15 seconds per iteration as defined in input job deck. An achieved speedup of 2.67X means that each model iteration of 15s of simulation time was executed in 5.6s of real wallclock time (on average).
• Computational requirements are 412 GFLOP per simulation time step as (measured empirically and) documented on the UCAR web site for this data model.
• Model was run as single MPI job.
• Benchmark was built and run as a pure-MPI variant. With larger process counts building and running WRF as a hybrid OpenMP/MPI variant may be more efficient.
• Input and output (netCDF format) datasets can be very large for some WRF data models and run times will generally benefit by using a scalable filesystem. Performance with very large datasets (>5GB) can benefit by enabling WRF quilting of I/O across designated processors/servers. The master thread (or rank-0) performs most of the I/O (unless quilting specifies otherwise), with all processes potentially generating some I/O.

See Also

• Weather Research & Forecasting (WRF) Model website
• 2.5 km CONUS dataset

Disclosure Statement

WRF, CONUS-2.5K, see http://www.mmm.ucar.edu/wrf/WG2/bench/, results as of 9/21/2009.

Significance of Results

A Sun Blade 6048 chassis with 12 Sun Blade X6275 server modules ran benchmarks using the NAMD molecular dynamics applications software. NAMD is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. NAMD was developed by the Theoretical and Computational Biophysics Group in the Beckman Institute for Advanced Science and Technology at the University of Illinois at Urbana-Champaign. NAMD is driven by major trends in computing and structural biology and received a 2002 Gordon Bell Award.

• The cluster of 12 Sun Blade X6275 server modules was 6.2x faster than 256 processor configuration of the IBM BlueGene/L.
• The cluster of 12 Sun Blade X6275 server modules exhibited excellent scalability for NAMD molecular dynamics simulation, up to 10.4x speedup for 12 blades relative to 1 blade.
• For largest molecule considered, the cluster of 12 Sun Blade X6275 server modules achieved a throughput of 0.094 seconds per simulation step.

Molecular dynamics simulation is important to biological and materials science research. Molecular dynamics is used to determine the low energy conformations or shapes of a molecule. These conformations are presumed to be the biologically active conformations.

Performance Landscape

The NAMD Performance Benchmarks web page plots the performance of NAMD when the ApoA1 benchmark is executed on a variety of clusters. The performance is expressed in terms of the time in seconds required to execute one step of the molecular dynamics simulation, multiplied by the number of "processors" on which NAMD executes in parallel. The following table compares the performance of NAMD version 2.6 when executed on the Sun Blade X6275 cluster to the performance of NAMD as reported for several of the clusters on the web page. In this table, the performance is expressed in terms of the time in seconds required to execute one step of the molecular dynamics simulation, however, not multiplied by the number of "processors". A smaller number implies better performance.

Cluster Name and Interconnect	Throughput for 128 Cores (seconds per step)	Throughput for 192 Cores (seconds per step)	Throughput for 256 Cores (seconds per step)
Sun Blade X6275 InfiniBand	0.013	0.010
Cambridge Xeon/3.0 InfiniPath	0.016		0.0088
NCSA Xeon/2.33 InfiniBand	0.019		0.010
AMD Opteron/2.2 InfiniPath	0.025		0.015
IBM HPCx PWR4/1.7 Federation	0.039		0.021
SDSC IBM BlueGene/L MPI	0.108		0.062

The following tables report results for NAMD molecular dynamics using a cluster of Sun Blade X6275 server modules. The performance of the cluster is expressed in terms of the time in seconds that is required to execute one step of the molecular dynamics simulation. A smaller number implies better performance.

Blades	Cores	STMV molecule (1)			f1 ATPase molecule (2)			ApoA1 molecule (3)
		Thruput (secs/ step)	spdup	effi'cy	Thruput (secs/ step)	spdup	effi'cy	Thruput (secs/ step)	spdup	effi'cy
12	192	0.0941	10.6	88%	0.0270	9.1	76%	0.0102	8.1	68%
8	128	0.1322	7.5	94%	0.0317	7.7	97%	0.0131	6.3	79%
4	64	0.2656	3.7	94%	0.0610	4.0	101%	0.0204	4.1	102%
1	16	0.9952	1.0	100%	0.2454	1.0	100%	0.0829	1.0	100%

spdup - speedup versus 1 blade result
effi'cy - speedup efficiency versus 1 blade result

(1) Synthetic Tobacco Mosaic Virus (STMV) molecule, 1,066,628 atoms, 12 Angstrom cutoff, Langevin dynamics, 500 time steps
(2) f1 ATPase molecule, 327,506 atoms, 11 Angstrom cutoff, particle mesh Ewald dynamics, 500 time steps
(3) ApoA1 molecule, 92,224 atoms, 12 Angstrom cutoff, particle mesh Ewald dynamics, 500 time steps

Results and Configuration Summary

Hardware Configuration

• Sun Blade[tm] 6048 Modular System with one shelf configured with
  • 12 x Sun Blade X6275, each with
    • 2 x (2 x 2.93 GHz Intel QC Xeon X5570 processors)
    • 2 x (24 GB memory)
    • Hyper-Threading (HT) off, Turbo Mode on

Software Configuration

• SUSE Linux Enterprise Server 10 SP2 kernel version 2.6.16.60-0.31_lustre.1.8.0.1-smp
• Scali MPI 5.6.6
• gcc 4.1.2 (1/15/2007), gfortran 4.1.2 (1/15/2007)

Key Points and Best Practices

• Models with large numbers of atoms scale better than models with small numbers of atoms.

About the Sun Blade X6275

The Intel QC X5570 processors include a turbo boost feature coupled with a speed-step option in the CPU section of the Advanced BIOS settings. Under specific circumstances, this can provide cpu overclocking which increases the processor frequency from 2.93GHz to 3.2GHz. This feature was was enabled when generating the results reported here.

Benchmark Description

Molecular dynamics simulation is widely used in biological and materials science research. NAMD is a public-domain molecular dynamics software application for which a variety of molecular input directories are available. Three of these directories define:

• the Synthetic Tobacco Mosaic Virus (STMV) that comprises 1,066,628 atoms
• the f1 ATPase enzyme that comprises 327,506 atoms
• the ApoA1 enzyme that comprises 92,224 atoms

Each input directory also specifies the type of molecular dynamics simulation to be performed, for example, Langevin dynamics with a 12 Angstrom cutoff for 500 time steps, or particle mesh Ewald dynamics with an 11 Angstrom cutoff for 500 time steps.

See Also

• NAMD - Scalable Molecular Dynamics
• NAMD Performance Benchmarks

Disclosure Statement

NAMD, see http://www.ks.uiuc.edu/Research/namd/performance.html for more information, results as of 6/26/2009.

Significance of Results

A Sun Constellation system, composed of 48 Sun Blade X6440 server modules in a Sun Blade 6048 chassis, running OpenSolaris 2008.11 and using the Sun Studio 12 Update 1 compiler delivered World Record SPEC CPU2006 rate results.

On the SPECint_rate_base2006 benchmark, Sun delivered 4.7 times more performance than the IBM power 595 (5GHz POWER6); this IBM system requires a slightly larger cabinet than the Sun Blade 6048 chassis (details below). 

On the SPECfp_rate_base2006 benchmark Sun delivered 3.9 times more performance than the largest IBM power 595 (5GHz POWER6); this IBM system requires a slightly larger cabinet than the Sun Blade 6048 chassis (details below).

• The Sun Constellation System equipped with AMD Opteron QC 8384 2.7 GHz processors, running OpenSolaris 2008.11 and using the Sun Studio 12 update 1 compiler, delivered the World Record SPECint_rate_base2006 score of 8840.
• This SPECint_rate_base2006 score beat the previous record holding score by over three times.
• The Sun Constellation System equipped with AMD Opteron QC 8384 2.7 GHz processors, running OpenSolaris 2008.11 and using the Sun Studio 12 update 1 compiler, delivered the fastest x86 SPECfp_rate_base2006 score of 6500.
• This SPECfp_rate_base2006 score beat the previous x86 record holding score by nine times.

Performance Landscape

SPEC CPU2006 Performance Charts - bigger is better, selected results, please see www.spec.org for complete results.

SPECint_rate2006

System	Processors				Performance Results		Notes (1)
	Type	GHz	Chips	Cores	Peak	Base
Sun Blade 6048	Opteron 8384	2.7	192	768		8840	New Record
SGI Altix 4700 Density System	Itanium 9150M	1.66	128	256	3354	2893	Previous Best
SGI Altix 4700 Bandwidth System	Itanium2 9040	1.6	128	256	2971	2715
Fujitsu/Sun SPARC Enterprise M9000	SPARC64 VII	2.52	64	256	2290	2090
IBM Power 595	POWER6	5.0	32	64	2160	1870	Best POWER6

(1) Results as of 23 June 2009 from www.spec.org.

SPECfp_rate2006

System	Processors				Performance Results		Notes (2)
	Type	GHz	Chips	Cores	Peak	Base
SGI Altix 4700 Density System	Itanium 9140M	1.66	512	1024		10580
Sun Blade 6048	Opteron 8384	2.7	192	768		6500	New x86 Record
SGI Altix 4700 Bandwidth System	Itanium2 9040	1.6	128	256	3507	3419
IBM Power 595	POWER 6	5.0	64	32	2184	1681	Best POWER6
Fujitsu/Sun SPARC Enterprise M9000	SPARC64 VII	2.52	64	256	2005	1861
SGI Altix 4700 Bandwidth System	Itanium 9150M	1.66	128	256	1947	1832
SGI Altix ICE 8200EX	Intel X5570	2.93	8	32	742	723

(2) Results as of 23 June 2009 from www.spec.org.

(2) Results as of 23 June 2009 from www.spec.org.

Results and Configuration Summary

Hardware Configuration:

1 x Sun Blade 6048

48 x Sun Blade X6440, each with

4 x 2.7 GHz QC AMD Opteron 8384 processors
32 GB, (8 x 4GB)

Software Configuration:

O/S: OpenSolaris 2008.11
Compiler: Sun Studio 12 Update 1
Other SW: MicroQuill SmartHeap Library 9.01 x64
Benchmark: SPEC CPU2006 V1.1

Key Points and Best Practices

The Sun Blade 6048 chassis is able to contain a variety of server modules. In this case, the Sun Blade X6440 was used to provide this capacity solution. This single rack delivered results which have not been seen in this form factor.

To run this many jobs, the benchmark requires a reasonably good file server where the benchmark is run. The Sun Fire X4540 server was used to provide the disk space required being accessed by NFS by the blades.

Sun has shown 4.7x greater SPECint_rate_base2006 and 3.9x greater SPECfp_rate_base2006 in a slightly smaller cabinet. IBM specifications are at: http://www-03.ibm.com/systems/power/hardware/595/specs.html. One frame (slimline doors): 79.3"H x 30.5"W x 58.5"D weight: 3,376 lb. One frame (acoustic doors): 79.3"H x 30.5"W x 71.1"D weight: 3,422 lb. The Sun Blade 6048 specifications are at: http://www.sun.com/servers/blades/6048chassis/specs.xml One Sun Blade 6048: 81.6"H x 23.9"W x 40.3"D weight: 2,300 lb (fully configured). 

Disclosure Statement:

SPEC, SPECint, SPECfp reg tm of Standard Performance Evaluation Corporation. Results from www.spec.org as of 6/22/2009 and this report. Sun Blade 6048 chassis with Sun Blade X6440 server modules (48 nodes with 4 chips, 16 cores, 16 threads each, OpenSolaris 2008.11, Studio 12 update 1) - 8840 SPECint_rate_base2006, 6500 SPECfp_rate_base2006; IBM p595, 1870 SPECint_rate_base2006, 1681 SPECfp_rate_base2006.

See Also

• Sun Constellation System - Overview
• SPEC Home Page

Welcome to BestPerf group blog!  This blog will contain many different performance results and the best practices learned from doing a wide variety of performance work on the broad range of Sun's products.

Over the coming days, you will see many engineers in the Strategic Applications Engineering group posting a wide variety topics and providing useful information to the users of Sun's technologies. Some of the areas explored will be:

world-record, performance, $/Perf, watts, watt/perf, scalability, bandwidth, RAS, virtualization, security, cluster, latency, HPC, Web, Application, Database