Hugo Rivero's Weblog

OpenSolaris shows up at SPEC

Industry standard benchmarks play an important role. And not just as product marketing tools. With a multitude of hardware and software combinations, most customers cannot allocate time and resources to do a thorough comparison of all the choices, and appreciate the contribution of standard bodies that define relevant workloads and consolidate performance results. SPEC (Standard Performance Evaluation Corporation) is a prime example. With its strict run rules and review process, it forces vendors to always put their best foot forward. Which makes the publication of a couple of OpenSolaris SPEC results a significant milestone.

The first one is SPECfp2006, often considered the leading indicator of a system's performance for HPC applications. It is a cpu-intensive benchmark, where the OS and compiler play a key role in extracting every ounce of performance from the underlying hardware. To get a good feel for how OpenSolaris and Sun Studio compare to other OS/Compiler stacks, I looked at all results using Intel's quad-core Xeon X5482 at 3.2GHz, all seven of them run on dual-socket systems. Here's the summary (click on the score to see the full report):

Operating System	Hardware	Compiler	Result (SPECfp2006)
OpenSolaris 2008.05	Sun Fire X2250	Sun Studio Express 11/08	27.8
SUSE Linux Enterprise Server 10	HP ProLiant DL160 G5p	Intel C++ and Fortran Compiler 10.1	27.5
SUSE Linux Enterprise Server 10	Intel Supermicro X7DWN	Intel C++ and Fortran Compiler 11.0	27.5
SUSE Linux Enterprise Server 10	IBM System x3450	Intel C++ and Fortran Compiler 11.0	26.9
SUSE Linux Enterprise Server 10	ASUSTeK DSEB-DG Motherboard	Intel C++ and Fortran Compiler 10.1	25.1
Windows XP Professional	Dell Precision T7400	Intel C++ Compiler 10.1, Intel Visual Fortran 10.0	22.2
Windows Vista Ultimate	Dell Precision T7400	Intel C++ Compiler 10.1, Intel Visual Fortran 10.0	22.1

The combination of OpenSolaris and Sun Studio gives other more entrenched solutions a run for their money. They are free to use, and Sun Studio comes with a nifty Performance Analyzer tool. Why not give them a try? If you're on OpenSolaris, run pkg install sunstudioexpress to get started.

The second benchmark is SPECjAppServer2004, which tests the performance of a multi-tier Web stack, including application server and database. It is quite active, with over 70 results from all major hardware and software players in the Enterprise Java space. What makes this result significant? That the OS (OpenSolaris 2008.05), the database (MySQL 5.0) and the Application Server (GlassFish Enterprise Server v2) are all open source. In fact, Sun is the only vendor that has published a result that doesn't rely on proprietary software.

Beyond getting kudos from FOSS advocates, there is a strong reason why this SPECjAppServer result is very relevant: customers are asking not for performance at any price, but performance at the right price. And the price/performance numbers speak for themselves, showing a 10x advantage over proprietary solutions. Take a look at Tom Daly's blog or BM Seer's blog for in-depth analysis. This more cost-efficient, open-source-based stack does not mean customers have to endure higher risk. All of the components, while free to download and use, can be covered by a 24x7 support contract, in compliance with the benchmark rules.

Disclosure Statement:
SPEC, SPECfp and SPECjAppServer are trademarks of the Standard Performance Evaluation Corporation. Results from www.spec.org as of 10/15/08. SPECfp2006 comparison based on results with a Processor type "Intel Xeon X5482". For the latest SPECfp2006 results visit http://spec.org/cpu2006/. For the latest SPECjAppServer2004 results visit http://www.spec.org/osg/jAppServer2004.

Posted at 11:49AM Nov 07, 2008 by hugo in Sun  |