Vijay Tatkar's Blog

All | Benchmarks | Business | Cloud Computing | General | Hardware | Linux/Unix | Performance | Software | Solaris | Sun | Sun Studio
« Google: Wont you... | Main | Performance Comparis... »
20060926 Tuesday September 26, 2006

Performance Comparison: SunStudio vs. GCC on BYTE benchmark (Nbench)
I am often asked about performance differences between Sun Studio Compilers and GCC. And whereas with performance, a single answer never works across the board, I am attempting to put out as much comparative information as I can, to show some of the differences (and hopefully advantages over GCC) as I can.
I have announced previous Sun Studio based SPEC World Record numbers in postings here (like this World Record SPECfp number and this mention of SPEC CPU2006 World Records ), and about STREAM (as in here), but these were not comparative numbers (vs GCC), so this is an attempt to fill that gap.
The first attempt is to take BYTE magazine's BYTEmark benchmark programs that are freely available at this location.
The benchmarks are designed to expose the capabilities of a system's CPU, FPU and memory system and were derived directly, without algorithmic change, from the BYTE web site.
The tests used here were ported to Linux and are actually run on a Solaris 10 system . The HW used here was a SunFire X4100 box with a dual-core 2.4GHz Opteron chip in a standard configuration.
In the following tables, the numbers are all Ratios index against a baseline of AMD K6/233 with 512KB L2-cache, gcc2.7.2.3 and libc-5.4.38 system.
Being ratios, Higher number is Better and so also in the Ratio's column, a ratio > 1 means SunStudio is better than GCC

Test
GCC4.1
SunStudio11
Ratio(SS11/GCC)
Numeric Sort
12.68
10.63
0.84
String Sort
16.07
20.93
1.30
Bitfield
15.15
16.15
1.06
FP Emulator
14.46
31.77
2.19
Fourier
11.69
25.55
2.19
Assignment
37.59
25.11
0.67
Idea
18.52
32.87
1.77
Huffman
15.10
17.02
1.13
Neural Net
24.87
33.98
1.37
Lu Decomposition
45.55
50.20
1.10
Memory Index
20.78
20.397 0.98
Integer Index
15.09
20.844
1.39
FP Index
23.772
35.190
1.48

Numeric Sort, FP Emulator, Idea and Huffman are part of Integer Index
String Sort, Bitfield and Assignment make up the Memory Index
The other tests are part of FP Index

Flags used for each were:
Sun Studio11: -fast -xarch=amd64
GCC-O3 -s -Wall -fomit-frame-pointer -funroll-loops


Posted by tatkar ( Sep 26 2006, 02:36:44 PM PDT ) Permalink Comments [6]

Like this post?  del.icio.us  bookmark it   |   submit to dig digg.com digg it   |   slashdot slashdot it   |   technorati Technorati it

Trackback URL: http://blogs.sun.com/tatkar/entry/sunstudio_vs_gcc_on_byte
Comments:

Um, are these timings? Smaller better? Or what...

Posted by Toby on September 26, 2006 at 03:17 PM PDT #

Vijay, do you have these numbers for Solaris as well? Btw, it might be helpful to readers to note that these are "scores" and larger numbers mean better performance.

Posted by Kristofer Spinka on September 26, 2006 at 03:24 PM PDT #

Ugh, sorry about that. It got dropped in the cut-and-paste from the editor to the Roller on which this blog was entered.
I've corrected it above. The numbers are ratios and so larger is better; similarly in the last column a ratio >1 implies Sun Studio is doing a better job than GCC

Posted by Vijay Tatkar on September 26, 2006 at 03:28 PM PDT #

I highlighted the fact that they are actually run on a Solaris 10 box to make that clearer.

Posted by 192.18.43.249 on September 26, 2006 at 03:38 PM PDT #

Nice data, and interesting results! Out of curiosity, I wonder how these results compare to previous Sun Studio compilers. I mean, looking at these results it shows that Studio compares quite nicely against GCC, but I don't know if that's always been the case. For those developers who had adhered to GCC because they had a less-than-optimal experience with Studio, I wonder how much better performance they'd see with the current Studio? Great info.

Posted by George Drapeau on September 26, 2006 at 10:35 PM PDT #

Good point!
One of the things that makes this hard is keeping track of which compiler to compare against. Earlier SunStudio compilers (before SunStudio 10) would not be able to exploit AMD64 ABI. Thus they probably need to be compared against earlier (current at that time) versions of GCC running 32bits.
This is a bit harder; I'll see if I can post 32bit versions of these results for both!

Posted by 192.18.43.249 on September 27, 2006 at 08:44 AM PDT #

Post a Comment:

Name:
E-Mail:
URL:

Your Comment:

HTML Syntax: NOT allowed

Who Am I?

Calendar

RSS Feeds

Search

Links

Presentations

Latest TechDays Presos

Navigation

Referers