Tuesday August 21, 2007

New SPECjAppServer2004 813.73 JOPS result using Glassfish 2 and Postgres

Sun SPECjAppServer2004 813.73 JOPS@Standard result improves performance and lowers price (again) using Glassfish V2 and Postgres 8.2 DB on T2000.

Sun has submitted a new benchmark result of 813.73 SPECjAppServer2004 JOPS@Standard which improves on the performance of the recent 778 JOPS number and at a reduced price for the tested system. The improvements are due to better database performance and a huge boost to the application server performance which enabled the number of application server machines in the tested configuration to be reduced from 3 to 2.

As I suggested in my comments regarding the recent Sun 778 JOPS result, at Sun we are in the early stages of working with open source communities such as the PostgreSQL community so really we have only just started to demonstrate  how far these open source products will scale and how much money they can save your organisation.

Highlights : -

• Approximately 4% better performance compared to the earlier 778 JOPS number

• Decreased the price of the system.
  By using Glassfish V2, we were able to reduce the cost of the system by about $US8K as we only needed 2 application server machines instead of 3.
  The total purchase cost of the tested configuration is now only $US 57, 425

• Improved price/performance from $US 84.98/JOPS to $US 70.57/JOPS (see below for full pricing of this result)

• All open source software result so the purchase cost of the software (operating system, application server and database) is zero!

• This system is supporting more than 6000 simulated concurrent users
  note that SPECjApp2004 run rules ensure that during the test there is no (significant) single point of failure so these virtual users are running in a simulated production quality environment. E.g. log disks and database are mirrored and striped, the StorageTek 2540 array has battery backed cache in case of power fail etc.
  
  

Cost comparison against HP result (costing details for HP result)

Pricing calculations for Sun 813.73 JOPS result

• I have used the bill of materials (bom) from the benchmark result to price this result.

• All hardware prices come directly from the Sun US website as at 20^th July 2007
  go to http://www.sun.com/servers , choose a server and use the “get it” tab to be taken to the Sun store where you will find the pricing information

• I have omitted minor items from the pricing like a keyboard etc

• The pricing doesn't include support but if it did the result would look even better compared to results using commercial (non open source) software!

SPEC required disclosure : -

SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation.
Results from www.spec.org as of 22^nd Aug 2007

All comparisons are based on the SPEC SPECjAppServer2004JOPS@Standard metric from www.spec.org or on pricing made using the bill of materials included in each SPECjAppServer2004 result
Sun 813.73 SPECjAppServer2004 JOPS@Standard (Sun Fire X4200, 4 chips / 8 cores, T2000 1 chip / 8 cores)
HP 874.17 SPECjAppServer2004 JOPS@standard (rx2660 , 4 cores, 2 chips)

Wednesday July 25, 2007

Regperf V1 - Running the Grails performance tests (howto)

Regperf (v1) – Running Grails Performance Tests
How to run the Regperf v1 tests using Grails framework
and Glassfish Java EE 5,0 server

Introduction

Damien Cooke and I have been building some tools to enable performance testing and comparisons of a simple application built in 3 frameworks Grails, Rails and Java EE 5.0 (JSF), this blog entry is one of a number we are posting between us to describe how to run these simple performance tests (which we presented at JavaOne). This blog entry describes how to run the simple Regperf application implemented in Grails , Damien will be posting how to run the Rails version of Regperf in the very near future and when I have finished the JavaServer Faces write up I will post this too.

The instructions and software has been tested on Solaris and Mac OSX but should run anywhere where Faban and Grails , Java and Postgres run

The main steps

1. Install the database

2. Install / create the Grails “Regperf” application

3. Setting up Faban and the Faban Regperf driver

4. Running the performance tests using the Grails runtime (jetty)

5. Deploying and testing the Grails application using Glassfish Java EE 5 application server

1. Install the database.

• we used postgres 8.4.2 (of course MySQL, JavaDB , Oracle etc are all okay too). For simplicity these instructions assume that Postgres DB is installed and running on the same host as your grails application.

• download and unpack the postgres Regperf dump
  

• create the db createdb regperfdb

• import the database schema pg_restore -d regperfdb grails-regperfdb.tar
  
  
  

2. Install / create the Grails application
   (assumes you have Grails installed at $GRAILS_HOME and $GRAILS_HOME/bin is in your PATH)

• cd $GRAILS_HOME

• grails create-app regperf

• edit $GRAILS_HOME/regperf/grails-app/conf/*Data* to point to your database
  E.g
  DevelopmentDataSource.groovy
  class DevelopmentDataSource {
      boolean pooling = true
      String dbCreate = "create-drop" // one of 'create', 'create-drop','update'
      String url = "jdbc:postgresql://localhost:5432/regperfdb"
      String driverClassName = "org.postgresql.Driver"
      String username = "regperf"
      String password = "regperf"
  }

• copy the JDBC driver into $GRAILS_HOME/regperf/lib

• copy the following Subscriber.groovy class into the $GRAILS_HOME/regperf/grails-app/domain directory
  class Subscriber {
      String name
      String address
      String email
      Date renewDate = new Date()
      String location
  }

• cd $GRAILS_HOME/regperf

• grails generate-all Subscriber

• grails run-app

• point your browser to http://localhost:8080/regperf and click on the SubscriberController link

3. Setup Faban and the Faban Regperf driver

• Download and install Faban (look at the bottom of the page for download link and installation instructions)
  Note Faban has a driver and also a harness for automatically queuing and running multiple benchmarks but in these instructions we are going to run just the driver component and again assuming localhost

• set the FABAN_HOME environment variable and add $FABAN_HOME/bin to your PATH

• mkdir $FABAN_HOME/output (this is where the tests results will be stored)

• download the Faban Regperf driver for Grails and unjar it into $FABAN_HOME/samples

• edit the $FABAN_HOME/samples/regperf-grails/config/run.xml
  change the value for outputDir to match your $FABAN_HOME/output directory
  modify the number of simulated users by changing the value for <scale>1<scale> you will get 10 users for each scale factor e.g. scale 1 will give 10 simulated users
  Note the number of loadedsubscribers should match the number of rows you load into the database (this is set initially to 100)
  refer to the Faban documentation for other values to change in the run.xml

4. Running the performance using the Grails runtime (jetty)

• cd $FABAN_HOME/samples/grails-regperf/sbin
  assuming that the database and the grails application is still running execute ./loader.sh 10

• start the rmiregistry (this runs in the background)
  ./registry.sh &

• start and execute the benchmark
  ./master.sh
  Starts up all of the simulated users and writes results of the run into $FABAN_HOME/output/xx directory.
  The results can be conveniently viewed with a web browser.

5. Deploying and testing the Grails application using Glassfish Java EE 5 application server

• Assuming Glassfish is installed and running on localhost on port 8080 (the default install)

• Grails allows the application to be packaged into a .war file and deployed to the application server, to create regperf.war
  cd $GRAILS_HOME/regperf
  grails war

• Login to the browser admin console. By default this is http://localhost:4848

• Choose “Deploy Web Application (.war)” from the Deployment column

• Browse to the $GRAILS_HOME/regperf directory and select regperf.war

• Click ok to deploy, and test for successful deployment by browsing to http://localhost:8080/regperf )

• Use the loader.sh and master.sh scripts as per step 4. above to load the database and run the tests (start the rmiregistry if it is not already running).  If you have Glassfish running on a different port or host adjust the run.xml to reflect this.

Notes:

• There are other values in the run.xml you might want to configure such as the rampUp, SteadyState and rampDown times.

• One some platforms we tested deletes seem to run faster than inserts , so over the run the number of rows in the database may decrease. For this reason it is probably a good idea to load a few more rows in the database than is indicated by the loadedsubscribers entry in the run.xml

• Running the performance tests with the driver, application and database all on one system is great for getting going but is of limited use for doing performance analysis of the application as it is hard to identify or separate the resource requirements of the 3 components when running on one box.

Disclaimer/Important note

Regperf and the simple performance testing tools offered here and on Damien's blog are designed to allow performance testing but do not constitute a “benchmark” per se. A benchmark has process, run rules, criteria for comparison and peer review or audit that can enable relevant performance comparisons to be made with a reasonable (or high) degree of confidence, so please feel free to use the tools to do performance testing of Grails, Rails but understand that this is “not a benchmark” it is just a few interesting and hopefully useful testing and tuning tools!

Tuesday July 17, 2007

Comments on JavaOne Session on Grails Rails and J EE

JavaOne session on Grails Rails and JEE

I noticed that the JavaOne session that Damien Cooke and I put together titled "Comparing the Developer experience of JEE 5, Ruby on Rails and Grails" has been commented on by a few folks firstly we noticed the "government computer news" and secondly and only recently we noticed an article from June 07  by the Grails community

I want to comment on these articles but first let me say that the intent of the session was to address Java developers like Damien and I that are not experts in both Grails and Rails but who want to know a bit more about both Rails and Grails and how they fitted with or competed against Java and J EE. One area we though would be valuable to explore was the performance characteristics of the frameworks, initially not in an exhaustive or robust way but to start looking at performance comparisons of these frameworks. Of course we realised saying anything about relative performance of the frameworks was going to get us offside with someone so we didn't try and tune anything or come to any stunning conclusions (yet) rather we wanted to just present some out of the box and very initial figures.

Comments on Government computer news

First up thanks to the author for the write-up it was much appreciated.
Overall I believe what is written is pretty accurate synopsis of what I said during the session importantly the author Joab Jackson highlights that I went to some lengths to point out that the tests were not benchmarks and that little (no) tuning was done on any of the platforms. What was also pointed out in this article is that somehow the wrong slides got loaded for our session and so I ended up presenting from something that still had typos and more unfortunately had the wrong performance graphs. Happily the JavaOne team has helped post an updated version of the presentation and so we now have the correct performance slides although I notice some typos are still there (mainly from the trademarking exercise)

Comments on the Grails blog entry

Well as per the invitation in the article Damien and I intend to contact the Grails team shortly. Certainly I think there are some things we can set straight and agree upon whilst other things we might have to agree to disagree. In the meantime hopefully the following responses to the points raised in the blog entry are at least a little helpful or clarifying.

1. J2EE comes out smelling of roses unfairly
   Well probably not a surprise given our backgrounds and the fact that we have at least a little experience in J2EE and J EE ( mainly with performance testing) however in the session you will see that I actually really like Grails and I am quite excited about it. I am particularly excited about the prospect of Grails applications deploying to J EE 5 application servers and using JPA. I am certainly excited about the easy way a Grails application can be deployed into an application server such as Glassfish today but I am looking forward to when more of the underlying plumbing of the appserver can/will be used rather than bundled in the .war
   Do I believe that in general J EE 5.0 applications will scale much better than Grails application today ..you bet (see SPECjAppServer related enties on this blog). However in future and perhaps as Grails uses more of the app server infrastructure I am excited about what might be achieved in terms of Grails performance (similarly we are also very interested in the performance achieved when deploying Rails apps to glassfish J EE application servers)

2. After all, we have run benchmarks that seem to be fair against Rails, and Grails
   I apologise if I have this wrong but the only grails / rails performance comparison tests I could find before JavaOne were based on a single Mac (laptop?) and had no J EE implementation in the mix. Further it appeared the application server (or mongrel server for Rails) was on the same box as the driver /load generator and the database server.
   Damien and I figured it would be helpful if we provided a JEE version of an application as well as a rails and grails version and if we created the same driver for each application. We developed this with Faban and as stated ran just some initial tests , but we did make sure that we had the hardware available to scale up a little so we could look not just at latency but also at scaling (a J EE 5.0 strength). By having each component (app server driver, db) on a separate box we were able to also examine and do some initial reporting on resource utlisation.
   So did we publish a good comparison? Heck no, again we have only done very initial work , we ran grails on jetty and rails on webbrick and we certainly did not optimise the version of glassfish v1 we were using and by the way glassfish v2 yeilds some very significant performance benefits over the v1 which we used.
   Bottom line this is only stage #1 of grails vrs rails vrs JEE performance and scalability testing.

3. There are so many typos in the document
   Well actually there are a lot of typos and the performance graphs were wrong, this was because somehow the wrong slides were loaded for the presentation in fact as I already mentioned some of the typos in this older version came from the trademarking process e.g. the reference to JavaScript^TM programming language.
   The reason I mention this is that it felt a little disingenuous that you would trash the work we did based on the typos which I feel were not necessarily the fault of Damien and I.

4. 4 simple bullet points for creating a CRUD J2EE application
   Actually seeing might be believing on this one, so perhaps you have to watch the video to see the demo I did, which was to take the grails application data (same database) and then without doing any coding on the fly I produced an equivalent application in Java Server Faces/Java Persistence Architecture using NetBeans 5.5. It was dead easy and in many ways similar to the Grails tasks i.e. generate objects from the DB, then generate the MVC code to create the application. Here though it is not the framework doing all the generation but the tools and so where I can generate a simple grails application with a text editor with Java I really need tools. I believe that this is a difference in philosophy between J EE and Grails and yes I agree there is a lot more to learn up front for a J EE CRUD application but in many ways the tools like Netbeans that are available for J EE mean that some (many) Java tasks have become trivial and the entry point to being productive in J EE is far lower than it has been in the past. Importantly I would also note that Java Persistence Architecture is also simplifying life for many J EE applications developers! Anyhow I believe I demonstrated how easy it is to build a JEE / JSF CRUD application so I stand by this point.

5. They say that one of the "problems" with Grails is that its made up of lots of other bits of tech
   I will stick by this and I really think it is one area that maybe the framework can be improved and simplified especially as/if things like JPA are embraced. The reason that all the different technologies are a problem is that Grails is attempting to be simple and dynamic but the fact that as a developer I have know about some many different technologies (best of breed or not) and learn more and more about them as my application complexity grows means that much of this simplicity is quickly lost. Also what about when you want to host a large application with a large number of users, the tuning of each of these at times unrelated technologies is going to be time consuming and a challenge and once again the deep understanding of each (or at least most) of the technology stack is going to be required to achieve a cost effective solution. I started to believe this might be an issue when I did "grails war" and saw the size of the .war file generated for a simple 1 table application.
   However as I say I think if/as things like JPA are embraced this will really help because there is lots of time and effort going into optimising JEE technologies and you can get a flavour for this at the SPECjAppServer2004 benchmark site.

6. Then there are things like "Not clear how to regenerate if you have made changes to the controller/views"
   Well I was having trouble with this and I was talking about the experience I had again figuring I might not be the only one. In the session I believe I mentioned that this was more likely to be my lack of knowledge of Grails rather than the framework.

7. Pah, oh well. Damien and Tom
   Well I have already confessed to not being a Grails guru, but I think Damien and I are fairly experienced developers and have good knowledge of performance measurement and I think  we are typical of Java developers who are starting to look at Grails and Rails so I would hope that the Grails developers would not write off much or all  of what we have presented especially when in fact we were actually trying to be helpful.

I hope the above clears up some things and provides at least basis for the points in the slides (without having to watch the session).  Damien and I are doing this investigation as we are interested in Grails and Rails but it isn't really our day job so it is taking a little while to get our tools posted. However we are really close now and I would welcome your feedback once we make the tools and a bit more testing information available.

Finally  I refer the Grails community to slides #40 and #41 which are titled "Grails cool stuff"  :-)

cheers
Tom

Tuesday July 10, 2007

Sun pushing price/performance curve with open source this time Postgres on T2000

Sun Pushes Price/Performance boundaries with SPECjAppServer2004 result of 778.14 JOPS featuring Postgres on T2000

Sun is once again pushing the price/performance envelope for open source software. On Monday 9^th July 2007 Sun published a SPECjAppServer2004 result of 778.14 SPECjAppServer2004JOPS@Standard. So what you may say , whats in this for me or why do I care ?

Well you might care because : -

1. You are probably using or planning to use open source software and this is an all open source result, the Solaris operating system is open source, the Sun application server (Glassfish V1) is open source, the database (Postgres) is open source , even the SPARC processor in the database server is open.

2. The price/performance of this result using open source is compelling. The entire cost of the Sun system using Postgres / T2000 is approx $US 65K (hardware and software) yet the performance is comparable to other SPECjAppServer2004 results using only commercial “closed source” products or mostly “closed source” products. For example the recent HP result of 874.17 JOPS* has a cost of approx $US 187K
   See HP pricing details here and see below for pricing calculations on the Sun result)

3. This demonstrates the benefits and performance of the Postgres database especially when running on the Sun cool threads servers (SunFire T2000).  Note : Jignesh Shah has done work on demonstrating Postgres scaling on T2000 on 2, 4, 6 and 8 cores.

4. SPECjAppServer2004 benchmark is a web application and quite likely models (to some degree) web applications that you are developing or deploying in your organisation. So yes SPECjAppServer2004 is a benchmark and benchmarks are limited but this one is designed to be relevant to the web applications in your business.

5. Mostly you might care because you are probably using or interested in using open source software and Sun is a major open source vendor and is working with important open source communities to drive the performance and price/performance of an enterprise supported open source stack ..this should help you to save money!

    

Interesting notes :

• We are in the early stages of a continuing effort at Sun to use industry standard benchmarks to drive the performance and price performance of the web application open source stack.... so expect even better performance and price/performance results in the near future.

• The focus of this particular benchmark was on the T2000/Postgres performance so we didn't work too much tuning on the Glassfish/X4200 application servers and in fact they are not being fully utilised. We are planing on taking advantage of the significant performance improvements with Glassfish V2 in the near future to further highlight the Glassfish and X4200 server price/performance advantages along with the Postgres/T2000 advantages !

• Notice that the Sun result with T2000/Postgres uses only 12 disk spindles in the StorageTek 2540 where most other comparable SPECjAppServer2004 results are using significantly more spindles/storage, the advantages here are  price reduction and an improvement in reliability!

• If you work with Postgres especially with web applications , check out the benchmark result “full disclosure report” and download the “full disclosure archive” for all of the postgres and glassfish tuning.

Community and Team Effort

Lots of thanks are due to members of the Postgres community for this result and in particular Dave Cramer who heads up the PostgreSQL JDBC team and to Gavin Sherry a Postgres contributor from Sydney Australia ! Jignesh Shah (Sun) provided much of the Solaris tuning so checkout Jignesh's blog at http://blogs.sun.com/jkshah/entry/specjappserver2004_and_postgresql also Sun's Josh Berkus (Postgres guru) mentored the entire Postgres side of things (http://blogs.ittoolbox.com/database/soup/archives/postgresql-publishes-first-real-benchmark-17470) .

Pricing calculations for Sun 778.14 JOPS result

• I have used the bill of materials (bom) from the benchmark result to price this result.

• All hardware prices come directly from the Sun US website as at 6^th July 2007
  go to http://www.sun.com/servers , choose a server and use the “get it” tab to be taken to the Sun store where you will find the pricing information

• I have omitted minor items from the pricing like a keyboard etc

• The pricing doesn't include support but if it did the result would look even better !

Required disclosure : -

All comparisons are based on the SPEC SPECjAppServer2004JOPS@Standard metric from www.spec.org or
on pricing made using the bill of materials included in each SPECjAppServer2004 result
Sun result details at http://www.spec.org/jAppServer2004/results/res2007q3/jAppServer2004-20070606-00065.html
HP result details at http://www.spec.org/jAppServer2004/results/res2007q2/jAppServer2004-20070410-00060.html
Sun Fire X4200 (6 chips, 12 cores) 778.14 SPECjAppServer2004 JOPS@Standard.
HP rx2660 (2 chips, 4 cores) 874.61 SPECjAppServer2004 JOPS@Standard.

SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation.
Results from www.spec.org as of 7/10/07

Monday June 18, 2007

Full pricing of the HP SPECjAppServer2004 874.17 JOPS@Standard Submission

As you will see from my other blog entries on the subject of SPECjAppServer2004 there is a part of the submission which is called the bill of materials (BOM). This is included in all of the published results so that users/customers or others can see exactly what components are required to enable the benchmark score to be achieved. The BOM includes all part numbers or names for the tested components and hence allows end users to do pricing of the submission. Depending on the vendor this pricing will be either a straight forward exercise or will take a little work. At Sun it appears that we are happy to publish all of our prices at http://www.sun.com but this isn't always the case with all vendors, however as you can see below using the BOM and the part numbers in the BOM you can get a pretty good idea of the full price of the submission even if the vendor isn't actively publishing all prices on their web site or in their performance claims. So the BOM is a great tool to keep the claims that vendors make regarding their benchmark results more honest, complete, accurate and relevant to your business.

I noticed the price/performance claims HP is making in recent posts and documents about their integrity servers, certainly the pricing information at the bottom of page 2 in this document seems a little incomplete. So I have used the BOM from the HP SPECjAppServer2004 result to give a “more complete” picture of the price of the HP benchmarked system.

Methodology

I have used the HP web site as the primary source for HP hardware prices and a TPC-C result from HP to price the operating system and several other components also being used on the SPEC submission. For some components where the price is not listed by HP I have found 3^rd party suppliers and included these prices, generally 3^rd party suppliers are cheaper than the original equipment manufacturer so that using these 3^rd party prices is very likely to yield a lower overall system price.

Conclusions/Observations

1. The full price of the HP 874 JOPS result is $US 187,701.52
   The price performance therefore on this submission is 187701.52/874.17 = $US 214.72/JOPS

2. HP is not leading price performance because Sun's has (far) superior price/performance proof points such as the recent submission with Sun Application Server Glassfish and MySQL 5 on Solaris 10. The result is 720 SPECjAppServer2004JOPS@Standard* and the price performance is $US 82.24/JOPS. All the pricing details for this result can be found on this blog at http://blogs.sun.com/tomdaly/date/20070618
   
   
   

*Footnote

The HP result can be found at http://www.spec.org/jAppServer2004/results/res2007q2/jAppServer2004-20070410-00060.html
The referenced Sun result can be found at http://www.spec.org/jAppServer2004/results/res2007q2/jAppServer2004-20070411-00063.html

Pricing for Latest Sun Glassfish/MySQL/Solaris SPECjAppServer2004 benchmark

720 JOPS pricing

Some while ago a technical hitch was noted on the Sun/Glassfish/MySQL result referenced here, the hitch was a small configuration error which mean't that the result didn't follow the SPECjAppServer run rules to the letter and so the result was removed from the SPEC web site (marked non compliant). Sun is committed to using and improving open source software and so with help from MySQL we resubmitted and republished this benchmark at SPEC. The updated result uses almost the same configuration as the initial result the signifcant changes were to use a StorageTek 3510 disk array and to use an updated release of the Sun 9.0 application server (Glassfish). You can find the new an improved result of 720.56 SPECjAppServer2004 JOPS@Standard at http://www.spec.org/jAppServer2004/results/res2007q2/jAppServer2004-20070411-00063.html

Summary of price/performance

So using the bill of materials (BOM) from the submission the price of the entire hardware and software stack is $US 59,260 (see below for calculations)
The price performance of this result is 59,260/720.56 or $US 82.24/JOPS

Please notice that the price analysis includes all of the major components hardware and software listed in the BOM.

I believe that this result demonstrates the leading price/performance for this benchmark (by a long way).

A couple of important notes / disclaimers :

• The SunFire X4100 servers used in the submission are now older models, the current and (very similar) model is the SunFire X4100 part number A86-FGZ2BH4GKBA listed under “Config 3-M2” for which the price is listed at $US 5945.00 at the Sun shop

• The prices below are in $US

• All prices for Sun products below come from the Sun web site http://www.sun.com and hence the web site is the correct place to make purchasing enquiries or to check on my calculations

• The prices are listed here for illustrative purposes only and are from the web site as at June 2007

• I haven't included support in this calculation it looks at acquisition cost only

• I haven't included the price of the screen and keyboard as this is a cost common to all submissions and only a minor amount

Supplier  Description                                Product #            Qty     Unit Price      Price
--------  ----------------------------------------   ------------------   ---     ----------      -------
Sun       Sun Fire X4100 (2x285,4x2GB,2X73GB)        A64-EGB2-2H-8G-CB7    3      $8995           $26,985
Sun       Solaris 10 RTU                                                   3      $0              $0
Sun       SunSpectrum Upgrade: 3YGOLD, 24x7          W9D-A64-24-3G         3      not priced      not priced 

Sun       Sun Fire X4100 (2x285,4x2GB,2X73GB)        A64-EGB2-2H-8G-CB7    1      $8995           $8995
Sun       Solaris 10 RTU                                                   1      $0              $0
Sun       Single-Port PCI Ultra320 SCSI HBA          SGXPCI1SCSILM320-Z    1      $340            $340      
Sun       SunSpectrum Upgrade: 3YGOLD, 24x7          W9D-A64-24-3G         1      not priced      not priced

Sun       Sun StorEdge 3320, 12x73GB, 1 RAID CONT    XTA3510R01A1R876      1      $22,495         $22,495
Sun	  Single-Port PCI Ultra320 SCSI HBA	     SGXPCI1SCSILM320-Z    1      $445            $445
Sun       SunSpectum Upgrade: 3Y GOLD, 24x7          W9D-SE3510-24-3G      1      not priced      not priced
Sun       17" Entry Color Monitor                    X7147A                1
Sun       PS/2 Keyboard & Mouse                      #320-1261             1

Sun       Sun Java System Application Server                                      $0              $0
          Platform Edition 9.0 

Sun       Sun Java System Application Server         SJSAS-PE9F-1PR        9      not priced      not priced
            Platform Edition 9.0  
            Premium Support per CPU for 1 year

MySQL     MySQL Database 5.0                                                      $0              $0
MySQL     MySQL Network Gold Support                                       3      not priced      not priced 
            for 1 year
                                                                                                  ==========
                                                                                                  $59,260
                                                                                                  ==========

Required Disclosure Statement:

• SPECjAppServer2004 3x Sun Fire X4100 appservers (12 cores, 6 chips) and 1 Sun Fire X4100 database (4 cores, 2 chips) 720.56 SPECjAppServer2004 JOPS@Standard.

Wednesday June 07, 2006

Pricing SPECjAppServer2004 Submissions

Pricing details for Sun's SPECjAppServer2004 712.87 JOPS result

One of the disclosures made in each of the SPECjAppServer2004 submissions is the “Bill of Materials” or BOM. The BOM is designed to enable users of the benchmark information to see all of the components that were used to achieve the result and to enable users to price the entire configuration in a consistent way. This then allows price or price/performance comparisons to other benchmark submissions and importantly it allows users of the benchmark to check price/performance claims made by vendors. To demonstrate how to use the BOM and also to further highlight the terrific price/performance of the Sun/MySQL 712.87 SPECjAppServer2004 JOPS result , I have priced the benchmark result (below) by reproducing the BOM from the benchmark page at www.spec.org and attaching list prices from the Sun (http://www.sun.com) and MySQL (http://www.mysql.com) websites.

Robert Lee has a good summary of the price/performance benefits of the Sun/MySQL result here, but to recap Sun's result of 712.87 SPECjAppServer2004 JOPS at a total acquisition price of $US51,780 represents $US72.64 / SPECjAppServer2004 JOPS while the comparable HP result of 1664.36 SPECjAppServer2004 JOPS $US1,034,103 represents $US621.32 JOPS.

A couple of important disclaimers :

• The prices below are in $US

• The prices for Sun products below come from the Sun web site http://www.sun.com and hence the web site is the correct place to make purchasing enquiries

• The prices are listed here for illustrative purposes only and are from the web site as at June 1 2006

• I haven't included support in this calculation it looks at acquisition cost only

• I haven't bothered to include the price of the screen and keyboard as this is a cost common to all submissions.

Supplier  Description                                Product #            Qty     Unit Price      Price
--------  ----------------------------------------   ------------------   ---     ----------      -------
Sun       Sun Fire X4100 (2x285,4x2GB,2X73GB)        A64-EGB2-2H-8G-CB7    3      $8995           $26,985
Sun       Solaris 10 RTU                                                   3      $0              $0
Sun       SunSpectrum Upgrade: 3YGOLD, 24x7          W9D-A64-24-3G         3      not priced      not priced 

Sun       Sun Fire X4100 (2x285,4x2GB,2X73GB)        A64-EGB2-2H-8G-CB7    1      $8995           $8995
Sun       Solaris 10 RTU                                                   1      $0              $0
Sun       Single-Port PCI Ultra320 SCSI HBA          SGXPCI1SCSILM320-Z    1      $340            $340      
Sun       SunSpectrum Upgrade: 3YGOLD, 24x7          W9D-A64-24-3G         1      not priced      not priced

Sun       Sun StorEdge 3320, 12x73GB, 1 RAID CONT    XTA3320R01A1T876      1      $15,460         $15,460
Sun       SunSpectum Upgrade: 3Y GOLD, 24x7          W9D-SE3310-24-3G      1      not priced      not priced
Sun       17" Entry Color Monitor                    X7147A                1
Sun       PS/2 Keyboard & Mouse                      #320-1261             1

Sun       Sun Java System Application Server                                      $0              $0
          Platform Edition 9.0 

Sun       Sun Java System Application Server         SJSAS-PE9F-1PR        9      not priced      not priced
            Platform Edition 9.0 
            Premium Support per CPU for 1 year

MySQL     MySQL Database 5.0                         
MySQL     MySQL Network Gold Support                                       3      $0              $0
            for 1 year
                                                                                                  ==========
                                                                                                  $51,870
                                                                                                  ==========

Required Disclosure Statement:

• SPECjAppServer2004 3x Sun Fire X4100 appservers (12 cores, 6 chips) and 1 Sun Fire X4100 database (4 cores, 2 chips) 712.87 SPECjAppServer2004 JOPS@Standard.

• SPECjAppServer 2004 3x Sun Fire V20z appservers (6 cores, 6 chips) and 1 Sun Fire V20z database (2 cores, 2 chips) 266.01 SPECjAppServer2004 JOPS@Standard.

• BEA Systems: SPECjAppServer2004 6x HP DL380 G4 appservers (12 cores, 12 chips ) and 1 HP rx8620-Intel Itanium 2 database (16 cores, 16 chips) 1,664.36 JOPS@Standard.

• All results from www.spec.org as of 05/25/06. SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation

Saturday May 27, 2006

Sun releases new SPECjAppServer2004 benchmark using completely free and open source software

Sun has just posted a new SPECjAppServer2004 result of 712.87 SPECjAppServer2004 JOPS using a totally open source software stack.  Just to make this clear , the application server is open source, the operating system is open source and the database is open source. The hardware is 4 * SunFire X4100 dual cpu dual core servers (i.e. each server has 4 cores, see the diag below). You can find the benchmark result and all the configuration details on the SPEC site here

Highlights

• It is an all open source result and a good demonstration of how to use open source to put together a system capable of running > 5000 simulated web application users for around $US 52K (which is basically the hardware cost as the software is all free)

• It demonstrates the scalability and performance of the free and open source Sun Application Server 9.0 (a.k.a project Glassfish).

• It demonstrates that a 4 core SunFire X4100 server running Solaris 10 and MySQL 5 can support > 5000 concurrent web users !

So why do I care about SPECjAppServer2004 ?

1. Because there is probably something in the workload that can be related to your business
   The SPECjAppServer2004 workload is designed to model a typical modern end to end web based application. The workload has several distinct domains which feature complex and different business transactions. The “customer” domain has logic which performs on line order entry with credit check, the Manufacturing domain models a just in time manufacturing operation and the Supplier domain performs inventory management and supply chain functions including xml calls to external simulated suppliers.
   
   If this isn't relating to your business yet then consider that the benchmark itself simulates concurrent on line users connecting to and using the application , these users connect both via HTTP and RMI. In the just released Sun 712.87 JOPS result there were > 5600 simulated concurrent users running against the appplication for the entire measurement period of 1 hour. [You can approximate the simulated users by looking at the benchmark result http=dealer injection rate * 10 + rmi =dealer injection rate * 3 ]
   
   Also, all of the transactions are persisted to the relational database and from the perspective of the database the entire application looks like a very heavy and highly contended OLTP workload, which makes SPECjAppServer2004 a very interesting tool to test and tune relational database performance!

2. Because Sun and other companies are using SPECjAppServer20004 to improve the performance of your application
   If the application(s) your company uses or develops need a J2EE or J EE 5.0 application server or perhaps JDBC to connect to a relational database or if it uses java or runs on the web then it is quite likely that the work Sun (and others) are doing with SPECjAppServer2004 is going to bring performance benefits to you. In general the way you take advantage of these benefits is to use the latest version of the infrastructure product, for example a (free) upgrade to the Sun application server 9 may give you significant performance improvement in your system with no change to your application. In fact in our testing we saw a measured increase in performance with SPECjAppServer2004 over equivalent tests with Sun application Server 8.1 and again this is with no changes to the application.

3. Because Sun is using SPECjAppServer2004 to demonstrate how you can save money !
   By utilising the SPECjAppServer2004 benchmark in conjunction with a completely open source software “stack” and by working with prominent open source providers like MySQL (www.mysql.org) Sun is demonstrating the reliability and enterprise readiness of open source software and is giving web application developers and deployers a good example of how to save money on IT infrastructure. For a good discussion on the price benefits of the open source Sun SPECjAppServer2004 result have a look at Robert Lee's blog However I encourage you also to check out the price of the system yourself, have a look at the “Bill of Materials” listed in the benchmark result at http://www.spec.org/jAppServer2004/results/res2006q2/jAppServer2004-20060509-00029.html , you can see the part numbers and complete system specifications of what was used then you can go to www.sun.com and price the components and you can check for yourself that the total acquisition cost of the system which supports > 5000 simulated users is around $US 52K !

Pointers and Resources :

• There is a lot of very good tuning information on the “full disclosure report” of the benchmark, you can also see all of the configuration information by downloading the “full disclosure archive”.

• Check out www.opensolaris.com for details on the open source Solaris operating system

• For Sun Application Server 9.0 go to the project Glassfish site at http://glassfish.dev.java.net

• Go to the MySQL site for details and to download MySQL 5.0 also check out Jenny Chen's blog on configuring MySQL and MySQL JDBC driver.

System Diagram (also available as part of the benchmark full disclosure archive)

Required Disclosure Statement:
SPECjAppServer2004 3x Sun Fire X4100 appservers (12 cores, 6 chips) and 1 Sun Fire X4100 database (4 cores, 2 chips) 712.87 SPECjAppServer2004 JOPS@Standard. SPECjAppServer 2004 3x Sun Fire V20z appservers (6 cores, 6 chips) and 1 Sun Fire V20z database (2 cores, 2 chips) 266.01 SPECjAppServer2004 JOPS@Standard. All results from www.spec.org as of 05/25/06. SPEC, SPECjAppServer reg tm of Standard Performance Evaluation Corporation
 

Tuesday June 28, 2005

Configuring Sun Application Server 8.1 (J2EE 1.4) and MySQL 5

Connecting to MySQL from the “free” Sun Application Server 8.1 Platform Edition (PE) J2EE server

Sun has been offering the "platform edition" of our J2EE 1.4 application server free for download and use for sometime now and I am writing this blog from the Sun JavaOne conference in San Francisco where Sun has just announced the open sourcing of the next version of our J2EE application server i.e. the Java EE 5.0 version. See project http://glassfish.dev.java.net/ for details and to download early versions of the code and also check out Jim Driscoll's blog.

Given that the application server is free and open source and the operating system (Solaris 10) is free and open source it seems to follow that many J2EE developers and users will be or are already interested in using open source and free databases with this free combination of application server and operating system. We have been running the SPECjAppServer2004 J2EE benchmark with the Sun application server 8.1 and MySQL 5.0 and we have learnt much about running the app server with MySQL and with Connector/J, so I thought we would share some of the config details which come from this testing to help you get your application running on this same combination. If you are interested in running either Postgres or Derby in conjunction with Sun application server 8.1 then check out Rajesh's blog entry for Postgres and Lance's blog for Derby.

Versions

I suggest using MySQL 5.0.4 (or later) of the MySQL database and version 3.1.8 (or later) of the Connector/J JDBC driver. It is important to note that the Sun application server 8.1 passes the compatibility test suite when using the above versions and this is a big deal because it protects your investment by keeping the platform consistent and your code portable. There are also a number of performance enhancements in the above releases and we have especially noticed significant improvement in the internal SPECjAppServer2004 results due to improvements by MySQL in the Connector/J driver. Also w.r.t versions don't forget if you already have app server 8.1 with JDK 1.4 then a move to J2SE 5.0 will likely boost you performance significantly.

Configuring the connection for MySQL 5 from the application server using the admin console

1. Under Resources, select JDBC and on right hand side frame, select Connection Pools

2. Click on New to add create a new connection pool

3. Provide some name such as mysql-pool and select Datasource type as javax.sql.ConnectionPoolDataSource

4. 4. Click Next and enter the Datasource Classname as com.mysql.jdbc.jdbc2.optional.MysqlDataSource

5. 5. Add the following properties to the connection pool:

6. DatabaseName <your-db-name>

7. port <your-selected-port-number> default port is 3306

8. user <DB-User-Name>

9. Password <DB-User-Password>

10. ServerName <DB-Server-Name>

11. Save the settings and click on Ping. If DB server is running and connection can be made, Ping will succeed. Please make sure you have copied the mysql-connector-java-3.1.8-bin.jar jdbc driver in $J2EE_HOME/domains/<domain-name>/lib/ext directory.

Example domain.xml

This is entire connection pool entry for the current testing we are doing with SPECjAppServer2004 (see www.spec.org/SPECjAppServer2004 for more details on the workload and check out the Connector/J documentation for an explanation of the properties.

<jdbc-connection-pool

connection-validation-method="table" datasource- classname="com.mysql.jdbc.jdbc2.optional.MysqlDataSource"
fail-all-connections="false" idle-timeout-in-seconds="300" is-connection-validation-required="false"
is-isolation-level-guaranteed="false" max-pool-size="80" max-wait-time-in-millis="90000"
name="SpecJPool" pool-resize-quantity="4" res-type="javax.sql.DataSource" steady-pool-size="80">

<property name="serverName" value="pdb"/>

<property name="portNumber" value="3306"/>

<property name="User" value="spec"/>

<property name="Password" value="spec"/>

<property name="DatabaseName" value="specdb"/>

<!-- Note : the following are tuning parameters not essential for connectivity

<property name="cachePrepStmts" value="true"/>

<property name="prepStmtCacheSize" value="512"/>

<property name="useServerPrepStmts" value="false"/>

<property name="alwaysSendSetIsolation" value="false"/>

<property name="useLocalSessionState" value="true"/>

<property name="elideSetAutoCommit" value="true"/>

<property name="useUsageAdvisor" value="false"/>

<property name="useReadAheadInput" value="false"/>

<property name="useUnbufferedInput" value="false"/>

-->

</jdbc-connection-pool>