Thursday December 22, 2005 Brian Doherty's Weblog
Thursday December 22, 2005 I'm not one to typically take a competitor's bait, but I'm really getting tired of BEA making baseless claims on JRockit being the 'world's fastest JVM', when there's plenty of evidence to the contrary.
BEA's latest tactic is the JRockit verses Sun Java™ runtime challenge. According to the anonymous blogger (how droll), there's only one rule:
- "You can't have more than 20 lines in a main()"
This rule is ambiguous. What do they consider a "line"? If I take it literally, it seems that I could submit any program, including a 1 million line program made up of many classes and methods, as long as main() is less than 20 lines. I suppose I could submit some industry or de facto standard benchmark subtly modified, if necessary, to have a main() less than 20 lines of code. I could just rename main() to oldmain() and introduce a new main() that calls oldmain(), and now I have a main() that's easily less than 20 lines of code. I suppose I could also submit a program with a main() comprised of thousands of Java statements all concatenated together on a single line. Somehow, though, I don't think that's what they are after. I think they are looking for little micro benchmarks. Otherwise, why would they even mention a line count limit?
Why do you think they want 20 line programs? Because they want to stack the cards in their favor. How's that, you say? Well, there are some significant differences in the JIT designs between HotSpot™ and JRockit. JRockit compiles methods as soon as it sees them, whereas HotSpot will run methods in the interpreter for a while and collect profiling data on them and use that profiling data to compile the methods based on dynamic runtime information. For simple little benchmarks, HotSpot will typically run interpreted for a while, and quite possibly for the entire run. However, JRockit will compile this method out of the gate and run the entire method in compiled code. Who do you think is going to win with most of these 20 line programs?
Why do these two platforms have different JIT policies? In some respects, that's part of the power of the Java™ technology platform. The Java Virtual Machine specification, and the API specification provide for different but compatible implementations of the Java platform. At Sun, we designed the HotSpot JVM to meet the needs of both client and server Java applications, and our JIT policies reflect that design decision. By putting off compilation until we're sure there's a win we can provide good startup time and memory utilization for client applications while still providing industry leading performance for all kinds of applications (not just one outdated benchmark). This design leverages well known trade-offs in computer science. Remember the 80:20 rule? 80% of your performance comes from 20% of your code. How about Donald Knuth's 'premature optimization is the root of all evil'? By not wasting time compiling code that your application rarely uses, we reduce startup time and reduce memory utilization. These are not just client issues. Reduced startup time also means reduced recovery times, important for high availability applications, and reduced memory utilization means larger heaps or more threads or other memory needs.
We talk about micro benchmarks every year at JavaOne. It would be nice if we didn't have to, but we regularly see poorly written micro benchmarks that don't measure what the author intended them to measure. In this challenge, the micro benchmarks are likely to be comparing the results from JRockit JIT compiled code to the results of HotSpot interpreted code. Is that what your real applications are going to see? Does that really sound like an 'Apples* to Apples" comparison?
Micro benchmarks can sometimes be useful, but you have to be very careful when writing the code and when interpreting the results. What you might think is a clear indication of one system being better than another may just be artifacts of default configurations, subtle timing differences or implementation differences that skew the results in ways only observable in the context of the micro benchmark. Changes in command line options (tuning) or subtle changes to the micro benchmark code to properly warm up the JVM or preserve the computation of some loop can produce dramatically different results. Ignoring such matters is likely to cause you to make decisions based on results that don't model the realistic conditions seen by real applications.
If you really want to see the performance difference between HotSpot and JRockit, run your own benchmarks on your own real applications. Make sure that you run them long enough that both environments are sufficiently warmed up, as both perform various dynamic optimizations. I'm confident that you'll find that HotSpot has very competitive performance compared to any JVM in the market. But don't forget to put both systems through your suite of reliability tests as well, as performance without reliability is worthless. If you find a case where JRockit outperforms HotSpot, then please file a bug against HotSpot, as we consider any realistic performance difference to be a bug. If you don't want to file a bug, then please post something about your benchmark in the discussions section of the Java Performance Community web site.
* HotSpot does run on Apple thanks to the excellent port done by the engineers at Apple. It also runs on HP's PA-RISC and Itanium based HP-UX systems thanks to the work by HP's engineers. Not to mention all the Sun supported platforms, Sun's real time Java implementation, also based on HotSpot, and countless other ports to various embedded systems.
Posted by briand ( Dec 22 2005, 10:34:36 AM CST ) Permalink Comments [6]
Tuesday December 06, 2005 Don't judge the UltraSPARC T1 book by its cover
At first glance, the new Sun Fire T1000 and Sun Fire T2000 systems look like your typical one, two, or four way, rack and stack, 1U/2U servers. Knowing that there's only one UltraSPARC T1 chip in these systems somewhat reinforces the notion that this is a small system. But don't let the physical size of this system or the chip-count fool you! From the perspective of your Java application, this is a pretty big system! The UltraSPARC T1 chip has up to 8 cores, with 4 hardware threads per core, for a total of 32 hardware threads. The Java Virtual Machine and your Java application see this as a system with 32 CPUs.
There are many applications that will dynamically create threads and other objects based on the number of CPUs available in the system. The JVM itself creates a pool of parallel garbage collection threads based on the number of available CPUs. This can result in some surprises, particularly for Java applications running in a 32-bit JVM that are already pushing the limits of the 32-bit virtual address space on systems with smaller numbers of CPUs.
When moving from a system with 4 hardware threads to 32 hardware threads, each additional application thread will result in additional reserved virtual address space for the thread's stack. With JDK 5.0 and the HotSpot JVM, the default options will result in a net increase of 28 parallel garbage collection threads, each with 512KB of reserved virtual memory for their thread stack, for a total of 14MB of additional reserved virtual memory. If the Java application is also sizing thread pools based on available CPUs, 512KB of virtual memory memory will be reserved for each of those thread stacks. If you are well below the 4GB virtual address space limit of a 32-bit process, then this additional virtual memory probably won't matter. However, if you are pushing the limits of the 32-bit virtual memory address space, the additional memory reservations may push you over the top, or get you uncomfortably close to it, and could result in out of memory errors.
Fortunately, there are some techniques to address this issue.
- Reduce the size of the Java heap.
Reducing the maximum size of the Java heap (-Xmx) will allow some of the freed address space to be used for other purposes. This is typically not a viable solution, though, as the additional CPU resources on the UltraSPARC T1 may actually require a larger Java heap in order to handle the increased application throughput. - Reduce the default thread stack size.
The default thread stack size can be changed using the -Xss option. Selecting a value such as 256k or 128k will reduce the amount of virtual memory reserved for each Java thread's stack. For applications that are highly recursive or otherwise require deep thread stacks, this too may not be a viable solution. - Limit the number of dynamically created threads.
The number of parallel garbage collection threads used by the HotSpot JVM can be set explicitly with the -XX:+ParallelGCThreads=N option. Java applications that dynamically create threads typically have options to override the default size of their thread pools. Reducing the sizes of thread pools might also reduce the throughput of the operations that these thread pools handle. Reducing the size of the parallel gc thread pool is typically a good idea on UltraSPARC T1 systems. Two parallel gc threads per core is a good starting point. - Partition work into multiple processes.
Many applications can partition their workload across multiple processes. When applying this approach, it's generally wise to also limit the number of threads created in each process as mentioned above. It might also be advantageous to run these processes in their own processor set or under the control of the Solaris Resource Manager. - Partition work into multiple zones.
This approach is similar to the 'partition work into multiple processes' approach. However, some applications can only partition a workload across processes running on different operating system instances. The Solaris Zones feature will allow this approach to work by providing different operating systems instances on one physical system. - Use a 64-bit JVM.
All these process address space issues can be completely avoided by adding the -d64 option to the command line. If the application has any native libraries, those libraries will need to be compiled for the 64-bit data model to make use of this option. Some applications will see increased latencies with a 64-bit JVM (due to larger data sizes and cache effects). However, by increasing the size of the Java heap, many applications will also see increased throughput. Many system administrators will find the single 64-bit JVM easier to manage than the partitioned alternatives.
If your application has never run on a 32 CPU machine, it may have it's own scalability problems. Fortunately, the HotSpot JVM has been scaling to larger numbers of CPUs and large Java heaps for years. If you find that your application doesn't scale well, try running it with JDK 1.5.0_06 with the -XX:+UseBiasedLocking option, which can help improve the performance of applications with significant amounts of uncontended locking. If you are having a specific problem with your Java technology application on the UltraSPARC T1 based platforms, please contact us at the Java Performance Community Website
The new UltraSPARC T1 systems are exciting products for Java applications that need throughput. Many existing Java technology server applications will scale up nicely on these systems with minimal tuning when running with JDK 1.5.0_06 and the HotSpot JVM. With a little tuning, you can get high levels of throughput with these systems and you can do so with significant savings in power consumption, cooling, and floor space.
Benchmark and power consumption results are available here.
[ T: http://technorati.com/tag/NiagaraCMT ]
Posted by briand ( Dec 06 2005, 12:05:48 PM CST ) Permalink Comments [0]
Monday December 05, 2005 The first program in a new programming language that just about any programmer writes is the quintessential "Hello World" program. While blogging is not technically a programming language, this is the first time I've written "Hello World!" where it actually has meaning!
I'm a software engineer at Sun working in the Java™ Performance Engineering group. We focus on performance measurement and analysis of the Java™ technology software stack. In our work, we utilize benchmarks from various sources, including industry benchmarks, internal benchmarks, and customer benchmarks, for performance analysis, regression detection, and competitive analysis purposes.
I'll occasionally post blogs on various Java™ technology performance topics. They may range from general advise to ramblings or frustrations. Since I'm also the author of the jvmstat tools, I'll likely be posting on that topic too. Hopefully, you'll find what I have to say informative, though I'll also try to keep it at least somewhat interesting.
Posted by briand ( Dec 05 2005, 02:44:51 PM CST ) Permalink Comments [2]
