BlogX 

Monday March 26, 2007

An improvement on the “Holder” thread-safety initialization idiom

// Listing 1
public class Resource {
  private static final Resource _Instance = new Resource();
  public Resource getInstance() {
    return _Instance;
}

// Listing 2 (prefer Listing 1 above if appropriate)
public class ResourceFactory {
  private static class ResourceHolder {
    public static Resource resource = new Resource();
  }

  public static Resource getResource() {
    return ResourceHolder.resource; 
  }
}

// Listing 3
public final class ResourceFactory {
    private ResourceFactory() {/*disallow instantiation*/}

    private static final class ResourceHolder {
    	private ResourceHolder() {/*disallow instantiation*/}
        public static final Resource resource = new Resource();
    }

    public static Resource getResource() {
        return ResourceHolder.resource;
    }
}

Thursday March 01, 2007

GlassFish startup time on Mac Pro 3GHz

I tested GlassFish startup times on my Mac Pro 3GHz and a MacBook Pro Core 2 Duo 2.33 GHz. Times exclude the 'asadmin' overhead and are from initialization of the PELaunch class to the end of the startup sequence in the PEMain class.

Mac Pro 3GHz:  6.3 seconds
MacBook Pro 2.33GHz:  8.3 seconds

These times are for an EE version ('maven configure-cluster'), and represent a full startup, not a quick startup which defers initialization of some components. Times include some optimizations that have not yet been committed to the cvs repository; add about 2 seconds for the current trunk.

The Mac Pro is a seriously fast machine for most tasks, and does not disappoint for GlassFish startup time. However, the largely single-threaded nature of the GlassFish startup process leaves about 70% of the available CPU power unused.

GlassFish build time on Mac Pro 3GHz

In my Nov 27, 2006 entry, I tested GlassFish build times on the MacBook Pro Core 2 Duo et al. Those times were stunningly fast, but it seems that the Mac Pro is much faster still. The figures quoted indicate the time to execute 'maven build' (time to perform 'maven checkout' and 'maven bootstrap-all' is not included, since those steps are highly dependent on network speed.)

Mac Pro 3GHz:  5 minutes 33 seconds
MacBook Pro 2.33GHz:  8 minutes 26 seconds

The Mac Pro is a seriously fast machine for most tasks, and does not disappoint when building GlassFish.

Friday February 09, 2007

Understanding thread-safe code

I’ve written quite a bit of threaded code over the years, much of it in C and C++, including interrupt-driven disk-driver code. But the Java language has its own approach to the problem which does require some careful study.

About two months ago, I heard Brian Goetz give a short talk on concurrency in Java. I promptly purchased Java Concurrency in Practice using my own funds, my take being that understanding concurrency is a key issue with Java, and therefore a requirement for a serious professional. The book is excellent—a must-read for anyone programming in Java, especially in a server environment. Scanning my old code for mistakes, I learned that I had (mostly) been doing things right. The one exception was an “unsafe publication” of an Object as a listener.

In reading Java Concurrency in Practice, I learned some important facts about Java’s concurrency which subsequent research revealed to be misunderstood by many of server engineers that I have interacted with. While a few engineers thanked me for filing bugs, there was an undercurrent of annoyance that I filed so many bugs approaching beta. People are happier not knowing sometimes, especially those concerned primarily with looking good to others.

The key concepts to be aware of include:

• Threads are everywhere—Swing and AWT, RMI invocation, Servlets & JSPs, Timers, RMI, etc.
• Visibility—why a variable might not be “seen” by threads other than the thread that set the variable.
• Publication and escape, thread confinement—when a variable escapes from its “owning” thread.
• Immutability—a “constant” object.
• Use of 'final'—making a variable (reference) immutable within an object.
• Race Conditions—two or more threads interacting to produce unexpected results.
• Deadlock—getting “stuck” forever.
• Performance—too few or too many threads worsening performance.

I won’t cover these here—get a copy of the book and read the excellent discussion.

In my recent work on the GlassFish codebase, attempting to achieve a high degree of concurrency with the startup code, it’s clear that over the years a great amount of thread-unsafe code has accumulated, making it somewhat risky to thread existing code that has for years been run on the main thread at server startup. This code has worked only because it ran on a single thread at startup.

But the world is changing—Sun is expected to offer 64-core machines in 2007, Intel already offers quad-core processors for the desktop, and indeed all of Apple’s line are dual or quad core. A server that starts in single-threaded mode is embarrassing for a company that will offer a 64-core machine in 2007!

Common bugs

The most common bug I’ve seen so far (roughly 100 instances) is the Null Check Idiom. This idiom suffers from both a race condition and a visibility problem; two (or more) threads could all conclude that “_Instance” is null, and then make a new one, each one overwriting the previous value. At the same time, each thread could continue to see its own copy, or the copy created by some other thread, all depending on many variables, including processor architecture, JVM implementation and other code running elsewhere (which might flush caches). An example is shown below.

private static Something _Instance=null;
public static Something getInstance() {
    if ( _Instance == null ) {
        _Instance = new Something();
    }
    return _Instance;
}

The simplest fix is to insert the keyword synchronized. A better fix is to make _Instance be final as in:

private static final Something _Instance=new Something();
public static Something getInstance() {
    return _Instance;
}

Afer all, if the class is loaded, the very first method called is going to be getInstance()—so the class might as well create itself when it’s loaded. There are other solutions, including an inner class, described in Java Concurrency in Practice.

Underused: final

Having now randomly inspected code for threading bugs across a wide swath of the GlassFish codebase, I can say with certainty that the keyword final is grossly underutilized.

I have some background from my college days in proving program correctness, and when I examine code, I like to see as many invariants as possible—and 'final' is one of those. An Object declared 'final' cannot change (though its contents can), and this is not only extremely helpful in understanding the correctness of the code, but it is inherently thread-safe (for the variable itself). Yet it is not just that 'final' isn’t used—much code is actively written in a way to make it hard to use it, making it not just thread-unsafe, but unclear in intent; is the variable intended to be changed later or not?

In many cases, the solution is trivial:

before: static Logger _logger = Logger.getLogger(...);
after: private static final Logger _logger = Logger.getLogger(...);

I gave up fixing such things in other people’s code after more than one idiotic request to run all the tests (in essence proving that the Java compiler works). Sloppy code like this should be corrected opportunistically, and adding 'final' has got to be the least risky change I can think of.

Sometimes the fix is a little more involved. It will look something like this:

class Something {
     private Integer instanceVar = null;
     Something(...) {
      instanceVar = new Integer(10);
    }
}

The code above is a particularly poor approach, since there's simply no reason to write it that way, and one must search for all uses of 'instanceVar' to see if there are threading issues. In 9 of 10 cases I’ve seen, it can be fixed trivially as follows:

class Something {
     private final Integer instanceVar;
     Something(...) {
      instanceVar = new Integer(10);
    }
}

The compiler will detect if 'instanceVar' is mutable. If so, then the variable cannnot be made final. But in some cases, it only looks like it’s mutable; a slight code change for initialization makes it immutable.

Sometimes it’s just a case of using a temporary variable or a method to allow use of 'final'. In the code below, variable 'instanceVar' can be made 'final' by using a temporary variable (no matter how complex the logic), then assigning 'instanceVar'. A method can be used for more comple cases.

class Something {
     private Integer instanceVar;
     Something( boolean maybe ) {
       instanceVar = new Integer(0);
       if ( maybe ) {
           instanceVar = new Integer(10);
      }
     else {
         instanceVar = new Integer(1);
     }
    }
}

Not taken seriously

That thread-safety is not taken seriously by some developers is evident in this very recent quote:

Theoretically what you did is correct for a thread safe code. But I believe it is not incorrect to alter the theoretical code to suit our requirements in real situations...

Huh? In this case, incorrect code is used to justify performance (unmeasured and almost certainly unmeasurable). What exactly is a “real situation”? I was just dumbfounded to read this—I’ll mince no words: it is simply an amateur hacker speaking, not a professional.

It doesn’t really cause a problem

If one can’t prove that any particular piece of thread-unsafe code actually results in a bug at a customer site, should it be left alone, like a cancer that one thinks won’t ever cause a problem?

That’s a common approach, usually justified by a dogmatic claim that there’s a risk to fixing it. But usually, the fix is simple and the risk very low, based on the bugs I’ve actually come across in the code base. There are more complex cases of course, but the case could be made that complex situations are an even stronger argument for resolving the problem, since the risk of sporadic failure might be much harder to understand, not to mention the code being unmaintainable—who’s to say what sudden instability might result from a code change that alters the flow of execution?

It is also a position that I consider untenable—we’ve all run software that sporadically misbehaves—maybe only once a week—what is the probability that a once-a-week bug will actually filter back to the software vendor, found as a threading bug, and fixed? Next to nil. So the software vendor blithely continues to place a low priority on fixing thread-unsafe code that “doesn’t actually cause a problem in practice”. It’s a lame excuse all too common in the software industry today, coming from an attitude that doesn’t honor or value the customer. Unfortunately, the FUD factor (Fear, Uncertainty, Doubt) is enough to justify such reprehensible behavior.

Friday January 12, 2007

Sun Ultra 20 dual-core Workstation

A dual core desktop box should be faster than a laptop, right?

Guess again--I compared my nearly-new Sun Ultra 20 workstation (dual core, 2GB memory) to my MacBook Pro 2.33GHz Intel Core 2 Duo for building Glassfish.

I have no clue how to find its clock speed--nothing unusual with inherently obtuse Solaris operating system, unusable for anyone but geeks. (If you know the name of the command, you can use the dismal 1970's era help, also known as "man". But if you don't know it (I don't), you can't find the right command to begin with!) With a Mac, I can just pick "About This Mac".

But the "workstation" is faster, right? Well...not exactly (times are minutes:seconds)

Apple MacBook Pro Core 2 Duo 2.33GHz:   6:50
Sun Ultra 20 workstation:  12:12

Some workstation...taking nearly twice as long to build, far noiser, consuming much more power, and using a full size hard disk as compared with a laptop hard disk (albeit 7200rpm), and a pitiful user interface compared to Mac OS X, the Sun workstation looks like yesterday's cold chicken.

But it's less expensive, right? Just for giggles, I configured a basic setup at US$3400, $700 more than an Apple 17" 2.33GHz MacBook Pro 2GB/100GB would cost without any discounts:

Sun Ultra 20 M2 Workstation
1 AMD Opteron Model 1218 Processor (Dual Core)
1 MB On Chip L2 Cache
2 GB DDR2-667 Memory
1 x 250 GB 7200 rpm SATA Disk Drive
1 NVIDIA Quadro FX 1500 Graphics Accelerator
1 DVD-ROM Drive
2 x 10/100/1000 BaseT Ethernet Ports
6 USB Ports
2 IEEE 1394a Ports
1 PCI-Express x16 Slot
2 PCI-Express x1 Slots
4 Conventional PCI Slots (32-Bit/33-MHz)
Solaris 10 Operating System Preinstalled

Do the math: a little more than 1/2 the speed, non-portable, 26% more money, an operating system that is useless for anybody but a unix geek.

Oh, and did I mention that it fried its boot disk within 1 minute of setting it up, and no Solaris 10 disk is included in the box to do a system restore, and the screen wouldn't light up without buying a DVI to VGA adapter? See my previous entry on that.

It's almost not fair having to compete with Apple.

Monday November 27, 2006

MacBook Pro Core 2 Duo Shines on Glassfish build speed

MBP C2D, internal drive:  8:25, then 11:18

MBP C2D, external 7200rpm FW800:  7:56, then 10:55

MBP, internal drive:  10:20, then 16:22

Total time: 8 minutes 25 seconds
real    8m26.179s
user    6m19.822s
sys     1m10.709s

Total time: 11 minutes 17 seconds
real    11m18.155s
user    9m3.773s
sys     1m32.450s

Total time: 7 minutes 56 seconds
real    7m56.584s
user    6m20.562s
sys     1m17.259s

Total time: 10 minutes 55 seconds
real    10m55.467s
user    9m4.108s
sys     1m39.841s

Total time: 10 minutes 20 seconds
real    10m21.409s
user    8m14.863s
sys     1m19.307s

Total time: 16 minutes 22 seconds
real    16m22.395s
user    13m51.397s
sys     1m47.987s

Tuesday November 21, 2006

Poor out-of-the-box Ultra 20 experience

Friday August 04, 2006

Javadoc shortcomings with super-interfaces

More than a few times, I've received questions about what routine to use in an AMX interface—questions as simple as getting the name of something [the method getName(), of course!].

For example, here is a question I received via email today, from an experienced developer:

I'm using the AMX interface "ProfilerConfig" to get the attribute values and don't see a way to get the profiler name and if it's enabled.

The problem is the poor presentation offered by the API documentation generated via the 'javadoc' tool. This tool offers no option to list all methods, including inherited ones, in the same listing as the methods defined by the interface itself.

As a result, they are hard to spot, being listed as part of other interfaces from which the ProfilerConfig extends. At the top of the page, you have to guess (!) which of the super-interfaces might contain the method of interest:

The method you’re looking for ought to be in the Method Summary table, right? Wrong!The Method Summary table shows only methods defined by the interface itself:

So it doesn’t list the method(s) you’re looking for. Do the method(s) not exist? Hmmm...the only way to find out is to search the page and/or visually scan the page and hope you don’t miss it by mistake. So you now have to look through apparently an apparently arbitrarily-organized list:

It’s no wonder that I regularly receive questions as to what method to use! It would be better if the 'javadoc' tool offered an All Method Summary akin to the currently-offered Method summary so the hapless reader could skim through all the available methods in alphabetical order, rather than having to look through the All Method list, then tediously look through each inherited class or interface one-by-one. After all, does it really matter which interface defined the method, given that the goal is quite possibly to invoke the method on an Object which implements it?

Thursday August 03, 2006

Glassfish build times on MacBook Pro with a faster hard disk—

Glassfish build times on a MacBook Pro can be sped up somewhat by use of a faster disk, but less than might be expected...

I tested Glassfish build times using different-speed hard disk setups.

The first setup was the 7200 rpm internal hard disk, with its 8MB onboard cache.

The second setup was a 2-drive striped RAID volume, with a 16GB partition created on the fastest part of the drives. The two drives were both Maxtor 7V300F0 300GB hard drives (about $119), 7200 rpm with 16MB cache each, known fast performers. They were installed in a FirmTek SeriTek/2EN2 and connected via FirmTek’s SeriTek/2SM2-E Express Card adapter. I used Apple’s Mac OS X Disk Utility to create the striped RAID volume. This setup is considerably faster for sustained I/O than the internal hard disk (3-5 times faster), so I was curious if the much faster transfer rates and larger caches would translate to faster Glassfish build times.

I ran the sequence of commands shown below. Build times increase with the 2nd build because the jar files are much larger at the beginning of any build after the first one. This is a performance bug in the build if your intent is to do an entirely fresh build.

Striped RAID:

maven checkout: 2:43
maven bootstrap-all: 1:39
maven build: 9:06
maven clean: 0:40
maven build: 14:03
maven clean: 0:40
maven build: 14:03

Internal disk:

maven checkout: 3:07
maven bootstrap-all: 1:38
maven build: 9:42
maven clean: 0:45
maven build: 14:55
maven clean: 0:43
maven build: 14:40

Conclusion—A substantially faster hard disk has on only a modest impact on build speeds (about 6%). This is somewhat surprising, but the single-threaded nature of the Glassfish build (which uses 100% of one core while running) effectively limits any benefits that might accrue from a faster hard disk setup.

Monday May 15, 2006

Appserver 9 Platform Edition (Glassfish) comments — Java Enterprise Edition (Java EE) has taken another step forward with the release of version 9.0, Platform Edition aka Java EE SDK. I'll simply refer to it here as “Glassfish”.

I’ve worked as an engineer on various AppServers now for 8 years, and I can say that Glassfish is the culmination of a lot of work, and offers perhaps the best web-based GUI around for an AppServer. The command line interface (CLI) 'asadmin' is also available, and it can do virtually everything for managing and monitoring the appserver.

Last, but not least on the management front, the AMX (Appserver Management Extensions) API in Glassfish offers an extensive easy-to-use JMX (Java Management Extensions) interface, and powerful features like real-time remote monitoring of the server logs via JMX Notifications, querying for past events in the logs of all server instances (in enterprise edition), and of course, extensive monitoring Statistics. AMX is where I’ve spent the better part of 2 years, and I’m pleased with how far it has come.

If you’re a customer or user, I’d be interested in hearing from you on how you’d like to be able to configure or monitor the AppServer, and if you have specific questions, especially those that might lend themselves to a blog, please email me (lloyd.chambers). At sun.com. Posted by llc ( May 15 2006, 11:31:43 AM PDT ) Permalink Comments [0]

Thursday May 04, 2006

Using JMX to configure and monitor Glassfish —

With Sun Java System AppServer 9.0 (Glassfish), extensive JMX (Java Management Extensions) support is included. You can configure nearly every aspect of the server, and you can monitor the server, including logging via near-real-time notifications. Check out the Glassfish AMX page for more details.

Speedy startup of Glassfish on MacBook Pro —

When developing software, a program that can be started quickly offers a more productive experience than one that takes a long time to start.

On my prior development system, a dual 2.2 GHz Opteron running WindowsXP with 2GB of RAM and a 2-disk striped RAID, starting glassfish took 20-30 seconds. Compare that to the 5.6 seconds on the MacBook Pro, which also includes the time to launch 'asadmin' itself. Note also that startup is I/O bound; only about 2.3 seconds of CPU time are used.

MB:/gf lloyd$ time run/bin/asadmin start-domain
Starting Domain domain1, please wait.
Log redirected to /gf/run/domains/domain1/logs/server.log.
Domain domain1 is ready to receive client requests.
Additional services are being started in background.
Domain [domain1] is running [Sun Java System
Application Server Platform Edition 9.0 (build )]
with its configuration and logs at:
[/gf/run/domains].
Admin Console is available at [http://localhost:4849].
Use the same port [4849] for "asadmin" commands.
User web applications are available at these URLs:
[http://localhost:8080 https://localhost:8181 ].
Following web-contexts are available:
[/web1 /asadmin ].
Standard JMX Clients (like JConsole) can connect to JMXServiceURL:

[service:jmx:rmi:///jndi/rmi://MB.local:8686/jmxrmi]
for domain management purposes.
Domain listens on at least following ports for connections:
[8080 8181 4849 3700 3820 3920 8686 ].

real    0m5.575s
user    0m2.064s
sys     0m0.245s

[See also Building Glassfish on MacOS 10.4 and MacBook Pro Experience Report]

Posted by llc ( May 04 2006, 03:47:34 PM PDT ) Permalink Comments [0]

Monday May 01, 2006

Building Glassfish on MacOS X — Notes on building Glassfish on a MacBook Pro [Read More] Posted by llc ( May 01 2006, 12:22:55 PM PDT ) Permalink Comments [0]

Thursday April 20, 2006

Building Glassfish—MacBook Pro vs the competition —

MacBook Pro background

I recently acquired a MacBookPro 2.16 GHz with the optional 7200rpm hard drive and 2GB memory (1 GB from Apple, 1GB from satech.com). Whereas past Apple laptops have been so anemic as to be useless for serious development, the MacBook Pro is a huge leap forward, whether you want to run MacOS, Linux or Windows (which you can on the MacBook Pro!). If you want to read more on the stellar performance and flexibility of the MacBook Pro, see my Apple MacBook Pro Experience Report at diglloyd.com.

Let's start with the stuff that gets overworked developers excited—build speed.

The Contenders

I setup my Glassfish build environment on the MacBook Pro in about 15 minutes (more on setup later). I compared three systems:

• the MacBook Pro 2.16 GHz with 7200rpm hard disk, 2GB RAM, MacOS X
• a dual 2.2GHz Opteron 248 box with 2GB RAM, 2 X 10000rpm striped RAID, Windows XP
• a PowerMac G5 Quad with 8GB RAM, 3 X 7200 RPM striped RAID, Mac OS X

My test consisted of timing the 'maven build' portion of the following:

maven clean clobber
maven bootstrap-all checkout
maven build <===  timed this portion

The Results

MacBook Pro: 9 minutes 53 seconds
Dual Opteron: 12 minutes 30 seconds
PowerMac Quad: 24 minutes
    

I have double-checked these results. If you want a *fast* development machine, get a MacBook Pro! If you like viruses, you can always run Windows XP on it (either booting directly or simultaneously with MacOS X by using virtualization software such as Parallels), and you can run Linux and/or Solaris as well.

As fast as it is on the build of Glassfish, the dual-core MacBook Pro used only 92% of the CPU cycles (out of 200% max). The Glassfish build is single-threaded, and the use of monolithic jars such as appserv-rt.jar make it impossible to run different parts of the build at the same time. Perhaps future revisions to the build will address this productivity-killer.

I can't explain the Quad's terrible performance, except to guess that Apple's implementation of 'java' must be particularly poorly implemented on PowerPC, since nothing else on my Quad runs that slowly. Quad or not, only one of its cores (of 4) gets used, due to the single-threaded build. When using XCode on some C++ development, I see 400% CPU usage during builds. Still, even if all 4 cores were used, the build time would be around 6 minutes, whereas if both cores of the MacBook Pro were used, the build time should drop to around 5 minutes.

Posted by llc ( Apr 20 2006, 05:59:06 PM PDT ) Permalink Comments [4]