Going on a Warning Hunt

Like the children's story "Going on a Bear Hunt" it's time for a "Going on a Warning Hunt" in the OpenJDK sources.

You'll probably see a set of bugs/rfes being filed and fixed over the next few months related to reducing the warning messages in the OpenJDK builds.

As the warnings get fixed, we also may want to dial up the warnings found with compiler options that ask for more warning diagnostics. We could even include the use of tools like pscan and findbugs, but that's beyond the scope of this hunt. Compiler options to get more warning diagnostics, treat warnings as fatal, and suppress specific warnings:

Compiler	Warnings Requested	Make Warnings Fatal	Disable Specific Warnings
gcc/g++	-W -Wall -Wno-unused -Wno-parentheses	-Werror	-Wno-option
Sun Studio C++	+w (Added recently in jdk Makefiles)	-errwarn=%all	-erroff=tag (Use -errtags to get tags)
Sun Studio C	-v	-errwarn=%all	-erroff=tag (Use -errtags to get tags)
Visual Studio	-W3	-WX	-wdnumber
Java Compiler	-Xlint:all	-Werror	In the source: @SuppressWarnings("warning")

Making warnings fatal can be problematic. Each compiler version or release can introduce new warnings and it's a bit frustrating to get a build failure because you used a slightly different compiler. So we may need to make sure the compiler is a specific version or release before making the warnings fatal. The Java Compiler may be the exception, but it may be a while before we get to the point where we can turn this on.

Fixing warnings in native code can be more risky than fixing them in Java code. Some common things to watch out for:

• Casting pointers to and from integers can be tricky, it shouldn't happen very often and when it does great care needs to be taken in making sure the complete pointer is contained in the integer type. Pointers will be 64 bits in some situations and will require a integer big enough to hold the full 64 bits. The type void* is the so called generic pointer and you may find it necessary to cast a pointer to the generic pointer type before casting it to anything else. Some people will use size_t as a guaranteed integer type that can hold a pointer, others use intptr_t which is the integral type result for pointer subtracts.
• Casts to long. Windows compilers may define long to be 32 bits even in 64 bit mode, so be careful casting any kind of pointer to a long.
• Functions like strlen() return the type size_t which may be a 64 bit integral type. It's not unusual to always cast this type to an int but be careful to consider if it ever makes sense that you might need a larger result than what can be held in a 32 bit integer.
• It's a good idea to always be explicit when you are downsizing an integral value. Passing an int to a function that expects a short probably warrants an explicit cast to int. Many compiler warnings relate to the implicit downsizing conversion the compiler will perform on assignments or arguments passed into function calls.
• No consequence statement warnings happen when the expression statement makes no assignments or function calls. This can happen with debug statements that result in (0). Macros like assert(expression) should result in nothing when not generating debug code.
• Suppressing warnings with options or #pragma directives should probably be a last resort.

So far the bugs I have included for this warning hunt are:

• 6255702: Need to clean up remaining var-args related warnings.
• 6214406: MS VC++ compiler warning at jni.h when using fastcall, enable optimization
• 6323942: jvmti.h causes C compiler warning
• 6375033: Hotspot build warnings need cleanup
• 6496269: Many warnings generated from com/sun/java/util/jar/pack/*.cpp when compiled on Linux
• 6712344: Add -errwarn=%all to Solaris Sun Studio compiles (all warnings being fatal)
• 6722802: Code improvement and warnings removing from the javax.swing.text pa ckage
• 6725543: Compiler warnings in serviceability native code
• 6725625: Compiler warnings in awt code
• 6725818: Compiler warnings in OpenGL code
• 6725819: Compiler warnings in gtk2 glue code
• 6725821: Compiler warnings in medialib code
• 6726309: Eliminate warnings from NIO code
• 6727661: Code improvement and warnings removing from the swing/plaf packages
• 6727662: Code improvement and warnings removing from swing packages
• 6727683: Cleanup use of COMPILER_WARNINGS_FATAL in makefiles

And the hunt is on. Lock and load. :^)

-kto

Posted at 05:07PM Jul 21, 2008 by kto in Java  |  Comments[9]

Alaskan Cruise Vacation

Just returned June 26th from a 10 day Alaskan Cruise aboard the Dawn Princess, leaving from and returning to San Francisco.

This was my family's 3rd cruise, and our second cruise to Alaska. It's the first time we left from San Francisco, having flown to Anchorage, Alaska for the start of our previous 7 day cruise in 1999 which left us in Vancouver, and having flown to Hawaii for the cruise we had around the islands. We enjoy the Alaskan cruises, and in particular the Princess Cruise Line, but what do we know, only having cruised 3 times. During the cruise they awarded someone a free cruise for having cruised with Princess Cruise Line (just Princess mind you) 72 times (over 700 days of cruising!).

For anyone that has never been on a cruise, it's a very spoiling experience. The food is excellent (and part of the cruise price), the rooms are small (but who stays in their room other than to sleep), the shows are free (granted, they should have paid me for a couple of them), the view is to die for (assuming you like the ocean), the staff was excellent (even in the face of a few rude passengers), and the ports in Alaska are fascinating.

Why did we do a cruise? Well, our first choice was to fly to the East coast, maybe Washington D.C., or New York. But if you add up the flights, the hotels, the food, the rental car, etc. The Alaska Cruise inside stateroom was something like $1100 or $1200, per person double occupancy. It included food, no rental car needed, no hotel costs, and leaving from San Francisco meant no airplanes or airport hassles. Granted, drinks, soda, shore excursions, and misc expenses will happen on a cruise, but overall, we decided on the cruise. (I'm really beginning to hate airplanes and airports).

The Dawn Princess holds something like 1,900 passengers (plus 900 or so crew members) and is no small ship, however it is one of the smaller Princess ships (some take 3,000 passengers). Our previous Alaskan Cruise was on the Sun Princess which is a twin to the Dawn Princess, and although the ship was fine, I think I would have rather tried a different ship, just to try a different ship. :^)

Here is the itinerary:

Day	Port	Arrival	Departure
1	San Francisco, California	12:00 AM	4:00 PM
2	At Sea
3	At Sea
4	Ketchikan, Alaska	7:00 AM	3:00 PM
5	Juneau, Alaska	8:00 AM	10:00 PM
6	Skagway, Alaska	7:00 AM	8:30 PM
7	Tracy Arm Fjord, Alaska (Scenic Cruising)	5:00 AM	10:00 AM
8	At Sea
9	Victoria, British Columbia	6:00 AM	2:00 PM
10	At Sea
11	San Francisco, California	7:00 AM

I'll try and add some photos when I can wrestle the camera away from my wife. :^)

It was a great cruise, had lots of fun. Next time we will probably try to visit some new ports in Alaska.

-kto

Posted at 02:36PM Jul 08, 2008 by kto in General  |  Comments[0]

Removing duplicate PATH entries

Seemed like everywhere people just kept adding things to PATH without regards to whether it was already in PATH. I don't suspect that long PATH entries are a performance problem in Linux and Solaris, but Windows??? I don't pretend to completely understand all the places in a Windows system where PATH is processed and repeatedly scanned and the directories repeatedly probed, but it seemed like an easy thing to fix for all platforms, ... or so I thought. So I investigated how I could remove duplicate entries in the PATH variable using some kind of shell commands, I ended up with an awk command that worked pretty well.

Below I added newlines in the single quote argument to awk, you may need to mush those lines together into one long line, I broke it up below so it was easier to read.

# Given a PATH like string and a separator, remove duplicate entries
removeDups() # string sep
{
  if [ "${osname}" = "windows" ] ; then
    printf "%s\n" "$1" | \
      sed -e 's@\\@/@g' | \
      ${AWK} -F"$2" \
       '{ \
          a[toupper($1)]; \
          printf "%s",$1; \
          for(i=2;i<=NF;i++){ \
            if(!(toupper($i) in a)){ \
              a[toupper($i)]; \
              printf "%s%s",FS,$i; \
            } \
          }; \
          printf "\n"; \
        }'
  else
    printf "%s\n" "$1" | \
      ${AWK} -F"$2" \
       '{ \
          a[$1]; \
          printf "%s",$1; \
          for(i=2;i<=NF;i++){ \
            if(!($i in a)){ \
              a[$i]; \
              printf "%s%s",FS,$i; \
            } \
          }; \
          printf "\n"; \
        }'
  fi
}

# OS name: Linux or SunOS, pot luck on Windows
osname="`uname -s`"
sep=":"
if [ `printf "%s\n" "${PATH}"` != "" ] ; then
  osname="windows"
  sep=";"
fi
if [ "`echo ${osname} | grep -i CYGWIN`" != "" ] ; then
  osname="windows"
fi

# Need particular AWK
AWK=awk
if [ "${osname}" = "SunOS" ] ; then
  AWK=nawk
fi

# Get new path setting
newpath=`removeDups "${PATH}" "${sep}"`

# Redefine your PATH setting
if [ "${PATH}" != "${newpath}" ] ; then
  echo "# Resetting PATH to remove duplicates"
  PATH_ORIGINAL="${PATH}"
  export PATH_ORIGINAL
  PATH="${newpath}"
  export PATH
fi

Maybe someone else can get something out of this. Or suggest an even better way. ;^)

-kto

Posted at 11:45AM Jun 12, 2008 by kto in General  |  Comments[5]

A Complicated Slide?

This Dilbert comic strip reminded me of a drawing I did a while back to try and explain the critical nature of our bug tracking system, and how all the various systems we have and use connect back to the bug database. Granted, right now the JDK team is using an internal bug tracking system (that will change someday), and the visibility is limited to what you can see via http://bugs.sun.com/.

The complicated drawing below is mine alone and is not an official JDK drawing and is undoubtedly not 100% correct, it is just my attempt to convey some of the connections to our bug tracking system:

And a few terms to define the words used:

• Builds Archives: Promoted builds, nightly builds, and the JPRT build archive.
• Alacrity: A system and set of machines used to run our more critical performance benchmarks in a controlled environment and report on performance results on JDK builds.
• GTEE: The formal test system that verifies our JDK builds pass all our tests (all our many different kinds of tests).
• JPRT: My balliwick of late. JPRT is a set of systems and machines (multiple instances of JPRT) that provide build and basic testing on all platforms and both VMs (client & server). JPRT stands for Java Programmer Rat Trap... I'm kidding, I'm not sure what the name stands for anymore. There was an old HotSpot PRT system and that stood for Performance Reliability Testing, and this newer system works with all the JDK repositories, so I named it JPRT. Anyway, it is a system that developers and integrators can submit repositories or source trees for builds on all 8 of our standard platforms (Solaris SPARC, Solaris X86, Windows, and Linux, both 32 and 64 bit). The builds are archived and JPRT can actually update the bugs and push changesets when the job is successful. This is a system we need to somehow make available externally but we don't know exactly how just yet.
• Testbase: Just represents all the various testbases we have for testing the OpenJDK. Some of these tests are actually in the JDK repositories. These include the JDK TCK (JCK).
• CR Bug Database: Our internal bug database, some of which is visible via the bugs.sun.com site. The whole point of making this drawing (quite a while back) was to demonstrate how the CR number is used in so many places and the bug tracking system is at the center of everything. Someday, this may be bugzilla for the OpenJDK.
• CCC: The committee that reviews any change to a user visible API in the JDK. All developers making changes that involving any change to a user visible API or command line option or file format, must file a CCC request and get approval from CCC before pushing their change into a public repository. This is currently an internal process but will be externalized at some point.
• Integrators: Those blessed souls that deal with the merges between teams and integrate all a teams changes into the master jdk7 repositories.
• Planning: The planning process will always refer to the CRs as part of the planning process.
• Source Repositories: The Mercurial repositories or TeamWare workspaces for some of the older releases.
• WebRevs: An old, extremely valuable, and historic way that Sun Engineers have used to do code reviews. All changes to the JDK are required to be reviewed by at least one other engineer, more than one at critical times of a release. See Jesse's webrev blog on the version of the webrev script we use in the JDK area.

I hope this helps someone and doesn't really confuse people. And no, the rocket ship is not blasting the webrevs. :^)

-kto

Posted at 10:57AM Jun 11, 2008 by kto in Java  |  Comments[2]

CSN Concert at Wente Vineyards

Last night drove down the street to Wente Vineyards and saw Crosby, Stills & Nash. They have a long history that sometimes includes Neil Young.

CSNY, gray hair and all! (We only saw CSN)	Teach Your Children, back when they were younger. :^)

Their harmony was still there, a great concert. Although a bit expensive at Wente, $207 per person, but that included a meal and we were probably 100 feet from the stage or so. Weather was fantastic too.

-kto

Posted at 03:42PM Jun 10, 2008 by kto in General  | 

OpenJDK Bug States

Until the OpenJDK project converts to Bugzilla, I thought some more information about how our internal bug tracking system works might help people watching the current situation.

The OpenJDK change requests (CRs) or bugs are currently visible at bugs.sun.com.

There are 11 states for a CR and the normal change of state is the following:

1. Dispatched: The initial state for a new CR.
2. Incomplete: Something critical is missing from the CR.
3. Accepted: CR was accepted, the first step in being investigated and fixed.
4. Defer: Work on this CR is on hold.
5. Cause Known: A basic understanding of the problem is known.
6. Fix Understood: A basic fix is now understood.
7. Fix in Progress: The assigned engineer is actively working on the fix or getting it integrated.
8. Fix Available: A changeset or the actual fix has been made available in a team area.
9. Fix Failed: Something went wrong with the fix.
10. Fix Delivered: The changeset or fix has been integrated into the master area and will show up in the next build promotion.
11. Closed: There are many reasons why a bug will be in the closed state. It might be verified as fixed, closed as a duplicate, closed as 'not a bug' or closed as 'will not fix'.

The states 1-9 are considered "unresolved", so until a bug becomes "10 - Fix Delivered" or "11 - Closed", it is still considered unresolved. So being "unresolved" may mean that a fix is available, just not in a position to be made part of a formal build promotion.

The bugs.sun.com interface is read-only and somewhat crude, but you can query the bug information, for example the Unresolved JDK Build Subcategory Bugs and RFEs. Hope this helps explain things.

-kto

Posted at 12:25PM May 11, 2008 by kto in Java  |  Comments[4]

ZZ Top

Last night we drove to Dixon, California and saw the legendary hard rock band ZZ Top.

ZZ Top as they looked in the 1980's and still look this way in 2008!	Sharp Dressed Man Music Video (One of my favorites)

-kto

Posted at 01:01PM May 10, 2008 by kto in General  |  Comments[1]

Sun Studio Compilers on LINUX?

With the magic of Mercurial, you can see changesets, like this one: http://hg.openjdk.java.net/jdk7/jdk7/hotspot/rev/485d403e94e1. Which Serguei Spitsyn integrated recently.

But wait, what does this changeset actually mean? Sun Studio on Linux? Does that make sense? YES! It does and it's true. Mind you, it's just Hotspot that can be built with the Sun Studio compilers on Linux right now, but it's an important piece, the Hotspot C++ code is not a trivial pile of code to compile and optimize correctly. My hat is off to the Sun Studio team in making this all happen. What should be interesting now is how well the rest of the tools work like dbx and the analyzer/collector.

More can be read about the Sun Studio Linux Compilers at the Sun Studio Site.

Humm, I guess I'm now on the hook to see if the rest of the OpenJDK can be setup to build with Sun Studio on Linux.... Back to work...

-kto

P.S. No, we are not abandoning gcc/g++, just providing choices on how the OpenJDK can be built.

Posted at 12:47PM May 07, 2008 by kto in Java  |  Comments[7]

InfoQ Article on Git, Mercurial, and Bzr

A while back Sébastien Auvray asked me some questions about the OpenJDK Mercurial conversion. His article was recently published at http://www.infoq.com/articles/dvcs-guide.

Has some interesting information and stats about Git, Mercurial, and Bzr.

-kto

Posted at 12:18PM May 07, 2008 by kto in Java  | 

JDK Build Readme Collection

A collection of links to various JDK Build readme files.

• OpenJDK Build Home Page
• OpenJDK 7 Build README
OpenJDK 6 Build README (inside the source bundle from the download site, file README-builds.html)
• OpenJDK NetBeans Build README
• JDK 7 JRL Build README
• JDK 7 Build README
• JDK 6 Build README
• JDK 5 SCSL Build README

Posted at 04:07PM Apr 30, 2008 by kto in Java  |