Joseph D. Darcy's Sun Weblog

Monday October 19, 2009

OpenJDK 6: b17 regression test results

Running with the usual jtreg flags, -a -ignore:quiet always and -s for the langtools area, the basic regression test results on Linux for OpenJDK 6 build 17 are:

• HotSpot, 24 tests passed.

• Langtools, 1354 tests passed.

• JDK, 3,118 tests pass and 26 tests fail.

With the sync of HotSpot 14 into OpenJDK 6, more tests were added:

0: b16-hotspot/summary.txt  pass: 3
1: b17-hotspot/summary.txt  pass: 24

0      1      Test
---    pass   compiler/6646019/Test.java
---    pass   compiler/6646020/Tester.java
---    pass   compiler/6661247/Test.java
---    pass   compiler/6663848/Tester.java
---    pass   compiler/6689060/Test.java
---    pass   compiler/6695810/Test.java
---    pass   compiler/6700047/Test6700047.java
---    pass   compiler/6711100/Test.java
---    pass   compiler/6712835/Test6712835.java
---    pass   compiler/6714694/Tester.java
---    pass   compiler/6716441/Tester.java
---    pass   compiler/6726999/Test.java
---    pass   compiler/6741738/Tester.java
---    pass   compiler/6756768/Test6756768.java
---    pass   compiler/6756768/Test6756768_2.java
---    pass   compiler/6757316/Test6757316.java
---    pass   compiler/6758234/Test6758234.java
---    pass   compiler/6775880/Test.java
---    pass   compiler/6795362/Test6795362.java
---    pass   compiler/6805724/Test6805724.java
---    pass   runtime/6819213/TestBootNativeLibraryPath.java

21 differences

In langtools all the tests continue to pass; a few tests were rearranged and some more tests were added:

0: b16-langtools/summary.txt  pass: 1,352
1: b17-langtools/summary.txt  pass: 1,354

0      1      Test
---    pass   tools/javac/T6558476.java
---    pass   tools/javac/T6882235.java
---    pass   tools/javac/annotations/pos/TrailingComma.java
pass   ---    tools/javac/api/6440333/T6440333.java
pass   ---    tools/javac/api/Sibling.java
pass   ---    tools/javac/innerClassFile/Driver.java
---    pass   tools/javac/innerClassFile/Driver.sh
pass   ---    tools/javac/links/T.java
---    pass   tools/javac/links/links.sh
---    pass   tools/javac/processing/model/util/elements/TestGetConstantExpression.java

10 differences

And in jdk, a few dozen new tests were added in b17 and the existing tests have generally consistent results:

0: b16-jdk/summary.txt  pass: 3,085; fail: 29; error: 4
1: b17-jdk/summary.txt  pass: 3,118; fail: 26

0      1      Test
error  pass   com/sun/jdi/DoubleAgentTest.java
---    pass   com/sun/jndi/dns/CheckAccess.java
---    pass   com/sun/org/apache/xml/internal/security/TruncateHMAC.java
---    pass   com/sun/security/sasl/util/CheckAccess.java
fail   pass   java/awt/Focus/FrameMinimizeTest/FrameMinimizeTest.java
fail   pass   java/awt/Graphics/DrawImageBG/SystemBgColorTest.java
error  ---    java/awt/PrintJob/EdgeTest/EdgeTest.java
error  pass   java/awt/PrintJob/MultipleEnd/MultipleEnd.java
fail   pass   java/awt/print/PrinterJob/ExceptionTest.java
---    pass   java/beans/Introspector/Test6660539.java
---    pass   java/beans/XMLEncoder/6777487/TestBox.java
---    pass   java/beans/XMLEncoder/6777487/TestCheckedCollection.java
---    pass   java/beans/XMLEncoder/6777487/TestCheckedList.java
---    pass   java/beans/XMLEncoder/6777487/TestCheckedMap.java
---    pass   java/beans/XMLEncoder/6777487/TestCheckedRandomAccessList.java
---    pass   java/beans/XMLEncoder/6777487/TestCheckedSet.java
---    pass   java/beans/XMLEncoder/6777487/TestCheckedSortedMap.java
---    pass   java/beans/XMLEncoder/6777487/TestCheckedSortedSet.java
---    pass   java/beans/XMLEncoder/6777487/TestEnumMap.java
---    pass   java/beans/XMLEncoder/6777487/TestEnumSet.java
---    pass   java/lang/Double/ToString.java
---    pass   java/nio/channels/Selector/RegAfterPreClose.java
fail   pass   java/rmi/transport/pinLastArguments/PinLastArguments.java
---    pass   java/security/AccessControlContext/FailureDebugOption.java
---    pass   java/util/SimpleTimeZone/EndOfDay.java
---    pass   java/util/TimeZone/ListTimeZones.java
---    pass   javax/sound/midi/Gervill/SoftChannel/NoteOverFlowTest2.java
---    pass   javax/sound/midi/Gervill/SoftReceiver/GetMidiDevice.java
---    pass   javax/sound/midi/Gervill/SoftSynthesizer/TestPreciseTimestampRendering.java
---    fail   javax/swing/JTextArea/Test6593649.java
---    pass   javax/swing/plaf/synth/Test6660049.java
---    pass   javax/swing/text/LayoutQueue/Test6588003.java
---    pass   sun/pisces/JoinMiterTest.java
error  pass   sun/security/ssl/javax/net/ssl/NewAPIs/SessionTimeOutTests.java
---    pass   tools/pack200/MemoryAllocatorTest.java

35 differences

OpenJDK 6: b17 Source Bundle Published

On October 16, the source bundle for OpenJDK 6 b17 was published.

Major changes in this build include the latest round of security fixes and, courtesy of Andrew John Hughes, the syncing of HotSpot 14 into the OpenJDK 6 HotSpot repository.

A detailed list of all the changes is also available.

Saturday May 02, 2009

OpenJDK 6: Regression test results for b16, take 2

The corrected source bundle for b16 has slightly different regression test results than the original one. The langtools and hotspot results are the same, but the jdk area has a different difference from b14/b15:

0: b14-jdk/summary.txt   pass: 3,077; fail: 26; error: 3
1: b16a-jdk/summary.txt  pass: 3,085; fail: 29; error: 4

0      1      Test
pass   error  com/sun/jdi/DoubleAgentTest.java
---    pass   java/awt/FontClass/CreateFont/BigFont.java
---    pass   java/awt/FontClass/CreateFont/DeleteFont.sh
---    pass   java/awt/FontClass/CreateFont/fileaccess/FontFile.java
error  pass   java/awt/FullScreen/UninitializedDisplayModeChangeTest/
              UninitializedDisplayModeChangeTest.java
pass   fail   java/awt/print/PrinterJob/ExceptionTest.java
---    pass   java/beans/PropertyEditor/TestEnumSubclass.java
---    pass   java/beans/PropertyEditor/TestEnumSubclassJava.java
---    pass   java/beans/PropertyEditor/TestEnumSubclassNull.java
---    pass   java/beans/PropertyEditor/TestEnumSubclassValue.java
---    pass   java/io/readBytes/MemoryLeak.java
pass   fail   java/rmi/transport/pinLastArguments/PinLastArguments.java
---    pass   java/util/concurrent/Semaphore/RacingReleases.java
---    pass   javax/print/attribute/MediaMappingsTest.java
---    pass   javax/sound/midi/Gervill/SoftChannel/NoteOverFlowTest.java
---    pass   javax/sound/midi/Gervill/SoftFilter/TestProcessAudio.java
---    pass   javax/sound/midi/Gervill/SoftLowFrequencyOscillator/TestProcessControlLogic.java
fail   pass   javax/swing/JColorChooser/Test6541987.java
pass   error  sun/security/ssl/javax/net/ssl/NewAPIs/SessionTimeOutTests.java
pass   fail   sun/tools/jhat/HatHeapDump1Test.java
pass   fail   sun/tools/native2ascii/NativeErrors.java
pass   ---    tools/pack200/MemoryAllocatorTest.java

22 differences

Comparing the initial and corrected builds of b16:

0: b16-jdk/summary.txt   pass: 3,079; fail: 27; error: 4
1: b16a-jdk/summary.txt  pass: 3,085; fail: 29; error: 4

0      1      Test
pass   fail   java/awt/Focus/FrameMinimizeTest/FrameMinimizeTest.java
---    pass   java/beans/PropertyEditor/TestEnumSubclass.java
---    pass   java/beans/PropertyEditor/TestEnumSubclassJava.java
---    pass   java/beans/PropertyEditor/TestEnumSubclassNull.java
---    pass   java/beans/PropertyEditor/TestEnumSubclassValue.java
pass   fail   java/rmi/transport/pinLastArguments/PinLastArguments.java
---    pass   java/util/concurrent/Semaphore/RacingReleases.java
---    pass   javax/sound/midi/Gervill/SoftChannel/NoteOverFlowTest.java
---    pass   javax/sound/midi/Gervill/SoftFilter/TestProcessAudio.java
---    pass   javax/sound/midi/Gervill/SoftLowFrequencyOscillator/TestProcessControlLogic.java

10 differences

Thursday April 16, 2009

OpenJDK 6: Regression test results for b16

Update: Regression test results for the corrected b16 source bundle are now available.

Running with the usual jtreg flags, -a -ignore:quiet always and -s for the langtools area, the basic regression test results on Linux for OpenJDK 6 build 16 are:

• HotSpot, 3 tests passed.

• Langtools, 1,352 tests passed.

• JDK, 3,077 tests pass, 26 tests fail, 3 tests have errors.

In this build, many of the bugs fixed were for security issues or ports of fixes already in JDK 7. The HotSpot test results were stable:

0: b14-hotspot/summary.txt  pass: 3
1: b16-hotspot/summary.txt  pass: 3

No differences

In langtools all the tests continue to pass and a new test was added:

0: b14-langtools/summary.txt  pass: 1,351
1: b16-langtools/summary.txt  pass: 1,352

0      1      Test
---    pass   tools/javac/processing/6512707/T6512707.java

1 differences

And in jdk, a few new tests were added in b16 and the existing tests have generally consistent results:

0: b14-jdk/summary.txt  pass: 3,077; fail: 26; error: 3
1: b16-jdk/summary.txt  pass: 3,079; fail: 27; error: 4

0      1      Test
pass   error  com/sun/jdi/DoubleAgentTest.java
fail   pass   java/awt/Focus/FrameMinimizeTest/FrameMinimizeTest.java
---    pass   java/awt/FontClass/CreateFont/BigFont.java
---    pass   java/awt/FontClass/CreateFont/DeleteFont.sh
---    pass   java/awt/FontClass/CreateFont/fileaccess/FontFile.java
error  pass   java/awt/FullScreen/UninitializedDisplayModeChangeTest
              UninitializedDisplayModeChangeTest.java
pass   fail   java/awt/print/PrinterJob/ExceptionTest.java
---    pass   java/io/readBytes/MemoryLeak.java
---    pass   javax/print/attribute/MediaMappingsTest.java
fail   pass   javax/swing/JColorChooser/Test6541987.java
pass   error  sun/security/ssl/javax/net/ssl/NewAPIs/SessionTimeOutTests.java
pass   fail   sun/tools/jhat/HatHeapDump1Test.java
pass   fail   sun/tools/native2ascii/NativeErrors.java
pass   ---    tools/pack200/MemoryAllocatorTest.java

14 differences

Wednesday April 15, 2009

OpenJDK 6: b16 Source Bundle Published

Update: Due to a misunderstanding, some of the expected fixes were missing from the original b16 source bundle. The corrected bundle has subsequently been posted.

On April 15, the source bundle for OpenJDK 6 b16 was published.

The previous source bundle was for b14. There was no distinct b15 per se; b15 was used to mark the transition from teamware to Mercurial. The b16 build is the first to be done purely in Mercurial.

There were 45 bug fixes in this build ranging from security fixes to sound updates; OpenJDK 6 b16 has all relevant security fixes from the recent JDK synchronized security release. A few fixes of note, during a build importing the binary plugs now defaults to false (6781572) and several fixes were applied that should render a few IcedTea patches unnecessary:

Friday February 13, 2009

FOSDEM 2009: OpenJDK 6 and Project Coin

I was pleased to attend and speak at my first FOSDEM conference this year. The classroom setting of the Free Java Developer Room was certainly different than the cavernous session halls of JavaOne! My talks were on OpenJDK 6 and Project Coin:

• OpenJDK 6: Where are we now? Where are we going?

• Project Coin: Small Language Changes for JDK 7

Mark spoke on modularity and overall plans for JDK 7 and Alex spoke on progress towards a universal VM.

Another enjoyable aspect of FOSDEM for me was meeting in person a number of people I'd corresponded with over email who had contributed to OpenJDK 6, including Karl Helgason of Gervill fame and Andrew Haley and other IcedTea engineers from Red Hat.

Monday February 09, 2009

OpenJDK 6: Mercurial Repositories Available

As announced by Kelly, after a successful trial the final read/write Mercurial repositories for OpenJDK 6 are now available. Thanks to Kelly for putting in the extra effort to create change sets approximating history information of the pre-Mercurial phase of OpenJDK 6! The build tags correspond to the state of the source code shipped with the indicated build.

Martin has already made a series of external-to-Sun pushes and I'm looking forward to more people contributing to the code base going forward.

Friday January 16, 2009

OpenJDK and the new plugin

One of the most exciting new features in 6u10 was the new Java Plug-in.

I'm happy to announce that within the next few months as part of OpenJDK, we are Sun are committed to open sourcing our Java Web Start implementation and the new plug-in implementation for NPAPI capable browsers; those browsers including Firefox 3, amongst others.

Monday December 08, 2008

OpenJDK 6: Trial Mercurial Repositories Available

Thanks to Kelly, trial Mercurial repositories for OpenJDK 6 are now available for evaluation. These trial repositories will be available read-only for about a week to find any problems before creating the final live repositories; for details, see Kelly's email to the jdk6-dev alias.

Thursday December 04, 2008

Build architectures and managing dependencies

Since the OpenJDK source was released, there have been various discussions about aspects of the build architecture, including how dependencies on third parties libraries should be managed. For example, on a Linux system should the JDK use its own libzip or the libzip that comes with the distribution? I think the appropriate answers to these and related question hinge on whether the final deliverable from the OpenJDK project is viewed as the source code itself or a binary built from the source.

Traditionally before OpenJDK, the end result of the Sun's JDK project that most people used were the JDK and JRE binaries. These binaries were meant to be universal in the sense of being usable on a given processor family across a version range of operating systems. For example, there was a single windows x86 binary for use on, say, Windows NT, Windows XP, etc., a single Solaris SPARC binary for use across Solaris 8 through Solaris 10, and effectively a single Linux x86 binary for use across different Linux distributions.

This "single binary" model drives decisions about what platform to produce the binary on, generally an older release, and what assumptions can be made about available system resources, generally rather weak ones. With a single binary deliverable, since fewer environment resources can be relied on, making the JDK build self-contained is necessary for it to be reliable in a wide variety of environments. With this delivery architecture, there is some justification to, for example, including a copy a library like libzip in the JDK build rather than relying on a system library, even though there are increased maintenance costs.

However, when an OpenJDK/IcedTea binary is being built on a particular Linux distribution for use only on that distribution, the constraints are different. If the build is being done by the OS vendor, the vendor controls the OS contents and knows whether or not system libraries like libzip are reliable and kept up to date. Since stronger assumptions can be made about the host environment, weaker conditions need to be fulfilled by the JDK source tree. For an OS vendor, relying on a single copy of native libraries for the OS and the JDK is preferable to building (and maintaining) multiple copies.

Going forward, I'd expect the JDK build to evolve to better accommodate options to use host platform resources. Perhaps modules systems in the future can help manage such dependencies more transparently.

Wednesday December 03, 2008

OpenJDK 6 b13 and JCK testing

The JCK tests probe the conformance of a Java platform implementation to a specification. For example, JCK 6b is the current test suite to determine conformance to the Java SE 6 spec. Official claims of conformance require not only passing the complete set of relevant tests, but also meeting the other requirements spelled out in the JCK user's guide.

Regarding the JCK, conformance is measured with respect to a binary rather than to a source base directly, which is sensible since a Java platform implementation will typically rely on and be affected by the properties of the host environment, including the OS, the C compiler, and the processor.

Previously Red Hat announced that using OpenJDK sources augmented with IcedTea patches, an OpenJDK binary on Fedora 9 passed the JCK and met the other conformance requirements.

Amongst other changes, after incorporating community-developed patches (notably 6748251 in b13) and following the OpenJDK 6 build instructions, inside Sun a binary resulting from building the unmodified OpenJDK 6 b13 sources on Redhat Enterprise AS 2.1 with gcc 2.95 (the official Linux build platform for Sun's 6 update releases) passed all the JCK 6b tests when run on Fedora Core 8 x86. That binary also meets all the other JCK requirements.

OpenJDK 6 binaries built from b13 (and later) sources on and for different host environments are now more likely to share those favorable conformance properties, but testing would be necessary to verify conformance status and to make any formal statements.

More information

• Free and Open Source Java FAQ on TCKs

• OpenJDK Conformance Group

OpenJDK 6: Some regression test results for b14

Running with the usual jtreg flags, -a -ignore:quiet always and -s for the langtools area, the basic regression test results on Linux for OpenJDK 6 build 14 are:

• HotSpot, 3 tests passed.

• Langtools, 1,351 tests passed.

• JDK, 3,077 tests pass, 26 tests fail, 3 tests have errors.

In this build, we upgraded from a HotSpot 10 base to HotSpot 11. HotSpot 11 is also being used in the 6u10 release. The set of included tests for the HotSpot versions differ:

0: b13-hotspot/summary.txt  pass: 5
1: b14-hotspot/summary.txt  pass: 3

0      1      Test
pass   ---    compiler/6547163/Test.java
pass   ---    compiler/6563987/Test.java
pass   ---    compiler/6571539/Test.java
pass   ---    compiler/6595044/Main.java
---    pass   compiler/6663621/IVTest.java
---    pass   compiler/6724218/Test.java

6 differences

In langtools all the tests continue to pass:

0: ./b13-langtools/summary.txt  pass: 1,351
1: ./b14-langtools/summary.txt  pass: 1,351

No differences

And in jdk, a few new tests were added in b14 and the existing tests have generally consistent results:

0: b13-jdk/summary.txt  pass: 3,072; fail: 23; error: 3
1: b14-jdk/summary.txt  pass: 3,077; fail: 26; error: 3

0      1      Test
---    fail   com/sun/org/apache/xml/internal/ws/server/Test.java
pass   fail   java/awt/TextArea/UsingWithMouse/SelectionAutoscrollTest.html
---    pass   java/awt/image/ConvolveOp/EdgeNoOpCrash.java
---    pass   javax/management/monitor/DerivedGaugeMonitorTest.java
pass   fail   javax/swing/JColorChooser/Test6541987.java
---    pass   javax/swing/JFileChooser/6484091/bug6484091.java
---    pass   sun/management/jmxremote/LocalRMIServerSocketFactoryTest.java
---    pass   sun/nio/cs/TestUTF8.java
---    pass   sun/security/ssl/com/sun/net/ssl/internal/ssl/SSLEngineImpl/EmptyExtensionData.java
---    pass   tools/pack200/MemoryAllocatorTest.java

10 differences

OpenJDK 6: Sources for b14 published

On December 3, the OpenJDK 6 b14 source bundle was published. Changes of note in this build:

• All relevant security fixes in the current round of security updates (6592792, 6484091, 6766136, 6755943, 6751322, 4486841, 6721753, 6734167, 6733336, 6726779, 6588160, 6733959, 6497740).

• Upgrading the HotSpot in OpenJDK 6 from HotSpot 10 to HotSpot 11 (6775772).

• Correcting legal notices (6772068, 6773363, 6706382).

• Fixing build documentation and other miscellaneous fixes (6775844, 6775851, 6683213, 6774170, 6741058, 6728126, 6745052).

This will be the last teamware based source drop; all future source drops will be based on the forthcoming public OpenJDK 6 Mercurial repositories.

Friday November 07, 2008

OpenJDK 6: Some regression test results for b13

For regression testing of OpenJDK 6, the big news in b13 is due to work by Jon on multiple fronts; by getting out a new version of jtreg and developing a number of test fixes, all the langtools regression tests now pass when run in "samevm" mode (jtreg -s ...). As the name implies, in same vm mode a test is run in the same jvm as the previous test as opposed to starting a fresh jvm for each test, giving much speedier test results.

The base jtreg flags for these test runs were -a -ignore:quiet with -s just used for the langtools tests. On a per workspace basis, the results on Linux for OpenJDK 6 build 13 are:

• HotSpot, 5 tests pass.

• Langtools, 1,351 tests pass.

• JDK, 3,072 test pass, 23 tests fail, 3 tests have errors.

Compared to the regression test results for b12, in b13 HotSpot is unchanged:

0: ./b12-hotspot/summary.txt  pass: 5
1: ./b13-hotspot/summary.txt  pass: 5

No differences

The langtools tests now all pass:

0: ./b12-langtools/summary.txt  pass: 1,346; fail: 2
1: ./b13-langtools/summary.txt  pass: 1,351

0      1      Test
pass   ---    tools/javac/6176978/T6176978.java
---    pass   tools/javac/T6725036.java
---    pass   tools/javac/T6759996.java
---    pass   tools/javac/VersionOpt.java
---    pass   tools/javac/generics/T6657499.java
fail   pass   tools/javac/processing/model/testgetallmembers/Main.java
fail   ---    tools/javac/versionOpt.sh
---    pass   tools/javadoc/6176978/T6176978.java

8 differences

And in jdk, several dozen new tests were added and some previously failing tests pass:

0: ./b12-jdk/summary.txt  pass: 2,985; fail: 36; error: 4
1: ./b13-jdk/summary.txt  pass: 3,072; fail: 23; error: 3

0      1      Test
fail   pass   com/sun/jdi/JdbReadTwiceTest.sh
---    pass   com/sun/org/apache/xml/internal/security/exceptions/LocaleTest.java
---    pass   com/sun/org/apache/xml/internal/security/transforms/ClassLoaderTest.java
---    pass   com/sun/tools/extcheck/TestExtcheckArgs.java
fail   pass   java/awt/Focus/DeiconifiedFrameLoosesFocus/DeiconifiedFrameLoosesFocus.html
pass   fail   java/awt/Focus/FrameMinimizeTest/FrameMinimizeTest.java
fail   pass   java/awt/TextArea/UsingWithMouse/SelectionAutoscrollTest.html
fail   pass   java/io/File/Basic.java
fail   pass   java/io/File/SetAccess.java
fail   pass   java/io/File/SetReadOnly.java
---    pass   java/lang/Boolean/Factory.java
---    pass   java/lang/Boolean/GetBoolean.java
---    pass   java/lang/Boolean/MakeBooleanComparable.java
---    pass   java/lang/Boolean/ParseBoolean.java
---    pass   java/lang/Byte/Decode.java
---    pass   java/lang/Double/BitwiseConversion.java
---    pass   java/lang/Double/Constants.java
---    pass   java/lang/Double/Extrema.java
---    pass   java/lang/Double/NaNInfinityParsing.java
---    pass   java/lang/Double/ParseDouble.java
---    pass   java/lang/Double/ParseHexFloatingPoint.java
---    pass   java/lang/Double/ToHexString.java
---    pass   java/lang/Float/BitwiseConversion.java
---    pass   java/lang/Float/Constants.java
---    pass   java/lang/Float/Extrema.java
---    pass   java/lang/Float/NaNInfinityParsing.java
---    pass   java/lang/Float/ParseFloat.java
---    pass   java/lang/Integer/BitTwiddle.java
---    pass   java/lang/Integer/Decode.java
---    pass   java/lang/Integer/GetInteger.java

---    pass   java/lang/Integer/ParsingTest.java
---    pass   java/lang/Long/BitTwiddle.java
---    pass   java/lang/Long/Decode.java
---    pass   java/lang/Long/GetLong.java
---    pass   java/lang/Long/ParsingTest.java
---    pass   java/lang/Math/AbsPositiveZero.java
---    pass   java/lang/Math/Atan2Tests.java
---    pass   java/lang/Math/CubeRootTests.java
---    pass   java/lang/Math/Expm1Tests.java
---    pass   java/lang/Math/HyperbolicTests.java
---    pass   java/lang/Math/HypotTests.java
---    pass   java/lang/Math/IeeeRecommendedTests.java
---    pass   java/lang/Math/Log10Tests.java
---    pass   java/lang/Math/Log1pTests.java
---    pass   java/lang/Math/MinMax.java
---    pass   java/lang/Math/PowTests.java
---    pass   java/lang/Math/Rint.java
---    pass   java/lang/Math/TanTests.java
---    pass   java/lang/Short/ByteSwap.java
---    pass   java/lang/Short/Decode.java
---    pass   java/lang/StrictMath/CubeRootTests.java
---    pass   java/lang/StrictMath/Expm1Tests.java
---    pass   java/lang/StrictMath/HyperbolicTests.java
---    pass   java/lang/StrictMath/HypotTests.java
---    pass   java/lang/StrictMath/Log10Tests.java
---    pass   java/lang/StrictMath/Log1pTests.java
---    pass   java/lang/ToString.java
---    pass   java/lang/annotations/AnnotationTypeMismatchException/FoundType.java
---    pass   java/lang/annotations/Missing/MissingTest.java
---    pass   java/lang/annotations/ParameterAnnotations.java
---    pass   java/lang/annotations/RecursiveAnnotation.java
---    pass   java/lang/annotations/UnitTest.java
---    pass   java/lang/annotations/loaderLeak/Main.java
---    pass   java/lang/annotations/package-info.java
fail   pass   java/net/ipv6tests/TcpTest.java
error  pass   java/nio/channels/SocketChannel/AdaptSocket.java
---    pass   java/rmi/activation/Activatable/checkActivateRef/CheckActivateRef.java
---    pass   javax/sound/midi/Gervill/EmergencySoundbank/TestCreateSoundbank.java
---    pass   javax/sound/midi/Gervill/ModelByteBuffer/RandomFileInputStream/Available.java
---    pass   javax/sound/midi/Gervill/ModelByteBuffer/RandomFileInputStream/Close.java
---    pass   javax/sound/midi/Gervill/ModelByteBuffer/RandomFileInputStream/MarkReset.java
---    pass   javax/sound/midi/Gervill/ModelByteBuffer/RandomFileInputStream/MarkSupported.java
---    pass   javax/sound/midi/Gervill/ModelByteBuffer/RandomFileInputStream/Read.java
---    pass   javax/sound/midi/Gervill/ModelByteBuffer/RandomFileInputStream/ReadByte.java
---    pass   javax/sound/midi/Gervill/ModelByteBuffer/RandomFileInputStream/ReadByteIntInt.java
---    pass   javax/sound/midi/Gervill/ModelByteBuffer/RandomFileInputStream/Skip.java
---    pass   javax/xml/crypto/dsig/GenerationTests.java
---    pass   javax/xml/crypto/dsig/ValidationTests.java
---    pass   javax/xml/crypto/dsig/keyinfo/KeyInfo/Marshal.java
---    pass   sun/net/idn/PunycodeTest.java
---    pass   sun/net/idn/TestStringPrep.java
---    pass   sun/security/pkcs11/ec/TestCurves.java
fail   pass   sun/tools/jps/jps-Defaults.sh
fail   pass   sun/tools/jps/jps-l_1.sh
fail   pass   sun/tools/jstatd/jstatdDefaults.sh
fail   pass   sun/tools/jstatd/jstatdExternalRegistry.sh
fail   pass   sun/tools/jstatd/jstatdPort.sh
fail   pass   sun/tools/jstatd/jstatdServerName.sh
fail   pass   sun/tools/native2ascii/NativeErrors.java

89 differences

OpenJDK 6: Sources for b13 published

On November 7, the OpenJDK 6 b13 source bundle was published. Changes of note in this build:

• In the langtools area, besides a few miscellaneous bug fixes (6760834, 6725036, 6657499), all the langtools regression tests now pass out of the box (6728697, 6707027) and if using the most recent version of jtreg, the langtools regression tests can be run in the much faster "same vm" mode of jtreg, enabled with the -s option (6749967, 6748541, 6748546, 6748601, 6759775, 6759795, 6759796, 6759996, 6760805, 6760930).

• Gervill update, including applying a patch from IcedTea (6758986, 6748247, 6748251).

• Publishing a few dozen additional existing regression tests as open source (6601457, 6759433, 6740185).

• JMX and monitoring fixes (6651382, 6616825, 6756202, 6754672).

• Man page updates (6757036, 6392810, 6504867, 6326773).

• Assorted other fixes (6746055, 6621697, 6756569, 6356642, 6761678).

Kelly has made good progress on the Mercurial repository, but some more work remain to be done. I'd expect to have trial repositories available within a week or two.

Tuesday November 04, 2008

OpenJDK 6 Genealogy

To address some of the questions that have come up recently in comments and elsewhere, I've prepared a diagram of OpenJDK 6's genealogy:

The original JDK 6 code base begat two lines of heirs: JDK 7 and the sequence of 6 update releases. The decision to open source the JDK code base came late in the life cycle of the development of JDK 6 so a build of JDK 7 was the first be published as open source. By the time JDK 7 was published as OpenJDK 7, the first post-GA update to JDK 6, 6u1, had already shipped. Work continued in both the OpenJDK 7 and 6 update trains. However, an open source implementation of the Java SE 6 specification was needed as well. After considering the alternatives, OpenJDK 7 build 20 was chosen as the basis of a backward branch to create OpenJDK 6 by removing changes inappropriate for a Java SE 6 implementation. Since then, all three trains, 6 update, OpenJDK 7, and OpenJDK 6, have continued evolving, with certain fixes being ported between the releases.

At some point in the future, the OpenJDK 7 code base will in turn presumably give rise to OpenJDK 7 update releases and OpenJDK 8.

Monday November 03, 2008

OpenJDK 6 and 6u10 features

There have been a number of inquiries about when the various new features of 6u10 would be available in OpenJDK.

OpenJDK 6 build 12 contained ports of bug fixes from a number of 6u10 component areas (corba, jaxp, jaxws, langtools). Most changes from the core jdk component area of 6u10 were not ported. The porting effort that took place of a relatively small number of bugs to a subset of the full OpenJDK code base was still a sizeable effort. The full set of changes made to the core jdk in 6u10 is many times larger with a proportionally larger porting cost. We at Sun do not plan to do a wholesale port of those 6u10 features from the core jdk to OpenJDK 6. However, over the coming months we will be porting those 6u10 features to OpenJDK 7 and we would welcome community assistance in backporting appropriate features from OpenJDK 7 to OpenJDK 6. (These jdk area features in 6u10 are separate from plugin and webstart functionality.)

Thursday October 02, 2008

OpenJDK 6: Logistics of Partial Merge with 6u10

A large fraction of my work for OpenJDK 6 build 12 was porting all of the cumulative fixes in selected areas of the 6u10 code base into OpenJDK 6. Internally, like the forest of Mercurial repositories of JDK 7, the code base of OpenJDK 6 is composed of a set of teamware workspaces for different areas: cobra, hotspot, jaxp, jaxws, jdk, and langtools. Previously, the non-HotSpot code lived in a single "j2se" workspace which was split as part of the JDK 7 transition to Mercurial. I worked on merging in the fixes from the corba, jaxp, jaxws, and langtools areas. Jon helped with langtools too.

A variety of techniques were used to first find the fixes in the 6 update train that were absent in the OpenJDK 6 code base and then apply them as appropriate. The three basic strategies were:

• Examining the bug database to find bugs fixed in some 6 update release but not in OpenJDK 6. Patches implementing those fixes could then be applied.

• Teamware bringover and merge. The teamware bringover command pulls down changes from the parent workspace; per-file SCCS histories track changes and are used to identify any merge conflicts that need to be resolved.

• Raw diffs of source files. Crude, but sometimes effective with the right preprocessing.

There were various complications using these techniques. Not all areas follow the same bug database practices so using the bug database alone was not sufficient. Standard practice for maintaining JDK code is to have each SCCS delta tagged with a set of bug numbers (or tagged as a merge or a copyright update) and to not add empty deltas when the code has not changed. However, some areas of the JDK, jaxws in particular, are effectively maintained externally and only occasionally synced. In those cases, this SCCS discipline is not necessary followed, rendering a teamware merge operation ineffective since many files will have spurious conflicts. However, even in areas where standard practices are followed, a simple teamware bringover and merge may not suffice. For example, a file may have been removed from OpenJDK 6 but not removed from the 6 update train; a simple bringover would recreate the file. Raw diffs of source files can also be informative, but some preprocessing is needed to bring both code bases to a common format.

Compared to the 6 update train, as part of open sourcing and in preparation for transition to Mercurial, the OpenJDK 6 code base has undergone a number of pervasive transformations. On a file level:

• (generally) a GPL license is used instead of the Sun TLDA

• SCCS keywords have been purged

• whitespace has been normalized, tabs replaced by spaces and source files end with a newline

Therefore, to compute a meaningful raw diff, each of these transformations needs to be applied to the 6 update source or undone from the OpenJDK 6 source; although diff -w can bridge inconsistent whitespace conventions of course. However, even after being brought to a common format, since OpenJDK 6 is a backward branch from JDK 7, certain refactorings are present in the OpenJDK 6 train but absent from the 6 update train, obscuring both teamware-based and diff-based file comparisons. On a workspace level, the 6 update releases still use a monolithic j2se workspace in contrast to the split component workspaces in OpenJDK 6. Consequently, the contents of some subdirectories are now spread to multiple areas, further complicating bringover logistics.

The initial approach to evaluate merging in changes to each OpenJDK 6 component workspace was to:

1. Generate a list of source and test directories of the component workspace. (The make directories were excluded from consideration in the merge attempt since the workspace split fundamentally changed the makefile structure.)

2. Do a trial bringover -n of those directories from a 6u10 workspace. A workspace has a "biological parent" which gives birth to it; however, a bringover can use a different parent for comparison purposes instead. In this case, for the merge operation I used the 6u10 j2se workspace to be a temporary adoptive parent of my OpenJDK 6 component workspace. However, the eventual results of the merge would be committed to the biological parent OpenJDK 6 workspace.

3. Refine the directory list to avoid including spurious files.

4. Evaluate the utility of the bringover results in terms of merging files and generating conflicts.

While this procedure provided useful information, a simple bringover from 6u10 into OpenJDK 6 was never sufficient to properly capture the set of fixes without introducing other sorts of regressions, such as open source related file and license cleanup. On a per component workspace basis, those complications were:

• corba: Generally the bringover from corba went very smoothly; there were only 12 conflicting files and 8 new files that would be created. However, none of the 8 new files ended up being added to the OpenJDK 6 corba workspace; four the new files now live in the jdk workspace after the workspace split and the other four were unneeded binary files previously removed from the workspace in preparation for open sourcing. Resolving the actual file conflicts was straightforward and the fixes for eight bugs were brought over. After the merge, the OpenJDK 6 corba workspace had only a slightly different structure than in 6u10 since the OpenJDK 6 corba was changed to no longer require a scheme interpreter during the build!

• jaxp: In several dozen directories, there were about 30 conflicting files and two new files that would be created. Many of the differences were small, such as purging of SCCS keywords in OpenJDK 6. Where an SCCS keyword was purged, the purge was kept in the merged result; likewise licensing refinements and cleanups in OpenJDK 6 were also preserved in the merge. However, all changes that affected the semantics of the running code were brought in from 6u10. In summary, for the 30 or so conflicting files, generally the code matches the code in 6u10 and the license matches the license in OpenJDK 6. In previous syncs with the JDK, jaxp didn't always follow bug database discipline so a marker merge bug was created to supplement the known fix that was brought over. One of the files that would have been created was previously removed in JDK 7 so it was not resurrected and the other potential new file was not necessary and thus not created either.

• jaxws: The jaxws code is externally maintained and occasionally synced; however, standard teamware delta practices are not followed, resulting in thousands of spurious merge conflicts being reported on a bringover. Therefore, by using a script that stripped off the leading comment in a file (removing license differences) and normalized whitespace, transformed versions of the OpenJDK 6 and 6u10 jaxws sources in a common format could be compared. The only nontrivial differences were for the upgrade to JAF 1.1.1. Other than difficulties getting the tests for these changes to work in jprt, incorporating the fixes was straightforward. In addition, the jaxws team verified no other fixes went into 6u10 that were not already present in OpenJDK 6 (the same security fixes had been applied independently to both releases).

• langtools: The javac in OpenJDK 6 inherited a number of restructuring changes from JDK 7 that permeated the code, limiting the effectiveness of both teamware-based and source-file based comparisons to find true differences. Instead, queries on the bug database were used to identify bugs fixed in some 6 update release, but not in OpenJDK 6. Finding the effective set of bugs fixed in OpenJDK 6 compared to the originally shipped JDK 6 was computed as:

  • The bugs fixed in JDK 7 before the backward branch to create OpenJDK 6.

  • PLUS bugs directly fixed in OpenJDK 6 since the inception of its workspaces.

  • MINUS any "antibugs" that were annihilated in OpenJDK 6. An antibug is a change from JDK 7 that is inappropriate for a Java SE 6 implementation, such as OpenJDK 6, that is fixed by undoing the change. For example, a change in JDK 7 that added a class or method in the java.* or javax.* namespaces is inappropriate for a Java SE 6 implementation and must be removed for Java SE 6 conformance.

  The set of bugs cumulatively fixed in 6u10 is just a union of the bugs fixed in each update release up to and including 6u10: 6u1, 6u2, 6u3, 6u4, 6u5, 6u6, 6u7, and 6u10. In the end, the patches for about five groups of langtools bugs needed to the applied.

When reviewing the changes to corba, jaxp, and jaxws, two sets of webrevs were generated, one comparing the merge result against 6u10 and another comparing the merge result against OpenJDK 6. Each was important to fully verify the correctness of the change both in terms of licensing and semantics.

Friday September 12, 2008

OpenJDK 6: Some regression test results for b12

Running with jtreg flags -a -ignore:quiet, the basic regression test results on Linux for OpenJDK 6 build 12 are:

• HotSpot, 5 tests pass.

• Langtools, 1,346 tests pass, 2 tests fail.

• JDK, 2,985 tests pass, 36 tests fail, 4 tests have errors.

For langtools, from the previous build a few new (passing) tests were added and @ignore tests were really ignored; using jtdiff, part of the jtreg package to compute the differences of the tests results:

0: ./b11-langtools/summary.txt  pass: 1,341; fail: 2; error: 6
1: ./b12-langtools/summary.txt  pass: 1,346; fail: 2

0      1      Test
---    pass   tools/javac/6176978/T6176978.java
---    pass   tools/javac/6627362/T6627362.java
error  ---    tools/javac/T6405099.java
error  ---    tools/javac/api/T6306137.java
error  ---    tools/javac/api/T6397104.java
error  ---    tools/javac/api/TestJavacTaskScanner.java
error  ---    tools/javac/code/ArrayClone.java
---    pass   tools/javac/enum/T6509042.java
error  ---    tools/javac/generics/inference/6365166/NewTest.java
---    pass   tools/javac/staticImport/6665223/T6665223.java
---    pass   tools/javac/synthesize/Main.java

11 differences

Also for the jdk area, some new passing tests were added and the -ignore:quiet option prevents some tests that knowingly produce an error result from running. The few new failures warrant a bit of investigation.

0: b11-jdk/summary.txt  pass: 2,989; fail: 32; error: 22
1: b12-jdk/summary.txt  pass: 2,985; fail: 36; error: 4

0      1      Test
error  ---    com/sun/crypto/provider/Cipher/DES/PerformanceTest.java
pass   ---    com/sun/org/apache/xml/internal/security/exceptions/LocaleTest.java
pass   ---    com/sun/org/apache/xml/internal/security/transforms/ClassLoaderTest.java
error  ---    com/sun/security/auth/callback/DialogCallbackHandler/Default.java
error  ---    com/sun/security/auth/callback/TextCallbackHandler/Default.java
error  ---    com/sun/security/sasl/gsskerb/AuthOnly.java
error  ---    com/sun/security/sasl/gsskerb/ConfSecurityLayer.java
error  ---    com/sun/security/sasl/gsskerb/NoSecurityLayer.java
pass   fail   java/awt/Focus/DeiconifiedFrameLoosesFocus/DeiconifiedFrameLoosesFocus.html
fail   pass   java/awt/Focus/FrameMinimizeTest/FrameMinimizeTest.java
pass   fail   java/awt/TextArea/UsingWithMouse/SelectionAutoscrollTest.html
---    pass   java/awt/Window/PropertyChangeListenerLockSerialization/PropertyChangeListenerLockSerialization.java
error  ---    java/io/SystemInAvailable.java
error  ---    java/lang/instrument/TransformerManagementThreadRemoveTests.java
error  ---    java/lang/management/ThreadMXBean/SynchronizationStatistics.java
fail   pass   java/nio/channels/SocketChannel/Connect.java
pass   ---    java/rmi/activation/Activatable/checkActivateRef/CheckActivateRef.java
error  ---    java/util/AbstractList/CheckForComodification.java
---    pass   java/util/EnumSet/BogusEnumSet.java
error  ---    java/util/zip/3GBZipFiles.sh
fail   ---    javax/management/Introspector/LegacyIntrospectorTest.java
error  ---    javax/management/remote/mandatory/URLTest.java
error  ---    javax/security/auth/kerberos/KerberosHashEqualsTest.java
---    pass   javax/sound/midi/Gervill/DLSSoundbankReader/TestGetSoundbankFile.java
---    pass   javax/sound/midi/Gervill/DLSSoundbankReader/TestGetSoundbankInputStream.java
---    pass   javax/sound/midi/Gervill/DLSSoundbankReader/TestGetSoundbankInputStream2.java
---    pass   javax/sound/midi/Gervill/DLSSoundbankReader/TestGetSoundbankUrl.java
---    pass   javax/sound/midi/Gervill/SF2SoundbankReader/TestGetSoundbankFile.java
---    pass   javax/sound/midi/Gervill/SF2SoundbankReader/TestGetSoundbankInputStream.java
---    pass   javax/sound/midi/Gervill/SF2SoundbankReader/TestGetSoundbankInputStream2.java
---    pass   javax/sound/midi/Gervill/SF2SoundbankReader/TestGetSoundbankUrl.java
fail   pass   javax/swing/JColorChooser/Test6541987.java
pass   ---    javax/xml/crypto/dsig/GenerationTests.java
pass   ---    javax/xml/crypto/dsig/ValidationTests.java
pass   ---    javax/xml/crypto/dsig/keyinfo/KeyInfo/Marshal.java
pass   ---    sun/net/idn/PunycodeTest.java
pass   ---    sun/net/idn/TestStringPrep.java
pass   ---    sun/security/pkcs11/ec/TestCurves.java
error  ---    sun/security/provider/PolicyFile/GrantAllPermToExtWhenNoPolicy.java
error  ---    sun/security/provider/PolicyParser/ExtDirs.java
error  ---    sun/security/provider/PolicyParser/ExtDirsChange.java
error  ---    sun/security/provider/PolicyParser/ExtDirsDefaultPolicy.java
error  ---    sun/security/provider/PolicyParser/PrincipalExpansionError.java
pass   fail   sun/tools/jps/jps-Defaults.sh
pass   fail   sun/tools/jps/jps-l_1.sh
pass   fail   sun/tools/jstatd/jstatdDefaults.sh
pass   fail   sun/tools/jstatd/jstatdExternalRegistry.sh
pass   fail   sun/tools/jstatd/jstatdPort.sh
pass   fail   sun/tools/jstatd/jstatdServerName.sh

49 differences

I did a quick trial run of the regression tests on windows too, but my new cygwin installs seems to be missing a few pieces and many of the shell tests fail. I assume most of those failures are spurious, so I'll wait until the cygwin install seems configured properly before rerunning and posting windows results.

OpenJDK 6: Sources for b12 published

On September 12, the OpenJDK 6 b12 source bundle was published. Changes of note in this build:

• An updated version of Gervill is included (6717691, 6740210).

• Functionality of several IcedTea patches which correct JCK failures have been applied (6735154, 6733501).

• I've ported fixes from the forthcoming 6u10 release in a subset of the component workspaces to OpenJDK 6:

  • corba (6372405, 5104239, 6191561, 6486322, 6465377, 6553303, 6372405, 6438259)

  • jaxp (6573268, 6738894)

  • jaxws (6538484, 6699090)

  • langtools (6627362, 6627364, 6627366, 6176978)

• Incorporation of various build fixes, including the gcc ciTypeFlow.cpp issue (6732815, 6734545, 6590549, 6720240, 6548261, 6681798, 6741642).

• A few more legal notices were corrected (6635663, 6719182).

• Configuration information for some networking tests can now be provided as a properties files specified on the jtreg command line; see the source for jdk/test/TestEnv.java for details (6720349).

• Assorted other fixes (6681889, 6739302, 6730743).

While the changes in this build have corrected a number of JCK 6b test failures, a handful of tests still fail; eliminating those few remaining failures is a goal for the next build or two. (JCK 6b was released several months ago and has now replaced JCK 6a as the relevant test suite to pass for Java SE 6 compatibility.)

Internally, the code for OpenJDK 6 is split into teamware workspaces with a similar structure to the JDK 7 Mercurial repositories (corba, jaxp, jaxws, langtools, jdk, etc.). For a subset of these component workspaces, I've brought over the fixes in corresponding areas from 6u10. OpenJDK 6 build 12 will be one of the last teamware-based OpenJDK 6 build before we transition to a public Mercurial repository. Around the time of the Mercurial transition we will also upgrade the HotSpot in OpenJDK 6 from HotSpot 10 to HotSpot 11; HotSpot 11 is also being used in 6u10. From that point on, the same HotSpot sources will be used for both OpenJDK 6 and the 6 update releases.

I expect build 13 within a few weeks with the Mercurial repositories to follow a few weeks after that.