Joseph D. Darcy's Sun Weblog

Wednesday December 03, 2008

OpenJDK 6 b13 and JCK testing

The JCK tests probe the conformance of a Java platform implementation to a specification. For example, JCK 6b is the current test suite to determine conformance to the Java SE 6 spec. Official claims of conformance require not only passing the complete set of relevant tests, but also meeting the other requirements spelled out in the JCK user's guide.

Regarding the JCK, conformance is measured with respect to a binary rather than to a source base directly, which is sensible since a Java platform implementation will typically rely on and be affected by the properties of the host environment, including the OS, the C compiler, and the processor.

Previously Red Hat announced that using OpenJDK sources augmented with IcedTea patches, an OpenJDK binary on Fedora 9 passed the JCK and met the other conformance requirements.

Amongst other changes, after incorporating community-developed patches (notably 6748251 in b13) and following the OpenJDK 6 build instructions, inside Sun a binary resulting from building the unmodified OpenJDK 6 b13 sources on Redhat Enterprise AS 2.1 with gcc 2.95 (the official Linux build platform for Sun's 6 update releases) passed all the JCK 6b tests when run on Fedora Core 8 x86. That binary also meets all the other JCK requirements.

OpenJDK 6 binaries built from b13 (and later) sources on and for different host environments are now more likely to share those favorable conformance properties, but testing would be necessary to verify conformance status and to make any formal statements.

More information

• Free and Open Source Java FAQ on TCKs

• OpenJDK Conformance Group

OpenJDK 6: Some regression test results for b14

Running with the usual jtreg flags, -a -ignore:quiet always and -s for the langtools area, the basic regression test results on Linux for OpenJDK 6 build 14 are:

• HotSpot, 3 tests passed.

• Langtools, 1,351 tests passed.

• JDK, 3,077 tests pass, 26 tests fail, 3 tests have errors.

In this build, we upgraded from a HotSpot 10 base to HotSpot 11. HotSpot 11 is also being used in the 6u10 release. The set of included tests for the HotSpot versions differ:

0: b13-hotspot/summary.txt  pass: 5
1: b14-hotspot/summary.txt  pass: 3

0      1      Test
pass   ---    compiler/6547163/Test.java
pass   ---    compiler/6563987/Test.java
pass   ---    compiler/6571539/Test.java
pass   ---    compiler/6595044/Main.java
---    pass   compiler/6663621/IVTest.java
---    pass   compiler/6724218/Test.java

6 differences

In langtools all the tests continue to pass:

0: ./b13-langtools/summary.txt  pass: 1,351
1: ./b14-langtools/summary.txt  pass: 1,351

No differences

And in jdk, a few new tests were added in b14 and the existing tests have generally consistent results:

0: b13-jdk/summary.txt  pass: 3,072; fail: 23; error: 3
1: b14-jdk/summary.txt  pass: 3,077; fail: 26; error: 3

0      1      Test
---    fail   com/sun/org/apache/xml/internal/ws/server/Test.java
pass   fail   java/awt/TextArea/UsingWithMouse/SelectionAutoscrollTest.html
---    pass   java/awt/image/ConvolveOp/EdgeNoOpCrash.java
---    pass   javax/management/monitor/DerivedGaugeMonitorTest.java
pass   fail   javax/swing/JColorChooser/Test6541987.java
---    pass   javax/swing/JFileChooser/6484091/bug6484091.java
---    pass   sun/management/jmxremote/LocalRMIServerSocketFactoryTest.java
---    pass   sun/nio/cs/TestUTF8.java
---    pass   sun/security/ssl/com/sun/net/ssl/internal/ssl/SSLEngineImpl/EmptyExtensionData.java
---    pass   tools/pack200/MemoryAllocatorTest.java

10 differences

OpenJDK 6: Sources for b14 published

On December 3, the OpenJDK 6 b14 source bundle was published. Changes of note in this build:

• All relevant security fixes in the current round of security updates (6592792, 6484091, 6766136, 6755943, 6751322, 4486841, 6721753, 6734167, 6733336, 6726779, 6588160, 6733959, 6497740).

• Upgrading the HotSpot in OpenJDK 6 from HotSpot 10 to HotSpot 11 (6775772).

• Correcting legal notices (6772068, 6773363, 6706382).

• Fixing build documentation and other miscellaneous fixes (6775844, 6775851, 6683213, 6774170, 6741058, 6728126, 6745052).

This will be the last teamware based source drop; all future source drops will be based on the forthcoming public OpenJDK 6 Mercurial repositories.