(As my mother might have asked...)

You all know about the Java technology releases (the platforms, and various optional packages). You might not know that each technology release is also accompanied by a TCK (Technology Compatibility Kit), which is a test suite designed to verify that implementations of the technology conform to the specifications.

This is a requirement of the Java Community Process (see the JCP process doc for the scary details).

The JCP requires three simultaneous deliverables: the specification, the TCK, and the Reference Implementation. Each of these depends on the others, as this diagram tries to explain:

The specification defines the functionality of the Java technology and how it may be accessed from Java programs. The Reference Implementation (RI) is a proof of concept that the specification is actually implementable. It doesn't have to be a commercially available product, but it does have to correctly implement the complete specification. The TCK verifies that the RI does in fact conform to the spec. Conversely, the TCK is tested by running it against the RI. Finally, the process of creating the TCK helps to clarify the specification.

To the extent that our specifications are of high quality (the public development and review process help to ensure this), the TCK and the accompanying Compatibility Requirements (more on these in a future posting) promote compatibility and interoperability between implementations.

My team is responsible for developing the TCK for the Java SE platform. (Just to confuse you further it's called the JCK, or Java Compatibility Kit). We also develop TCKs for various technologies that start out as standalone "optional packages" but that we expect will be incorporated into the platform at some point (for example, JAXP and JAXB).

The interdependence of these three components complicates the TCK development process. If we start too soon both the spec and the RI will still be unstable, and we'll have to make many changes as they evolve. If we leave it too late we won't be able to develop enough tests to provide reasonable coverage of the spec before the implementation ships. We can't execute our tests until we have an implementation to run against, and once we do so we can never be quite sure whether test failures are due to errors in the implementation or bugs in the tests. Somehow it all comes together eventually, but the journey can be quite a challenge.

The first step in developing conformance tests is to review the spec carefully. During this process we provide feedback to the spec authors where we find ambiguities or where we feel that clarification is needed. We identify "assertions" (statements of specific behaviour that potentially can be tested) within the spec. Deciding which assertions to test is something of a black art. It's usually not possible to test everything (though we come pretty close for critical specifications such as the Java Language and the Java Virtual Machine specs). We try to focus in areas where the likelihood or consequences of incompatible implementations are greatest.

We take this very seriously. Over the past ten years we've spent approximately 250 engineer-years (really!) developing the JCK. The JCK currently consists of about 80,000 test cases and several million lines of code. We run the entire test suite against every build of the platform and on multiple configurations. In addition to ensuring compatibility, it forms a critical component of our quality assurance processes.

Unlike Java SE itself, we don't release snapshots of the JCK as we're developing it. We do however release it for public review at the end of the development cycle. If you want to take a look at the last published version, see here. You might also want to take a look at this blog from Graham Hamilton in which he discusses the licensing terms that apply to that release.

More later...

The core strength of the Java platform is compatibility. There can be all kinds of implementations from all kinds of vendors, but they all have to be compatible. This means that developers can write portable Java applications that will reliably execute on any available run-time, while vendors can invest their energy in clever implementation techniques to build the best compatible implementations.

Java compatibility is promoted through a standards-based process. Specifications are developed publicly and can be licensed by anyone, while compatibility (conformance) test suites are used to verify that implementations conform to the specifications. This white paper provides a useful introduction to the whole process.

This is what I do for a living. I'm the manager of the group that's responsible for developing the conformance test suites for the Java SE platform and related technologies. You have to be a little weird to want to do this kind of work, since it involves lots of arguments along the lines of "it all depends on what the meaning of the word 'is' is". It helps if you're the kind of person who likes to read the fine print in legal documents rather than skipping over them with a glazed expression and hoping that everything will turn out OK. I like to read the fine print.

I'll be using this blog to talk about the standards process in general and Java compatibility in particular. Stay tuned.