Jonathan Gibbons
Improving javac diagnostics
In javac land, we're looking at improving the diagnostic messages generated by the compiler ...
Here's one of my anti-favorite messages that I got from the compiler recently:
/w/jjg/work/jsr294/7/langtools/test/tools/javac/superpackages/CompileErrorTest.java:90: getTask(java.io.Writer, javax.tools.JavaFileManager, javax.tools.DiagnosticListener<? super javax.tools.JavaFileObject>, java.lang.Iterable<java.lang.String>, java.lang.Iterable<java.lang.String>, java.lang.Iterable<? extends javax.tools.JavaFileObject>) in javax.tools.JavaCompiler cannot be applied to (<nulltype>, javax.tools.StandardJavaFileManager, <nulltype>, java.util.List<java.util.List<java.lang.String>>, <nulltype>, java.lang.Iterable<capture#81 of ? extends javax.tools.JavaFileObject>) JavaCompiler.CompilationTask task1 = c.getTask(
It is my pet peeve message type ("can't apply method...") and also includes wildcards, captured type variables, and a <nulltype>. The text of the error message, excluding source file name and highlighted lines is a whopping 577 characters :-) Who says we don't need to improve this?
We have various ideas in mind. This first list is about the content and form of the messages generated by javac.
Omit package names from types when the package is clear from the context. For example, use
Objectinstead ofjava.lang.Object,Stringinstead ofjava.lang.String, etc, when those are the only classes namedObject,Stringetc, in the context.Don't embed long signatures in message. Instead, restructure the message into a shorter summary plus supporting information.
For example,
Method name cannot be applied to given types required: types found: typesDon't embed captured and similar types in signatures, since they inject wordy non-Java constructions into the context of a Java signature. Instead, use short placeholders, and a key.
For example, in the message above, replace
java.lang.Iterable<capture#81 of ? extends javax.tools.JavaFileObject>
by
java.lang.Iterable<#1>
with a note following the rest of the message:
where #1 is a capture of ? extends javax.tools.JavaFileObject
Put these suggestions together, and the message above becomes:
/w/jjg/work/jsr294/7/langtools/test/tools/javac/superpackages/CompileErrorTest.java:90: method JavaxCompiler.getTask cannot be applied to given types
required: (Writer, FileManager, DiagnosticListener<? super JavaFileObject>,
Iterable<String>, Iterable<? extends JavaFileObject>)
found: (#null, StandardFileManager, #null, List<List<String>>, #null, Iterable<#1>)
where:
#1 is the capture of ? extends JavaFileObject
It is a lot shorter (less than half the length of the original message, if you're counting), and more importantly, it breaks the message down into segments that are easier to read and understand, one at a time. It still has a long file name in it, and I'll address that below.
The following ideas are more about the presentation of messages. javac is typically used in two different ways: batch mode (output to a console), and within an IDE, where the messages might be presented as "popup" messages near the point of failure, and in a log window within the IDE.
When used in batch mode, either directly from the command line or in a build system, the compiler could allow the user to control the verbosity of the diagnostics. If you're compiling someone else's library, you might not be worried about the details in any warnings that might be generated. If you're compiling your own code, you might be comfortable with a quick summary of each diagnostic, or you might want as much detail as possible.
When used in an IDE, it would be good to provide the IDE with more access to the component elements of the diagnostic, so that the IDE could improve the presentation of the message. For example,
- display the base name of the file containing the error, and link it to the compilation unit, instead of displaying the full file name as above
- use different fonts for the message text and the Java code or signatures contained within it
- hyperlink types used in the diagnostic to their declaration in the source code
- given the resource key for the message, an IDE could use the key as an index into additional documentation specific to the type of the error message, explaining the possible causes for the error, and more importantly, what might be done to fix the problem.
Here's how these ideas could be used to improve the presentation of the example message.
CompileErrorTest.java:90: method JavaxCompiler.getTask cannot be applied to given types [More info...]
required: (Writer, FileManager, DiagnosticListener<? super JavaFileObject>,
Iterable<String>, Iterable<? extends JavaFileObject>)
found: (#null, StandardFileManager, #null, List<List<String>>, #null, Iterable<#1>)
where:
#1 is a capture of ? extends JavaFileObject [More info...]
OK, I'll leave the real presentation design to the UI experts, but I hope you get the idea of the sort of improvements that might be possible.
Finally, we'll be looking at improving the focus of error messages. For example, this means that if the compiler can determine which of the arguments is at fault in a particular invocation, it should give a message about that particular argument, instead of about the invocation as a whole. However, care must also be taken not to narrow the focus of an error message incorrectly, so that the message becomes misleading. A typical example of that is when the compiler is parsing source code, and having determined that the next token is not one of A or B, it then checks C. If that is not found the compiler may then report "C expected", when a better message would have be "A, B or C expected." This means that such optimizations have to be studied carefully on a case by case basis, whereas all of the preceding suggestions can be applied more generally to all diagnostics.
So, do you have any "pet peeve" messages you get from the compiler? Do you have suggestions on how the messages could be improved, or how they get presented? Add a comment here, or mail your suggestions to the OpenJDK compiler group mailing list, compiler-dev at openjdk.java.net.
Thanks to Maurizio and others for contributing some of the suggestions here.
See Also
Posted at 12:00PM May 07, 2008 by jonathangibbons in javac | Comments[2]
Evolving KSL
As some of you may know, we've made changes recently to the KSL project that was started last year.
We thought it would be fun to share some of the ideas we had along the way.
 |
For inspiration, we decided to throw a bunch of ideas into a kitchen sink, for real, to see what that might inspire. See how many of your favorite language ideas are represented here. Hint: There are no wrong answers, but there have been some creative ones. :-) |
 |
Of course, as soon as we did that, this little guy wanted to get in on the act. He may not quite understand the gist of KSL, but you can't fault his enthusiasm: there's a keyboard, a couple of mice, a couple of monitors and even a KVM cable in the sink, if you look carefully! |
 |
For information about what we finally came up with, see the Compiler Group KSL page. |
(Thanks to Alex for helping with the first photo, and to Greg Quayle for creating the sculpture in the second.)
Posted at 04:20PM May 06, 2008 by jonathangibbons in javac | Comments[2]
jtharness vs jtreg
Now that the source code for the OpenJDK Regression Test Harness (jtreg) is available, this provides an overview of how jtreg relates to JT Harness.
In the beginning, there was a need to run compatibility tests on all Java plaforms, and thus was born the "Java Compatibility Kit" (JCK), and the test harness to run those tests (JavaTest). Not long after, there was a need to run a different type of test, regression tests, for JDK. They are fundamentally a different style of test (think white box and black box), but rather than write another, different harness, we adapted JavaTest, and thus was born "jtreg". Eventually, we needed to be able to run tests on different types of platforms, such as Personal Java, a precursor to the J2ME platform. JavaTest evolved to fill the need, and gave rise to the basic architecture you see today.
As you can see, JavaTest, now open sourced under the name JT Harness, is composed of a number of distinct components
- The core harness. This provides the general ability to represent tests, test results, and to sequence through the execution of a series of tests.
- Utilities. These provide support to the other JT Harness components.
- The user interface. JT Harness provides a rich graphical user interface, as well as a command line interface.
And, deliberately, there's a hole. By itself, JT Harness cannot directly run tests in any test suite. It needs to be given a plugin for each test suite, that define how to locate and read tests, how to configure and execute tests, and other additional information, often used by the GUI, to provide documentation specific to the test suite. The plugin code is not complicated and is often a direct use or simple extension of the utility classes provided by JT Harness. Note: most test suites do not need to use all of the JT Harness utility classes, which explains why JT Harness may have more build-time dependencies than are required to build and run any individual test suite.
Thus, the complete picture is more as follows:
This is the form used for most test suites, such as JCK and other TCKs. The code to plugin to the harness is bundled in the test suite, along with a copy of JavaTest or JT Harness itself. These test suites all leverage the user interfaces provided by the harness, both the CLI and GUI. When the user runs JT Harness, and tells it to open a test suite, the harness looks inside the test suite for details of the plugin to be used in conjunction with that test suite.
Because jtreg evolved from JavaTest early on, and specifically, before the JavaTest user interfaces evolved to what you see today, jtreg has a slightly different architecture.
jtreg still utilizes the harness' plugin architecture to configure how to read and execute the tests in the test suite, but it provides its own custom command line interface for setting up a test run. The jtreg CLI provides options tailored to running the JDK regression test suite. It can also invoke the JT Harness GUI, but this is just for monitoring a test run, and does not (yet) provide much support for reconfiguring a test run with different options from within the GUI, as you can do for other test suites.
jtreg is also different from other test suites in that jtreg is distributed separately from the tests that it gets to execute. TCKs are generally built and distributed as self-contained products in their own right, whereas the JDK regression tests do not get prebuilt: they exist and are executed in their source form in each JDK developer's repository. Per the The JDK Test Framework: Tag Language Specification, the tests are compiled and executed as needed.
For more information, see ...
Posted at 02:54PM May 05, 2008 by jonathangibbons in jtreg | Comments[0]
OpenJDK Regression Test Harness (jtreg) - open source
The OpenJDK Regression Test Harness, also known as "jtreg", is now available with an open source license.[Read More]
Posted at 08:54PM May 01, 2008 by jonathangibbons in jtreg | Comments[1]
M&Ms: Milestones and Members
M&M's are great, so here's a few with no calories whatever.[Read More]
Posted at 05:33PM Mar 04, 2008 by jonathangibbons in javac | Comments[0]
The King is dead. Long live the King!
Well, the phrase may not be entirely apt, but it does carry the right sentiment, especially to the monarchists amongst us.
As you may have read, Peter is moving on to new adventures, but you can be sure that there are still plenty of adventures still to be had with javac, and the rest of us on the javac team are looking forward to carrying on with everything that is coming up.
With the compiler being open sourced last year, and the recent announcements regarding OpenJDK, and JDK 7 ahead, these are exciting times, albeit somewhat turbulent at times. So, stay tuned for more details, and in the meantime, thanks go to Peter for all his contributions.
Posted at 03:01PM May 18, 2007 by jonathangibbons in javac | Comments[4]
Unit Testing Wisdom From An Ancient Software Start-up
JavaOne contained many gems of wisdom, of varying sizes. This one indirectly came my way during the week, and is fun enough to pass along. It is self-described as "Good advice on developer and unit testing, packaged as twelve cryptic bits of ancient Eastern wisdom." You can find it here. It is also available in PDF. Enjoy -- and thanks to Alberto Savoia for making it available.
Posted at 07:12PM May 14, 2007 by jonathangibbons in jtreg | Comments[0]
Office Tango
Tom Lehrer sang a song called "The Masochism Tango". Here at Sun, we call it "Office Moves".[Read More]
Posted at 08:00AM Nov 17, 2006 by jonathangibbons in Sun | Comments[2]
com.sun.javatest.*: JT Harness API
JavaTest provides some useful APIs for specialized usage ...[Read More]
Posted at 08:00AM Nov 16, 2006 by jonathangibbons in jtreg | Comments[0]
javatest.png: JavaTest in pictures
A look at the evolution of the JavaTest harness, over the years.[Read More]
Posted at 08:00AM Nov 15, 2006 by jonathangibbons in jtreg | Comments[2]
Way to go, JT Harness
A brief background overview of the harness underlying jtreg: the JavaTest harness, which is now available as JT Harness. [Read More]
Posted at 08:07AM Nov 14, 2006 by jonathangibbons in jtreg | Comments[0]
wwwww.jtreg: The Who What Where When Why of jtreg
A brief background overview of jtreg, the JDK regression test harness. [Read More]
Posted at 12:00AM Nov 13, 2006 by jonathangibbons in jtreg | Comments[0]
Hello World
Who am I?[Read More]
Posted at 01:41PM Oct 04, 2006 by jonathangibbons in Sun | Comments[6]