Joseph D. Darcy's Sun Weblog

Tuesday March 31, 2009

Project Coin: The Call for Proposals Phase is Over!

Update: Added links to proposal for switch for all types and simple expressions and a link to a revised method chaining proposal.

Project Coin's call for proposals phase is now over! Thirty four days long, the proposal period included nearly 70 proposals being sent to the mailing list, 19 coming in over the last two days, and over 1100 messages on the list discussing those proposals and related topics. With the flurry of pre-deadline activity over, the more deliberative task of finishing reviewing and evaluating the proposals awaits. Including several sent in a few hours after deadline, the proposals received since week four are:

• Type inference for variable definition/initialization using the 'auto' keyword, Tim Lebedkov

• Simplified StringBuffer/StringBuilder syntax, Derek Foster

• language support for JSR 292, John Rose

• switch (...) instanceof feature, Jeroen van Maanen

• Simple Operator Overloading, Ruslan Shevchenko

• Conditional Statement, Matt Mastracci

• Enhanced while statement, Marek Kozieł

• Indexing access syntax for Lists and Maps, Shams Mahmood Imam

• Concise Collection Initialization Syntax, Shams Mahmood Imam

• open and close brace optional for single statement try, catch, finally, method declaration, Glenn Marshall

• abstract enums, Derek Foster

• Generic Specification by Method, Ron Thomas

• Allow the class literal of the surrounding class to be referenced with the 'class' keyword, Peter Runge

• Simply implicit method chaining (revised), Ulf Zibis

• Extend switch .. case statement for Object types and simple expressions (revised), Ulf Zibis

• Rethrows Clause, Mark Mahieu

• Collection Literals, Joshua Bloch

• Underscores in numbers, Derek Foster

• Templated Construction Expressions (i.e., Expressions Embedded in Strings), John Rose

• Glue classes based on Extension Methods, Marek Kozieł

• Improved declaration using final keyword of internal method variables and for-loop statement, Andrei Navodnik

• Sameness operators, Derek Foster

• Addition to Comparable interface, Roy van Rijn

• Accepting a subclass as an element type in a for loop, Jean-Louis Ardoint

• @OverrideAll annotation, Gabriel Belingueres

The figure below graphs when proposals were received; nothing like an impending deadline to focus the mind!

Sometime after the next for further consideration cut is made, I'll post some thoughts on and reaction to the call for proposals phase as a whole. This will not include a detailed analysis of why each proposal was or was not chosen; however, there will be discussion of common aspects of proposals that led them to be selected or not.

Friday March 27, 2009

Project Coin: Week 4 Update

Update: Corrected to include Stephen Colebourne's enhanced enhanced for loop proposal.

Further update: Added links to updated large arrays and compile time access proposals.

Project Coin's fourth week saw continued lively traffic on the mailing list. As the submission deadline approaches, a flurry of new proposals were sent in:

• Named method parameters, Paul Martin

• Enhanced for each loop iteration control, Stephen Colebourne

• Loosen Constructor super()/this() call restrictions, Mike Duigou

• Simplified syntax for dealing with parameterized types, J. Lowden

• Compiletime information access (revised), Ruslan Shevchenko

• @Unloadable Class defination, Daniel Cheng

• Large arrays (revised), James Lowden

• Naked dot, Makarenko Alexandre

• Auto-assignment Parameters , Mark Mahieu

• Unchecked Exceptions as Subclasses of Checked Exceptions (revised), Alan Snyder

• Narrow Hexadecimal and Binary Integer Literals, Bruce Chapman

• Byte and Short Integer Literal Suffixes, Bruce Chapman

• Unsigned Integer Widening Operator, Bruce Chapman

• Binary Literals, Derek Foster

• Return 'this', Marek Kozieł

• Improved Support for Optional Object Behaviors at Runtime, Alan Snyder

• 'final' without explicit type, Marek Kozieł

• Language Escape Operator, Bruce Chapman

• Using Ordinary Syntax for Reflective Method Invocations, Alan Snyder

The field of over two dozen proposals previously sent in over the first three weeks of Project Coin was narrowed to six proposals still in consideration for inclusion in JDK 7. The proposals submitted this week and until the end of the call for proposals period will be similarly evaluated for their appropriateness to be added to the language. Finally, the combined list of candidate changes will be produced.

Tuesday March 24, 2009

A trail of coins

For those interested in specifically following Project Coin related posts, my Project Coin entries are tagged with "projectcoin":
http://blogs.sun.com/main/tags/projectcoin

Project Coin: For further consideration...

In the first three weeks of Project Coin over two dozen proposals have been sent to the mailing list for evaluation. The proposals have ranged the gamut from new kinds of expressions, to new statements forms, to improved generics support. Thanks to all Java community members who have sent in interesting, thoughtful proposals and contributed to informative discussions on the list!

While there is a bit less than a week left in the call for proposals period, there has been enough discussion on the list to winnow the slate of proposals sent in so far to those that merit further consideration for possible inclusion in the platform.

First, Bruce Chapman's proposal to extend the scope of imports to include package annotations will be implemented under JLS maintenance so further action is unnecessary on this matter as part of Project Coin. Second, since the JSR 294 expert group is discussing adding a module level of accessibility to the language, the decision of whether or not to include Adrian Kuhn's proposal of letting "package" explicitly name the default accessibility level will be deferred to that body. Working with Alex, I reviewed the remaining proposals. Sun believes that the following proposals are small enough, have favorable estimated reward to effort ratios, and advance the stated criteria of making things programmers do everyday easier or supporting platform changes in JDK 7:

• Strings in switch, Joe Darcy

• Improved Exception Handling for Java, Neal Gafter

• Automatic Resource Management, Josh Bloch

• Improved Type Inference for Generic Instance Creation, Jeremy Manson

• Elvis and Other Null-Safe Operators, Written by Neal Gafter and submitted by Stephen Colebourne

• Simplified Varargs Method Invocation, Bob Lee

As this is just an initial cut and the proposals are not yet in a form suitable for direct inclusion in the JLS, work should continue to refine these proposed specifications and preferably also to produce prototype implementations to allow a more thorough evaluation of the utility and scope of the changes. The email list should focus on improving the selected proposals and on getting any remaining new proposals submitted; continued discussion of the other proposals is discouraged.

The final list of small language changes will be determined after the call for proposals is over so proposals sent in this week are certainly still in the running! The final list will only have around five items so it is possible not all the changes above will be on the eventual final list.

Friday March 20, 2009

Project Coin: Week 3 Update

Project Coin's third week was another week of lively traffic on the mailing list. New proposals were sent in:

• Extended arrays declaration and access syntax with use of enumerated types, Alexey Kuznetsov

• Embedded Expressions for String Statements, Stefan Schulz

• Access to Generic Type Parameters at Compile-Time, David Walend

• Small property support, Ruslan Lazukov

existing proposals were revised:

• Static Methods in Interfaces (v1.1) , Reinier Zwitserloot and Roel Spilker

• Enhanced String literal for Java: version 1.3 with implementation, Alexey Kuznetsov

and discussion continued on ARM and other proposals. The scoping and utility of a few pre-proposals was discussed on the list too.

Ten days remain to get language change proposals in! (Purely libraries changes will be handled by other JDK 7 processes.)

Friday March 13, 2009

Project Coin: Week 2 Update

After the vigorous start of week 1, the pace of new proposals being sent to the list slowed:

• (pre-proposal) Source and Encoding keyword, Reinier Zwitserloot and Roel Spilker

• Extend Scope of Imports to Include Package Annotations, Bruce Chapman

• Enhanced String constructs for Java, Felix Frost

• Allow the use of relational operators on Comparable classes (withdrawn), Vilya Harvey

• Method and Field Literals , Jesse Wilson

However, brisk discussion continued on refining and exploring ARM blocks and their variations.

Thursday March 12, 2009

Language Model Changes as of JDK 7 Build 50

To date, there have been a few API changes to javax.lang.model.* in JDK 7. Early in the release, SourceVersion.RELEASE_7 was added to correspond to any new source-level changes coming in JDK 7 (6458819). Eventually, there will be changes to support the modulue construct being added by JSR 294; changes may or may not be needed for language changes coming from Project Coin and JSR 308. Once modulues are added, the JSR 269 API elements meant to cope with the expression problem will be tested. In the mean time, some minor API cleanups have occurred, one to make handling exceptions easier (6794071), another to group common functionality in mixin interfaces (6460529), and some minor API clarifications (6501749). Along the way, there has been a little general bug fixing too (6478017, 6583626, 6498938).

Small tweaks and bug fixing will continue to occur in the javax.lang.model.* and javax.annotation.processing packages throughout JDK 7 in addition to changes needed to support new language features.

Design tips: Exception types

Expanding on a few slides from my JavaOne talk last year, here are a few tips to keep in mind when designing exception types.

First, all exceptions are serializable since Throwable implements Serializable; therefore, like all other serializable classes, exception types should declare a serialVersionUID field to ease evolving the type in the future. Using's javac's -Xlint:serial option will warn about missing serialVersionUID fields on serializable classes, amongst other possible issues.

Second, when adding a new exception class, consider providing more information beyond just a distinct name, such as methods to return information about what specific situation triggered the exception and possibly how to recover from it. Providing this additional information can interact with being serializable; when the additional information is not logically serializable, the specification may need to allow the information to be unavailable after deserialization.

Third, when multiple related exceptions types are added, a common direct superclass allows a single catch block to handle the related exceptions uniformly. (Having a common super-exception would still be useful even if multi-catch is added to the language in JDK 7.)

Looking at the exceptions in the JSR 269 API, various methods note the possible impact of serialization-deserialization on the returned values. JSR 269 provided a trio of similar exceptions for the situation of encountering a kind of object unknown in an earlier version of the language, such as a JDK 6 era annotation processor coming across a module from JDK 7:

• javax.lang.model.element.UnknownAnnotationValueException

• javax.lang.model.element.UnknownElementException

• javax.lang.model.type.UnknownTypeException

However, the original JSR 269 API does not have a common direct superclass to group these related conditions. That deficiency was addressed in JDK 7 build 48 with the addition of javax.lang.model.UnknownEntityException as the direct superclass of these three exceptions (6794071). Retrofitting this change is binary compatible because UnknownEntityException directly extends RuntimeException as did the old exceptions and serialization compatibility is preserved by the existing serialVersionUID fields in the old exceptions.

Tuesday March 10, 2009

All about JDK 7

Those interested in following JDK 7 happenings from Sun engineers can track "jdk7" tagged entries on http://blogs.sun.com/main/tags/jdk7.

Sunday March 08, 2009

The crested butte of Crested Butte

Last week I was off attending my first Java Posse Roundup in scenic Crested Butte Colorado, pictured below. There were many good discussions related to JDK 7 and other programming, and non-programming, topics.

Unfortunately, there were some difficulties with my flight back. Once again, the plane I was flying on had to be rebooted, but at least this time the passengers didn't need to be reinstalled! I missed my scheduled connection in Denver and caught the next flight to the bay area a few hours later. The wait in Denver was made more pleasant by the airport's free-after-a-short-ad wi-fi and recharging stations for electronic gear. More airports should have those amenities!

Friday March 06, 2009

Project Coin: Week 1 Update

In its first week, Project Coin enjoyed a vigorous start with well over a dozen proposals submitted:

• Strings in switch, Joe Darcy

• Block Expressions for Java, Neal Gafter

• Improved Exception Handling for Java , Neal Gafter

• Automatic Resource Management, Josh Bloch

• Improved Type Inference for Generic Instance Creation, Jeremy Manson

• Improved Wildcard Syntax for Java, Neal Gafter

• Multiline strings, latest revision, Ruslan Shevchenko

• Use "default" keyword for default visibility, updated to use "package" instead, Adrian Kuhn

• Elvis and Other Null-Safe Operators, Written by Neal Gafter and submitted by Stephen Colebourne

• Simple Resource Clean-up, Roger Hernandez

• Import Aliases for Classes and Static Methods, Phil Varner

• Lightweight Properties, David Goodenough

• 'This' type, Marek Kozieł

• Static Methods in Interfaces, Reinier Zwitserloot

• fold keyword, Gabriel Belingueres

• Multiple switch expressions and case ranges, Pinku Surana

• Simplified Varargs Method Invocation, Bob Lee

Traffic on the the list has been high, with lots of feedback and analysis leading to some revised proposals.

A few general comments on the proposals that have been sent in so far to help refine those proposals and improve future proposals before they are sent in.

The proposals submitted to Project Coin should already be well thought-through. The goal is to have in short order specifications approaching JLS quality, preferably with a prototype to help validate the design. The feedback on the list should be much closer to finding and illuminating any remaining dark corners of a proposal rather than fleshing out its basic structure. If a proposal does not cite chapter and verse of the JLS for parts of its specification, that is a good indication the proposal is too vague. All affected sections of the JLS should be listed, including binary compatibility and the flow analysis in definite assignment.

It is fine if someone posts to the list to solicit help writing a proposal for a given change.

Proposal writers should be aware of the size and scope parameters established for the project; for background see:

Criteria for desirable small language changes

• Guidance on measuring the size of a language change

Also, proposal writers should search Sun's bug database for bugs related to the change. The URL for the database is http://bugs.sun.com; Java specification issues are in category Java SE and subcategory specification. Of course the database is also searchable with your favorite search engine restricted to that site. Besides the evaluation field from the bug database, the external comment can often also have valuable insight into and discussion of alternatives to solving the problem or reasons why the problem shouldn't be solved.

As has already been happening on the list, authors and advocates of a proposal are responsible for responding to feedback and incorporating changes into any subsequent iterations of the proposal. For now, I think it is adequate to just send the revised proposals to the list. Only if there turns out to be frequent change would a more formal tracking system be warranted. Keeping such discussions on the list is important both to allow easy, centralized tracking of the proposal drafts and also for future language archaeologists who are curious about why a particular decision was made.

After a few iterations of feedback and refinements, the specification and compilation strategy should be sufficiently detailed to provide high-confidence that the proposal is practical and can be reduced to practice. For example, I think the initial proposal for the admittedly simple strings in switch change provides adequate detail on these fronts.

Sunday March 01, 2009

Project Coin: Proposal for Strings in switch

Below is a Project Coin language proposal form I wrote for Strings in switch; send any comment to the Project Coin mailing list.

PROJECT COIN SMALL LANGUAGE CHANGE PROPOSAL FORM v1.0

AUTHOR(S): Joseph D. Darcy

OVERVIEW
Provide a two sentence or shorter description of these five aspects of the feature:
FEATURE SUMMARY: Should be suitable as a summary in a language tutorial.

Add the ability to switch on string values analogous to the existing ability to switch on values of the primitive types.

MAJOR ADVANTAGE: What makes the proposal a favorable change?

More regular coding patterns can be used for operations selected on the basis of a set of constant string values; the meaning of the new construct should be obvious to Java developers.

MAJOR BENEFIT: Why is the platform better if the proposal is adopted?

Potentially better performance for string-based dispatch code.

MAJOR DISADVANTAGE: There is always a cost.

Some increased implementation and testing complexity for the compiler.

ALTERNATIVES: Can the benefits and advantages be had some way without a language change?

No; chained if-then-else tests for string equality are potentially expensive and introducing an enum for its switchable constants, one per string value of interest, would add another type to a program without good cause.

EXAMPLES
Show us the code!
SIMPLE EXAMPLE: Show the simplest possible program utilizing the new feature.

String s = ...
switch(s) {
 case "foo":
   processFoo(s);
   break;
}

ADVANCED EXAMPLE: Show advanced usage(s) of the feature.

String s = ...
switch(s) {
case "quux":
   processQuux(s);
   // fall-through

 case "foo":
 case "bar":
   processFooOrBar(s);
   break;

 case "baz":
    processBaz(s);
   // fall-through

 default:
   processDefault(s);
   break;
}

DETAILS
SPECIFICATION: Describe how the proposal affects the grammar, type system, and meaning of expressions and statements in the Java Programming Language as well as any other known impacts.

The lexical grammar is unchanged. String is added to the set of types valid for a switch statement in JLSv3 section 14.11. Since Strings are already included in the definition of constant expressions, JLSv3 section 15.28, the SwitchLabel production does not need to be augmented. The existing restrictions in 14.11 on no duplicate labels, at most one default, no null labels, etc. all apply to Strings as well. The type system is unchanged. The definite assignment analysis of switch statement, JLSv3 section 16.2.9, is unchanged as well.

COMPILATION: How would the feature be compiled to class files? Show how the simple and advanced examples would be compiled. Compilation can be expressed as at least one of a desugaring to existing source constructs and a translation down to bytecode. If a new bytecode is used or the semantics of an existing bytecode are changed, describe those changes, including how they impact verification. Also discuss any new class file attributes that are introduced. Note that there are many downstream tools that consume class files and that they may to be updated to support the proposal!

One way to support this change would be to augment the JVM's lookupswitch instruction to operate on String values; however, that approach is not recommended or necessary. It would be possible to translate the switches to equivalent if-then-else code, but that would require unnecessary equality comparisons which are potentially expensive. Instead, a switch should occur on a predictable and fast integer (or long) function value computed from the string. The most natural choice for this function is String.hashCode, but other functions could also be used either alone or in conjunction with hashCode. (The specification of String.hashCode is assumed to be stable at this point.) If all the string labels have different lengths, String.length() could be used instead of hashCode. Generally a String.equals() check will be needed to verify the candidate string's identity in addition to the evaluation of the screening function because multiple string inputs could evaluate to the same result.

A single case label, a single case label with a default, and two case labels can be special-cased to just equality checks without function evaluations. If there are collisions in String.hashCode on the set of case labels in a switch block, a different function without collisions on that set of inputs should be used; for example ((long)s.hashCode<<32 ) + s.length()) is another candidate function.

Here are desugarings to currently legal Java source for the two examples above where the default hash code do not collide:

// Simple Example
if (s.equals("foo")) { // cause NPE if s is null
 processFoo(s);
}


// Advanced example
{  // new scope for synthetic variables
 boolean $take_default = false;
 boolean $fallthrough = false;
 $default_label: {
     switch(s.hashCode()) { // cause NPE if s is null
     case 3482567: // "quux".hashCode()
         if (!s.equals("quux")) {
             $take_default = true;
             break $default_label;
         }
         processQuux(s);
         $fallthrough = true;
               case 101574: // "foo".hashCode()
         if (!$fallthrough && !s.equals("foo")) {
             $take_default = true;
             break $default_label;
         }
         $fallthrough = true;
     case 97299:  // "bar".hashCode()
         if (!$fallthrough && !s.equals("bar")) {
             $take_default = true;
             break $default_label;
         }
         processFooOrBar(s);
         break;

     case 97307: // "baz".hashCode()
         if (!s.equals("baz")) {
             $take_default = true;
             break $default_label;
         }
         processBaz(s);
         $fallthrough = true;

     default:
         $take_default = true;
         break $default_label;
     }
 }
 if($take_default)
     processDefault(s);
} 

In the advanced example, the boolean "fallthrough" variable is needed to track whether a fall-through has occurred so the string equality checks can be skipped. If there are no fall-throughs, this variable can be removed. Likewise, if there is no default label in the original code, the $take_default variable is not needed and a simple break can be used instead.

In a translation directly to bytecode, the synthetic state variables can be replaced with goto's; expressing this in pseudo Java source with goto:

// Advanced example in pseudo Java with goto
switch(s.hashCode()) { // cause NPE if s is null
case 3482567: // "quux".hashCode()
   if (!s.equals("quux"))
       goto $default_label;
   goto $fooOrBarCode_label;
  case 101574: // "foo".hashCode()
   if (!s.equals("foo"))
       goto $default_label;
   goto $fooOrBarCode_label;

case 97299:  // "bar".hashCode()
   if (!s.equals("bar"))
       goto $default_label;

   $fooOrBarCode_label:
   processFooOrBar(s);
   break;

case 97307: // "baz".hashCode()
   if (!s.equals("baz"))
       goto $default_label;
   processBaz(s);

default:
$default_label:
   processDefault(s);
   break;
} 

Related to compilation, a compiler's existing diagnostics around falling through switches, such as javac's -Xlint:fallthrough option and @SuppressWarnings("fallthrough"), should work identically on switch statements based on Strings.

TESTING: How can the feature be tested?

Generating various simple and complex uses of the new structure and verifying the proper execution paths occur; combinations to test include switch statements with and without fall-throughs, with and without collisions in the hash codes, and with and without default labels.

LIBRARY SUPPORT: Are any supporting libraries needed for the feature?

No.

REFLECTIVE APIS: Do any of the various and sundry reflection APIs need to be updated? This list of reflective APIs includes but is not limited to core reflection (java.lang.Class and java.lang.reflect.*), javax.lang.model.*, the doclet API, and JPDA.

Only reflective APIs that model statements in the source language might be affected. None of core reflection, javax.lang.model.*, the doclet API, and JDPA model statements; therefore, they are unaffected. The tree API in javac, does model statements, but the existing API for switch statements is general enough to model the revised language without any API changes.

OTHER CHANGES: Do any other parts of the platform need be updated too? Possibilities include but are not limited to JNI, serialization, and output of the javadoc tool.

No.

MIGRATION: Sketch how a code base could be converted, manually or automatically, to use the new feature.

Look for sequences of if ("constant string".equals(foo)) or if (foo.equals("constant string")) and replace accordingly.

COMPATIBILITY
BREAKING CHANGES: Are any previously valid programs now invalid? If so, list one.

All existing programs remain valid.

EXISTING PROGRAMS: How do source and class files of earlier platform versions interact with the feature? Can any new overloadings occur? Can any new overriding occur?

The semantics of existing class files and legal source files and are unchanged by this feature.

REFERENCES
EXISTING BUGS: Please include a list of any existing Sun bug ids related to this proposal.

5012262 Using Strings and Objects in Switch case statements.

URL FOR PROTOTYPE (optional):

No prototype at this time.