Search

Categories

Links

Referers

Full Disclosure

Aug 17 2006, 09:38:43 PM PDT »Java»Language Comments [42]
Some clever guys have written up a proposal on closures and been kind enough to put my name on it. I was just sitting in the room trying my best not to look too stupid ;-)

A closure is an anonymous function (aka lambda expression) in which all free variables can be accessed (even if they are not final). Furthermore, in the tradition of Smalltalk, the closure should be able to return directly from the method enclosing the closure definition (aka non-local return).

In the Java™ programming language, closures can be simulated to some extent with anonymous classes but these cannot access non-final local variables and have no way to exit the method containing the definition besides throwing an exception (which should not be used for control flow).

Closures have been a favorite of most Smalltalk freaks and has gotten renewed interest in languages such as Ruby.

Anonymous classes, although useful, can be a tad verbose and tiresome to type so this proposal must address the proper closure of free variables and non-local return as well as suggest a consice syntax.

Post a Comment:
Comments are closed for this entry.
Comments:

Nice. My 2 cents:

"Referencing Names from the Enclosing Scope" fails to define how non-final locals will be handled. Is there an UnmatchedEnclosingVariableAccess if the closure escapes its scope? Or will non-final locals be boxed?

I have a feeling that especially non-local control transfer should not be regarded as a must. I think people are pretty much used to the semantics of inner classes, they just hate the syntatic impact.

In doubt, a solution that is closer to "concise anonymous inner classes" than to "closures like in smalltalk" would still be immensely useful, even if they do not match the expectations of those who have seen the holy land. I'd definetely prefer it to "Closed, not feasible".

Matthias

Posted by Matthias Ernst on August 18, 2006 at 06:02 AM PDT #

I tend to agree. Here are the things that would make Closures be good enough for me:
  1. Concise syntax. Also part of this one is the fact that anonymous inner classes extend an interface or class, and so you start to need different interfaces for different scenarios. Plus, you have to implement/override a method, etc. With real closure support, this would go away. If I want to make a closure that takes parameters, and even a certain set of parameters, then let me do it, and without making a specific interface for that set of parameters. Super concise. Again, the word here is concise.
  2. Allow direct invocation of a variable to which a closure is assigned. With anonymous inner classes, you do something like: myReferenceVar.someMethod(...); With closures, I'd like to do this: myReferenceVar(...);
Hmm.. can't think of much else off the top of my head. Personally, I'm willing to give up the other more "pure" smalltalkish stuff. I also think this anonymous inner class style approach would lend itself better to the coming and present reality of multi core (mutli threaded) development.

Posted by Andy DePue on August 18, 2006 at 08:47 AM PDT #

It is entirely possible I am missing something so feel free to correct me... but, I must honestly say I am not a big fan of anonymous functions or the syntax you decided on using. I feel they degrade readability for not enough gain. I *am* in favor of being able to use anonymous classes without the restriction of the "final" keyword, but I don't favor going beyond this. Anonymous functions look a lot like function pointers to me and I feel that these are best replaced by interfaces in Java in their current form. Isn't there a way to remove the "final" restriction from anonymous classes without these kinds of syntax changes?

Posted by Gili on August 18, 2006 at 10:08 AM PDT #

Peter, I allowed myself to post a link to this blog to Javalobby. You might want to take a look at some of the responses:
http://www.javalobby.org/java/forums/t77733.html

Thanks,
Dmitri

Posted by Dmitri Trembovetski on August 18, 2006 at 10:29 AM PDT #

Peter, I can hardly believe you are going to come up seriously with the stuff like non-local transfer and exception upon access of local variables of statically enclosing method. I can't stop thinking this proposal is a joke.

Posted by Eugene Vigdorchik on August 18, 2006 at 12:21 PM PDT #

Hi Peter, I really liked this proposal, but I've got a few questions:
  • Are or can functions/closures (be made) <code>Serializable</code>?
  • In practice, will functions/closures classes extend a specific supertype, such as Function?

Posted by Michael Nascimento Santos on August 18, 2006 at 02:04 PM PDT #

Are or can functions/closures (be made) Serializable?
I doubt it. Webservice and the like are probably a better solution than serializable closures.
In practice, will functions/closures classes extend a specific supertype, such as Function?
Most likely.

Posted by Peter von der Ahe on August 18, 2006 at 02:31 PM PDT #

Eugene, we are not joking. The concepts are well known from Smalltalk and Ruby.

Posted by Peter von der Ahe on August 18, 2006 at 02:34 PM PDT #

This sounds really great. But it still quite nebulous about how generics will interact with closures. I think this is a fundamental step towards Java's future.
Maybe it's time to think about including type alias, just like typedef in in C/C++. Just think what would be like using C++ templates and function pointers without it.
Correct me if I'm wrong, but as it stands from the PDF, it would not be possible to implement a generic currying method.

Posted by Rodrigo Kumpera on August 18, 2006 at 02:50 PM PDT #

[Trackback] Gilad Bracha , Peter Ahe and Neal Gafter blogged about adding closures . We should expect blog from James Gosling on the same topic real soon, since he is also one of the authors of the proposed specification. It looks like all of that ...

Posted by Euxx on August 18, 2006 at 02:57 PM PDT #

Hi Peter,

I should read the proposal in detail of course, but if we're allowing lambda-calc-like things, can we please also support Curried functions, ie partially evaluated with one or more of the arguments to a multi-arg function.

Just think what HotSpot could do in generating the specialised version of a function! Even in Standard ML days 20 years ago I found it a useful optimisation.

(For the record, what I call a Curried function is a specialised partial evaulation of say a new function "twice" = "f 2" where f is "x * y", ie takes two args.)

Occasionally I simulate this in C++ or Java, but to have direct support for it in the language and the JIT/runtime would be fab. I do occasionally have dreams about it. Yes, sorry, I do!

Rgds

Damon

Posted by Damon Hart-Davis on August 18, 2006 at 04:23 PM PDT #

We are thinking about how this interacts with generics as well as type aliasing (the latter being independent of closures).

Posted by Peter von der Ahe on August 18, 2006 at 05:12 PM PDT #

Closures in Java would be <strong>great</strong>. Java needs language enhancements like that.

Posted by Tom on August 18, 2006 at 08:15 PM PDT #

I like it, I'm confused by the big words and I'm not sure about the brevity of the syntax (I suppose strong typing doesn't help that) but if I can do the following I'm happy:

 boolean hasOranges = 
    list.detect(
       boolean ( Object item ) {
          return item.equals( "orange" );
       } );

After all, external iterators are a sign of weakness in an object oriented language. ;)

http://www.pragmaticprogrammer.com/ppllc/papers/1998_05.html

Posted by Robert DiFalco on August 19, 2006 at 08:22 AM PDT #

How about allowing methods to be called/referenced directly? Then closures wouldn't really be needed (from my naive standpoint). Something like:

// # used here to help the compiler.
Method m = myObject#myCall (Object, String);

Then:

public void iterateOver (Method m, List myitems)
{

   for (Item item : myitems)
   {

       // It would be nice to use . here but
       // # would also be possible to help the
       // compiler...
       // Would also be useful to have a 
       // "callMethod" method that allows direct
       // access rather than passing an array
       // of args.
       m.myCall (item, 'hello world'); 

   }  

}

It would also be nice to reference methods in a Class directly as well. For example:
public void doSomething (Class c)
{

   c#eval ('hello world');

}

Posted by marcusk on August 19, 2006 at 05:58 PM PDT #

Actually missed out the passing of the object to call the method on (Sunday morning stupidity), so the reference getting should be:

Method m = myObject#myCall (String);

And item would be the object to call the method on.

Posted by marcusk on August 19, 2006 at 08:01 PM PDT #

Very. Bad. Idea. Very. This proposal is really a joke like Eugene Vigdorchik pointed out. Why could anybody really propose adding something to a language the language already contains? Add some syntactic sugar is really enough to make the use of closures (which already exists with annonymous inner classes) more convenient. So add two things: transient local vars: <code> transient int val = 10; val = 20; </code> which is rewritten by the compiler to: <code> final int[] val = new int[] { 10 }; val[0] = 20; </code> With this you can make every local write-accessible from an local inner class, it's really only simple syntactic sugar and it shows that some allocation and indirection is happening with 'val'. And then make the callsite a litle bit more comfortable, i.e. allow <code> new ClosureType()(int x) { ... } </code> instead of <code> new ClosureType() { void eval(int x) { ... } } </code> (the above would work for every class with only a single method with a (int x) signature). With those both simple additions, closure usage would be a little bit ore comfortable in Java without having to create a new huge extension which conflicts with millions of lines of existing code and would make the language even more compliated.

Posted by Karsten Wagner on August 21, 2006 at 06:31 AM PDT #

I must say that I think this is an excellent idea. The lack of first-class functions in Java is really evident after spending some time in C# /Ruby/Javacript (gasp) -land. One thing I miss in Java in particular is the delegate concept.

If you think that you can mimic the same behavior with annonymous inner classes, then you've never really worked with true first class functions.

Posted by Marcus Breese on August 21, 2006 at 11:12 AM PDT #

Agreed with Marcus Breese - delegates in C# are a pretty minimal implementation of this kind of thing, but they're a feature I really miss in Java. Sure, the same thing can often be done with anonymous classes, but the syntax for those is so much more clumsy.

Posted by Simon on August 21, 2006 at 10:36 PM PDT #

Annonymous inner classes don't mimic the behaviour, they work practically identically! I've worked in Ocaml so I know about the advantages of closures, but Ocaml is a full fleged functional programming language with type inference and currying. You can't expect the same gains if you put closures into Java without changing Java otherwise completely. But yes, sometimes closures can be usefull in Java too - but then you already have it (under then name 'annonymous inner classes' which need a little bit more typing but also have advantages over this proposal). I've written more about this in my blog.

Posted by Karsten Wagner on August 22, 2006 at 05:32 AM PDT #

Closures aren't just syntactic sugar for anonymous inner classes. They might be implemented that way, but conceptually they are very different. - There is an important conceptual difference between

return new Iterator(){
hasNext(){...}
next(){...}
...
};

and

obj.addWhateverListener(new WhateverListener(){
public void whatever(Argument a){...}
});

To me, the first example is what anonymous classes should be used for - things which are used as objects, not overweight function pointers or callbacks. The other important difference between anonymous classes and closures is that closures have full access to the enclosing scope's variables. Karsten Wagner's suggestion of putting transient on local variables, which are then syntactic sugared into 1 element arrays seems a good one to me, although I'd say just let the compiler detect the usage as it does now and do the conversion on its own.

Basically, you can do closures in current java, but you have to use the extra verbose anonymous inner class syntax, which specifies all sorts of unneeded information - in the listener example, you state that this is a "whatever" callback 3 separate times - once in the function call seems enough to me. You also have the odd final requirement - why not just have the exact same effect as 1 element arrays?

Posted by Ben on August 22, 2006 at 06:49 AM PDT #

Yes, closures are very useful, but can we improve the readability of the proposed syntax? "Thou shalt not add reserved words to the language" is an honorable goal, but it has its costs.

Posted by EricR on August 22, 2006 at 02:59 PM PDT #

I have logged a request for an enhancement to Java for shorter syntax for common operations. One of these common operations is anonaymous inner classes. Which as many people have pointed out have much in common with closures and first class functions. The aim of the proposed syntax changes are to remain backwards compatible with existing Java and provide a natural progression towards shorter syntax.

You can vote for and see detail of the proposal at:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6389769

The example from Neil Gafter's blog:


public static void main(String[] args) {
     int plus2(int x) { return x+2; }
     int(int) plus2b = plus2;
     System.out.println(plus2b(2));
 }
Would be with my proposal:

public static void main(String[] args) {
     var plus2b = Object() int e(int x) return x+2;
     System.out.println(plus2b.e(2));
 }

Which seems very competative in terms of syntax and is a much less radical proposal and yet also has many other usage examples, including every variable delaration! Another example is, closure, instead of the current:

textField.addActionListner( new ActionListener() {
    public void actionPerformed( ActionEvent notUsed ) {
        textArea.append( textField.getText() );
    }
});

You could write with my proposal:

textField.addActionListner( ActionListener() ( notUsed ) textArea.append( textField.getText() ); );

What do people think - is this a less radical alternative, yet achieves more? In particular simplifying many declarations not just closures and first class functions.

Posted by Howard Lovatt on August 22, 2006 at 10:26 PM PDT #

Jaggregate welcomes this proposal with open arms, and a willingness to help the proposal round into shape. Yay!

Posted by Paul Holser on August 23, 2006 at 06:22 AM PDT #

Peter, I love closures, closures are great, and closures is the biggest missing feature of the Java language. But I think that closures should have different semantics. Closures should be implemented as interfaces implementations. So I could write (2 examples):
Runnable r = { System.out.println("hello"); }

transform(list, (T o) { return t.getColor(); });
This should be equivalent to:
Runnable r = new Runnable() {
    void run() { System.out.println("hello"); }
}

transform(list, new Transformer<T> {
    U transform(T t) { return t.getColor(); }
});

So closure is implementation of single-method interface. In this case we aslo solve big problem with dynamic method invocation (there is no dynamic invocation), no additional space allocation when converting closure to interface (closure is already an interface).

Posted by Stepan Koltsov on August 24, 2006 at 07:46 AM PDT #

"o closure is implementation of single-method interface. In this case we aslo solve big problem with dynamic method invocation (there is no dynamic invocation), no additional space allocation when converting closure to interface (closure is already an interface)." I just wnat to say, if closure is a implementation of a single-method interfacem It also have to malloc a space fot it allocation(== create of a new object)

Posted by fun on August 24, 2006 at 11:58 AM PDT #

Just wanted to add I am in favor of Stepan Koltsov's syntax. If you do insist on adding enclosures (I'm still not a big fan) then his syntax seems to make the most sense to me.

Posted by Gili on August 24, 2006 at 02:36 PM PDT #

Closures are great !! Sometimes we have to create one-single-method interfaces just to have a callback. We all always code something like this: <code> public static interface Callback{ Object execute(Object param); } </code> BeanShell already has closures... guys at JCP could check it. I think some syntactic sugar to anonymous inner classes and arrays to acess local variables are the starting point.

Posted by Cláudio Gualberto on August 24, 2006 at 04:45 PM PDT #

Stepan Koltsov suggestion of simpler syntax is similar to my RFE:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6389769

The main difference between his propsal and mine is that he is adding some new syntax, I am proposing just making current compulsory bits optional. His examples, with the syntax I suggested are:

Runnable r = Runnable() () System.out.println( "hello" );

transform( list, Transformer< T >() ( t ) return t.getColor(); );

I think there is considerable advantage in just making the syntax shorter, rather than introducing new concepts that have at least some overlap with existing facilities. Furthermore innner classes have advantages:

  • One concept; the class
  • Can extend interfaces or classes
  • If extending classes, you can inherite implementation
  • You can have multiple methods and fields in one object
  • You have dynamic dispatch

In summary: closures are a poor man's inner class. Move on - it's a backward step.

Posted by Howard Lovatt on August 24, 2006 at 09:28 PM PDT #

Oops, the runable example should be:

var r = Runnable() () System.out.println( "hello" );

Posted by Howard Lovatt on August 24, 2006 at 09:33 PM PDT #

As an old functional programmer, I think first class closures sounds like the best thing since sliced bread (and a one-up on the delegates of That Other Language).

Mentioning delegates, it would seems consistent to allow turning methods into functions (i.e., delegates), e.g., by making an expression like (object.methodName) evaluate to a function. Instant delegates ... except that it can clash with a field of the same name, so some other syntax :(

There's one thing that I see missing: Recursive closures. Is there a way for a closure function to call itself? Perhaps something similar to function expressions in JavaScript (ex. (function omega(){omega()})(), where the name of the function is only visible inside the body).

Another thing I'd like to see for closures is proper tail recursion, something the JVM will probably have a very hard time supporting with its current feature set. Or maybe I should just go back to ML :)
Regards
/L

PS. and I object to the word "closure" being used for the syntactic construct. I would personally prefer that being called a "function expression", and the value of that expression is then a closure. :)

Posted by Lasse Reichstein Nielsen on August 25, 2006 at 03:50 PM PDT #

hi Peter, I would like to ask are there any plan to change the JDK(propably the API) to support closure. Since we can't add any Collection.each() to the existing API. what is ur plan to add closure support to the collection? IF u can provide more usful use cases, I think more people will support ur proposal.

Posted by Francis on August 26, 2006 at 09:26 PM PDT #

As Francis indicates, closures are limited if you can't use them on collections. I'm sure that this has been thought about, but no-ones talking about it. I wrote up one solution, extension methods, in my blog

Posted by Stephen Colebourne on August 27, 2006 at 03:45 AM PDT #

There's a third group as well: those who understand closures, have worked with closures in other languages, and don't like the propsed Java closures as they think it's more complex than it has to be and also it's too late to have them in Java cause methods like each, etc. that use closures won't be added to Collection classes, etc. as that would break compatibility...

Posted by Behrang on August 27, 2006 at 09:37 AM PDT #

>> they think it's more complex than it has to be Hard to get simpler than...
 
    <return_type> ( <args, ... )

I would only add the following variant. Given:
 
     public Object detect( boolean ( Object ) )

You should be able to invoke it as:
 
     found = list.detect( 
        boolean ( Object each )
        {
           return each.equals( search );
        } );

I think that is the standard case. But you should also be able to do the following:
 
   interface Something
   {
       boolean matches( Object arg );
   }

   found = list.detect( aSomething.matches );

Seems pretty simple to me. >> it's too late to have them in Java cause methods like each, >> etc. that use closures won't be added to Collection classes, >> etc. as that would break compatibility... Why would it break compatability? R.

Posted by Robert DiFalco on August 27, 2006 at 01:19 PM PDT #

>>Why would it break compatability? R.

Because if I wrote a sub class of Collection (say something returned from a method that has to hold onto my object's state, like Map.values or Map.keySet), then my subclass of Collection needs to implement the method. This is one of the problems of using interfaces + single inheritance.

The solution for all of this sort of utility method is stuff like the Collections class, which holds all the methods that only need the Collection (or List/Map/Set) to be implemented. Each would have to be implemented like this, but it makes these utility methods second class citizens, which is why the extension methods someone mentioned above might be a good thing to include

Posted by Ben on August 27, 2006 at 01:26 PM PDT #

Maybe there is a way you can change existing APIs to accept function types without breaking backward-compatiblity. I've written a short description of my suggestion for Anonymous interfaces and methods.

Posted by Avner on September 08, 2006 at 12:51 PM PDT #

Peter, I've added an RFE request to add traits to java instead (internal review ID of 790230). As I've defined them they seem to be able to handle most of the use cases for closures and allow reuse of implementation without the downsides of multiple inheritance and with a reasonable syntax to boot.

Posted by Mike Atkinson on September 09, 2006 at 01:48 PM PDT #

Hi @Ben I have written some tests and I can't see how backward compatability will be broken if Collection interface is modified. Sure everyone says it will but it won't! (Can someone enlighten me with actual code and not simply saying thats the way it works) Sure any new code will not be runnable in an older JVM because of class version stamp but that already happens. Any old code will not be able to call any new methods defined. That already happens with new 1.5 interfaces such as Queue. But it will still start up without any errors. We are not removing functionality from the Collection interface but rather extending it. This does not break backward compatability, existing clients can still call and use the new code as long as we don't specify the target higher than our target platform. Summarise: existing code will be binary compatible with the new changes, existing code will not be source compatible but this is already so (eg existing code (say Java 1.4) extends LinkedList and now try to modify it using Java 5 libraries; it will be broken)! Am i wrong, misguided or both :)

Posted by snoobab on September 12, 2006 at 02:12 AM PDT #

Well, I know it won't be compile-compatible if you add a new method to the Collection interface (you may still be able to modify it to a limited extent in other ways though) - you won't have implemented the method, and will therefore get an error. (If you always extend from AbstractCollection you might not run into problems, but you can't assume that).

What will happen if you try to use the old class files, I have no clue, but I would guess that it either breaks on the class verifier or something when loading, or throws a NoSuchMethodError when you try to call the new method.

Basically I'm saying that if I have (pretending Collection has nothing but add and get for simplicity)

public class Something {
       public Collection<Thing> getThings(){
       	      return new Collection(){
	      	     public void add(Thing it){ /* add it */}
		     public Thing get(int i){ /* return something */}
	      };
       }
}
and then Java version x+1 adds the "void doStuff()" method to Collection, then my Collection subclass won't compile because it doesn't implement doStuff

Posted by Ben on September 12, 2006 at 05:54 PM PDT #

Even if the JVM doesn't support tail recurion, we need syntax to support recursive closures, don't we?

Posted by tuc on September 21, 2006 at 11:00 PM PDT #

I have written an example using closures to demonstrate how clearer and smaller code can be written using function types and closures. Have a look at my post on my blog titled: Closures in Java. Thanks Nishant Saini Daffodil Software Limited

Posted by Nishant Saini on September 24, 2006 at 06:34 AM PDT #

Java is a trademark of Sun Microsystems, Inc.
Copyright © 2006,2007 Peter von der Ahé