Friday Aug 18, 2006

Achieving Closure

Neal Gafter and Peter Ahe have both blogged about the new proposal for adding closures to the Java programming language.
One question that naturally arises is "what took you so long?".
I personally argued for adding closures since 1997/98. My blood pressure still rises measurably when I recall the response I got at the time: "Our customers aren't asking for it, so why add it?". Explaining that people usually ask for minor increments to what they know (e.g., they never asked for Java or for garbage collection) availed me naught.
I've learned that moving the world forward is an agonizingly slow process. Like the giant rat of sumatra, closures seemed to be a story for which the world was not yet prepared.
In the intervening decade, the widespread adoption of scripting languages has exposed a lot of people to the idea of first class functions. It has also shown that another argument used against closures - that they are too abstract for ordinary programmers - is just as imbecilic as the one about customer demand.
The fact that C# is moving in the same direction also helps create awareness of closure. We don't necessarily want to emulate C#, but I believe the case for closures stands on its own merits. The fact that C# is doing something is not an argument why we shouldn't.
Since the late 90s, I've brought the topic up now and again. At times, even I have reluctantly been convinced that it is too late, because we've done so many things that would have been easy with closures in different ways. This means the benefits aren't as high as in a language like Scheme, or Self, or Smalltalk. The cost is non-trivial, to be sure. There are still some who argue his point right now.
However, the need for closures keeps coming up. Nothing has been decided yet - we're starting a discussion here - but I am optimistic.

Comments:

I've read the proposal and believe that your draft spec will be very useful.
class ClosureTest {
    /** Implementing C#'s "using" construct */
    public static void using(Closeable clos, void() commands) {
        try {
            commands();
        } finally {
            try { clos.close(); } catch (IOException ex) {ex.printStackTrace();}
        }
    }
    
    public static void main(String[] args) {
        Connection conn = ...;
        using (conn) {
            RecordSet rset = ...;
            using (rset) {
                rset.next();
            }
        }
    }
}
Would function types have to be completely specified over and over again, as you must do with generics, or would some syntatic construct, like C++'s typedef, be added to the language?
/**
 * Alias for a function type for methods that
 * receive a single double parameter and returns
 * a double
 */
public alias double(double) function; 

Posted by Edson Watanabe on August 18, 2006 at 01:57 PM PDT #

Hi Gilad, Reading through your posts I sense frustration. I've come across your work in a previous life, and I must say it's Brilliant. From Strongtalk to Java must be a difficult transition :^). I take my hat off to you for still keeping up the faith, albeit from within the enemy camp :^). I've downloaded and played with Strongtalk and my impression was that it is comparable in speed to Java. I know very little about the JVM implementation, but given its heritage and the fact that Java was rushed to market I'm guessing that the current implementation doesn't provide much room for manoeuvre when it comes to dynamic features like closures etc. How much headroom is there in the current JVM for the kind of changes you speak of? Why not just port the Java language to the Strongtalk VM? (even as an experiment?) It seems to me that the VM as a platform is more important than the language long term. Wouldn't it be better to bite the bullet if needed and swap platforms? Or does the current JVM implementation still have aways to go? Be as frank as you can. Cheers, Paul.

Posted by Paul on August 18, 2006 at 02:28 PM PDT #

Gilad, I've had groups of undergrads working for several years to build a new version of Common Lisp (CLforJava) that is intertwined with Java (see the paper in the ILC 2005 conf). Implementing closures/continuations is the one big bear that's been sitting in the corner of the room. Having it built in would let us go forward with a rational implementation. Keep pushing! Jerry

Posted by Jerry Boetje on August 18, 2006 at 02:54 PM PDT #

[Trackback] Gilad Bracha , Peter Ahe and Neal Gafter blogged about adding closures . We should expect blog from James Gosling on the same topic real soon, since he is also one of the authors of the proposed specification. It looks like all of that ...

Posted by Euxx on August 18, 2006 at 02:57 PM PDT #

Yes! Closures are a wonderful thing. But so is call/cc. With call/cc one can program in comfortably in a procedural or functional style and be able to write event-driven code without having to turn one's code into spaghetti.

Check out OKWS, libasync for a taste of what resourceful people do with C++ to get closures and write event-driven applications in C++ in continuation passing style... No call/cc there...

Here's a question though: how does one make closures thread-safe? By making captured variables read-only?

Posted by Nico on August 18, 2006 at 06:01 PM PDT #

Methods as first-class objects are definitely needed in Java, I speak from personal experience where I worked around missing closures. Well, I succeeded at last, but the code-review was a nightmare, because the orignal idea got completly lost in the technical clutter of anonymous classes and nested hierachies.
Anyway, 3 remarks/questions:
1. Could a function of type T being a subtype of U simply defined by "T satisfies @Override when used in a subclass of a class defining U"? I think this is what a Java-programmer expects and is familar with.
2. How could a function type call itself? Please add an optional name or a keyword "self" to enable recursive closures
int(int) fac = (int n) faculty {return n>0?n*faculty(n-1):1;};
int(int) fac = (int n) {return n>0?n*self(n-1):1;};
3. Dropping the final requirement is nice and makes the code much simpler, but it makes me a bit uneasy. A function referencing its definition scope could thus be seen as a curried function; so you are not far from everything that is required to support comprehensive currying:
int factor=2;
doit(mult(factor,int b));// passes function that takes one int
doit((int b){return mult(factor,b)}); //same as above
doit((int b):mult(factor,b)); //dto.
Further idea: factor in the fragment above is meant to stay a reference (so that mult could modify the original value of factor), while
doit(mult(int c=factor,int b));
could pass factor as a parameter.

Posted by Carsten on August 19, 2006 at 04:10 AM PDT #

SUN never cared to implement Closures in Java.The reason why they are caring about now is Microsoft introduced them in C#.

So the funda is IF YOU WANT A NEW FEATURE IN JAVA DONT ASK SUN,(ITS TIME WASTE), BETTER ASK MICROSOFT, ONCE MS IMPLEMENTS THAT FEATURE IN THEIR LANGUAGE, THE NEXT YEAR SUN WILL CONSIDER THAT FEATURE. (This is also the quickest way of getting your favourite feature in JAVA).

Posted by kishore on August 20, 2006 at 01:40 AM PDT #

Hi, Gilad, I met you at the Lang.Net symposium a couple of weeks ago and lamented to you that Java did not support closures. At the time, you acted as if closures wasn't currently a priority. I was curious if my inquiry sparked Sun's closure proposal, since you are listed as the first author of the proposal.

Posted by Wesner Moise on August 20, 2006 at 07:49 AM PDT #

Hi Gilad. I want to say something before this feature is set in stone: Please don't add anonymous functions to Java, instead improve support for anonymous objects and make anonymous functions just sugar for a default type. I talk more about it here but the idea is: improve support for type inference and allow structural subtyping so anonymous objects can be created and made conformant to whatever types they implement. As much as I like functional programming (Haskell is my favorite PL, Java is just for my day job) I think that we should make the standard unit of abstraction (i.e. the class/object) simpler to use so the overhead is minimum, instead of introducing another, non-orthogonal, form of abstraction and making the language more complex.

Posted by Daniel Yokomizo on August 21, 2006 at 05:55 AM PDT #

C has pointer to functions, once we define one, we can assign real functions to it, pass it it as a argument to another functions. Is closure some thing like that?

Posted by pulihora on August 21, 2006 at 01:33 PM PDT #

Not quite. Since you mention C, for a flavour check out the GNU C extensions, specifically nested functions, and labels as values.

Specifically a closure is a really a {pointer to function, pointer to captured variables} tuple. The captured variables should be part of what you think of as the function call frame on the stack, so really this is {pointer to function, frame pointer}. GNU C nested functions do that, but the compiler/runtime constructs a function on the stack so as to not have to pass the frame pointer explicitly with the function pointer. Of course, these things can go out of dynamic scope! LISP closures, Perl5 closures, etc... can't go out of scope because they captured variables are on the heap, rather than on the stack. A program might consist entirely of heap-stored call frames, and some Schemes, etcetera are indeed implemented this way. See call-with-current-continuation (or "call/cc").

Also, think callbacks, on steroids.

Posted by Nico on August 21, 2006 at 03:00 PM PDT #

so can we get continuations in Dolphin, too? seems like we're most of the way there with this proposal.

Also, any chance of closures being serializable? Would there be any changes to the security model required to make that happen?

Posted by rvsrvs on August 21, 2006 at 03:49 PM PDT #

Some quick responses:
1. Wesner asks if the proposal was inspired by our conversation. Sorry, but I'm an old Smalltalker, and I've had closures for Java on my mind since I first saw java in 1995. So, no.
2. Are closures serializable? No, the Java serialization framework is inherently tied to the JVM's nominal type system, and cannot cope with anonymous code, because such code has no stable name across versions. Also, there are issues with serializing the surrounding class and the stack (and no, we are not doing continuations). Except for the stack, the issues are exactly the same for anonymous classes, which cannot be reliably serialized.
3. Aliases/typedefs are hardly an issue for function types. They are much more of an issue with generics. Any support for that would likely come through some enhancement of the import facility, and is a wholely separate discussion.
4. Paul asks if the JVM is up to supporting this. Well, without VM extensions it is rather lacking, but still doable. However, we are likely to add VM support, if any of this gets done.
As for the suggestion of just putting some sugar on anonymous classes, there are certainly people who argue that position. I think that's lame, but nothing is set in stone.

Posted by Gilad Bracha on August 21, 2006 at 05:48 PM PDT #

One more point: while closures are not serializable, we are investigating the possibility of classes that support function type interfaces. These could be made serializable.

Posted by Gilad Bracha on August 21, 2006 at 05:51 PM PDT #

If there will be really an additional method to create closures in Java, then we also need a feature which is available in every functional progamming language (and for a very good reason): Tupels.

Without tupels closures are quite useless because they often have to return multiple values. With inner classes those values can be stored in a field of the class and queried with a second method, but with the new closure proposal this won't work.

Sure, it's always possible to simply declare a new class which holds the return values or use the array-trick - but it's also always possible to use inner classes instead of the new closure proposal. So if simplification of coding is really the goal here, then tupels are a must.

Tuples are quite easy to implement and even if they (like the new closure proposal) aren't good for Java because they blur the way to program in the language and will kill Java eventually, it's simply a necessary complement to closures and will proposed nontheless earlier or later when people start to use the new closures (if it really - and sadly - gets realised).

Posted by Karsten Wagner on August 22, 2006 at 06:40 PM PDT #

Tuples are a very valuable construct and I'm all for them . They do synergize nicely with closures. We'll see if we do anything about tuples in Java. Of course, the claim that closures are useless without tuples is unadulterated drivel

Posted by Gilad Bracha on August 22, 2006 at 09:53 PM PDT #

I am in the camp that thinks inner classes are a superior concept. There advantages are:

  • You have one concept; a class
  • You can extend abstract classes, i.e. partial implementations, as well as interfaces
  • You can pass multiple methods and state as one object

Therefore I have logged a request for an enhancement to Java for shorter syntax for common operations. One of these common operations is anonaymous inner classes. Which as many people have pointed out have much in common with closures and first class functions. The aim of the proposed syntax changes are to remain backwards compatible with existing Java and provide a natural progression towards shorter syntax.

You can vote for and see detail of the proposal at:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6389769

The example from Neil's blog:

public static void main(String[] args) {
     int plus2(int x) { return x+2; }
     int(int) plus2b = plus2;
     System.out.println(plus2b(2));
 }
Would be with my proposal:
public static void main(String[] args) {
     var plus2b = Object() int e(int x) return x+2;
     System.out.println(plus2b.e(2));
 }

Which seems competative in terms of syntax and is a much less radical proposal and yet also has many other usage examples, including every variable delaration! Another example is, closure, instead of the current:

textField.addActionListner( new ActionListener() {
    public void actionPerformed( ActionEvent notUsed ) {
        textArea.append( textField.getText() );
    }
});

You could write with my proposal:

textField.addActionListner( ActionListener() ( notUsed ) textArea.append( textField.getText() ); );

Posted by Howard Lovatt on August 23, 2006 at 12:15 AM PDT #

Jaggregate (jaggregate.sf.net) thanks you much!

Posted by Paul Holser on August 23, 2006 at 06:19 AM PDT #

[Trackback] In comments to Gilad Bracha blog about adding closures to java language one of the commenters tried to suggest a shortened syntax for anonymous classes. So, he could write something like this: var plus2b = Object() int e(int x) return x+2...

Posted by Euxx on August 23, 2006 at 08:40 AM PDT #

[Trackback] I am going to assume you have seen the proposal . Essentially it is a parallel of anonymous inner classes with enhanced functionality. Developer vs library As with properties, the proposal addresses two sides which perhaps should be inde...

Posted by Tom Hawtin on August 23, 2006 at 02:40 PM PDT #

Post a Comment:
Comments are closed for this entry.