Thursday, December 13, 2007

What flavor of closures?

I just attended Josh Bloch's presentation at JavaPolis, where he asks the community whether they want Java to support function types, or if they'd prefer that people write these things the way they do today. His examples are carefully selected from the most twisted of the test suite. Compiler test suites are a good place to find the most twisted but unrealistic uses of any given language feature. I thought it would be interesting to look at the question in the context of a real API. You probably know my opinion, but just to be clear, here is an excerpt from Doug Lea's fork-join framework

/**
 * An object with a function accepting pairs of objects, one of
 * type T and one of type U, returning those of type V
 */
interface Combiner<T,U,V> {
  V combine(T t, U u);
}
class ParallelArray<T> {
  /**
   * Returns a ParallelArray containing results of applying
   * combine(thisElement, otherElement) for each element.
   */
  <U,V> ParallelArray<V> combine(
    ParallelArray<U> other,
    Combiner<? super T, ? super U, ? extends V> combiner) { ... }
}

And the equivalent code ported to use the features of the closures spec:

class ParallelArray<T> {
  /**
   * Returns a ParallelArray containing results of applying
   * combine(thisElement, otherElement) for each element.
   */
  <U,V> ParallelArray<V> combine(
    ParallelArray<U> other,
    { T, U => V } combiner) { ... }
}

The question Josh asks is this: which version of this API would you prefer see?

The point he makes is that function types enable (he says "encourage") an "exotic" style of programming - functional programming - which should be discouraged, otherwise the entire platform will become infected with unreadable code. Although functional programming is just as possible with or without function types - they are just shorthand for interface types, after all - Josh prefers the language provide syntactic vinegar for these techniques.

Part of his talk was about the problems of being able to use nonlocal return by default in a closure. See my previous blog post for a description of how this theoretical problem won't exist in the next version of the spec, and doesn't exist in the prototype today.

Finally, Josh showed that if you want to use something like eachEntry to loop over a map, and you want to be able to use primitive types for the loop variables, autoboxing doesn't work and you'd have to define 81 different versions of the eachEntry method (one for each possible primitive type in each position). That's true, just as it's true that you'd have to define 81 different versions of the Map API if you want to be able to handle primitives in them. If it turns out to be a good idea to make autoboxing work for the incoming arguments to a closure, that is a small tweak to the closure conversion. These kinds of issues can be addressed in a JSR.

Josh's vision for an alternative is Concise Instance Creation Expressions along with adding a moderate number of new statement forms.

45 comments:

Macneil said...

I don't think function types are a good fit for Java. Java programmers are used to interfaces for anonymous inner classes and supporting closures should complement this style, rather than be something kind-of-like-but-mostly orthogonal to something already there.

Forcing the interface to be there also provides an excellent location for people to turn to to read Javadoc comments.

Even in your example, note how the comments for Combiner don't have a parallel in the other example, nor is there even an obvious place for it to go. And now think about that in the context of a very large library. I would love to open up the Javadoc page for the "Combiner" interface and click on the "Uses" link to show me all methods that take it as an argument, or return it as a result.

It should also be clear that the function types have limitations when it comes to throws declarations. If you need to add another exception type to your function type, in one you just have to change the interface and let the compiler tell you (via error messages) what else needs to be changed. With the function types, however, it's up to you to find all possible uses, which is very hard considering the function type style doesn't even give you anything you could grep for! (I.e., a name! This is related to the "Uses" example above.)

The function types certainly looks better, but I could imagine sugar improving the other form. Given the lack of something to grep for, I see function types being a liability in the long run. Josh's desire to add "vinegar" isn't something I agree with. Make the syntax easier, but keep the structure named so that it can be centrally specified, documented, and searchable.

Ben Maurer said...

Hey Neal,

I hope all is going well back at the Googleplex :-). Say Hi to the rest of the folks for me.

I honestly have a hard time understanding how anybody can advocate the first form of closures. The killer seems to be the "? super T" clauses. First, the syntax for this form is a bit strange -- every year when we have students at CMU implement something with generics, they get confused by the meaning of such clauses.

Josh's syntax also requires that somebody really understand how the type system works in order to correctly write the signature.

I sure hope that a reasonable solution can be found...

Stefan Schulz said...

I'm not sure, if this is a useful comparison, as the definition part most times is not the problematic one. As CICE implies, it's focus is on creating SAMs concisely, i.e., when invoking a method that takes a SAM, it nearly only takes the method body to be passed.

Rémi Forax said...

Functional programming => unreadable code,
i agree if the VM allows tail call recursions. Ruby or Groovy are readable.

Anyway, i hope you will be able to reach a consensus with josh about closure.
Where are click and hack ?

Brian Slesinsky said...

The declaration would be more readable as:

<U,V> ParallelArray<combine(
ParallelArray<U> other,
{ (T, U) => V } combiner) { ... }

The parentheses are a strong hint to a Java programmer that this is a list of parameters to a function call, and it clarifies that the => token is of lower precedence.

Francis said...

Go Big or Go home..
Rather then the syntax sugar, I think a true closure is a better solution for now and for the future.

Francois Proulx said...

Hi Neal,
I've been reading for blog for a while now, played a bit with your JSR166y reimplementation and loved it! I am of the opinion that we really should have closures for Java, as a JSR initially based on your ideas. On the topic of confusing programmers with new syntax and "exotic" (functional) programming style ?!? I really think that have closures like I saw them in your spin on JSR166y is much clearer to read. Besides, closures will mostly be used where we are currently using anonymous classes. There are only a handful of cases where "casual" programmers (those who might be confused with exotic styles...): adding Swing listeners, defining a Thread's Runnable.... In these instances, I really don't think it is more complex to read a closure. Sure, it might start to be a bit more exotic when you start dealing with <T, U, V> type method declarations, but how many developers actually need to play with the API-side of this? Most of them will use JDK-provided APIs (such as JSR166y).

Hooray for closures! In the meantime, as a Sun Campus Ambassador in Montreal, I'm evangelizing my friends to closures so that they'll be fighting with us when the JSR comes around. :-)

Keep up the good work.

Mark Mahieu said...

Doug Lea had some interesting comments about the fork-join framework and its supporting interface types, in relation to closures.

It's worthwhile comparing the number of classes and interfaces in the closures and non-closures version of fork-join. There's something like a 50% reduction in the closures version... that's a big signal-noise ratio improvement.

Fork-join isn't the only library which would benefit in that manner. I've seen plenty of decent libraries end up ignored or under-utilized due to being swamped by myriad variations of general interface types like Combiner, Function, Mapper, Generator etc. I've ignored or under-utilized them myself :-)

After all, what does the javadoc for the Combiner interface tell you that the corresponding function type doesn't say in place, as part of the method signature, rather than somewhere else on another page in the javadoc? What can it tell you? It can't tell you much about what it does, because the user of the API is providing that all important detail when they create an implementation. It could tell you how it will be used by the combine() method, but surely that information belongs as part of combine()'s documentation.

True, with grep or an IDE you could search for all uses of this interface since it has a name, but how often does anyone do that with such a general interface? I certainly don't recall ever doing it with something like Runnable. Far more likely I'd want to search for uses of the combine() method itself.

There's also an interesting sentence in the CICE proposal:

"At the same time as this facility is added to the language, it would make sense to add a few more single method interface types, such as Predicate, Function, and Builder"

Presumably not to support any "exotic" styles of programming though... ;-)

Howard L said...

Maybe a better approach would be to copy Scala:

/**
* An object with a function accepting pairs of objects, one of
* type T (or a super class of T) and one of type U (or a super class of U), returning those of type V (or a subclass of V)
*/
interface Combiner<super T,super U,extends V> {
V combine(T t, U u);
}
class ParallelArray<T> {
/**
* Returns a ParallelArray containing results of applying
* combine(thisElement, otherElement) for each element.
*/
<U,V> ParallelArray<V> combine(
ParallelArray<U> other,
Combiner<T,U,V> combiner) { ... }
}

This gives the clarity of the current Java approach. In particular the separate type Combiner that clearly documents intent and yet is less verbose. Presumably this is why Scala went this way.

Anonymous said...

Hi Neal,

I understand that closures have to support a lot of language functions, but what I really want them for is for this:

JButton button = new JButton();
button.addClickListener{
button.setText("Hello");
}

compared to:

final JButton button = new JButton();
jbutton.addClickListener(new Actionlistener(){
public void actionPerformed(ActionEvent e){
jbutton.setText("Hello");
}
});

Is this possible with your new language changes? From my perspective this would seem trivial to implement, as I have written a preprocessor to do this. We don't use it these days because of ide usage which would not understand it. The javac compiler should be smart enough to be able to figure this out though shouldn't it?

This is going be a very common use in "business programming" - simply reducing the code needed hooking up complex interfaces or event driven systems.

I have seen a lot of very complex code bandied around with closures, I think there must be a way to make it nice for "simple" uses as well. Perhaps you should do an article with before and after's for a whole lot of simple and advanced use cases?

e.g. MouseListener (5 methods to implement), ActionListener, Predicates with collections (convert for to functional style) etc.

Neal Gafter said...

Howard: adding declaration site variance to the language is definitely a far deeper change to the type system than function types, which are an abbreviation for something the type system already understands.

Neal Gafter said...

Anonymous: yes, our spec allows you to register callbacks much more concisely:

button.addClickListener({ =>
button.setText("Hello");
});

Or, using the control invocation syntax:

button.addClickListener() {
button.setText("Hello");
}

Josh Bloch said...

Neal says "The point [Josh] makes is that function types enable (he says "encourage") an "exotic" style of programming - functional programming - which should be discouraged, otherwise the entire platform will become infected with unreadable code." He is (perhaps unintentionally) putting words in my mouth. The exotic style that I'm talking about isn't just functional programming, it's *higher-order* functional programming, and I *don't* think it should be discouraged. On the contrary, I think it should be encouraged. In Scala, Scheme, Haskell, ML, and other appropriate languages. I think it has a severe "impedence mismatch" with Java.

More seriously, I don't think Neal does my presentation justice when he cites the one quote in the previous paragraph. I gave a whole slew of reasons why Doug's Fork-Join framework is better off with traditional nominal types. Simply put, nomal types (iterfaces) are richer: they have names, which enable IDE autocompletion and grep; they get high quality javadoc, which allows for semantic contracts, such as "Ops.Reducer must be associative." This topic is treated in more depth on slides 29-35 of the talk.

Neal: I respectfully encourage you to post a link to my talk in this blog entry, so your readers can look at the slides and decide for themselves.

Thanks,

Josh

Cow_woC said...

In my opinion we need to clean up Generics before even considering closures. I am still not convinced that closures benefit more than they harm. If Generics' existing syntax is bad (and I agree it is) then we should clean it up, but this has nothing to do with the need for closures.

HT said...

Neal,

{a,b=>v} syntax is not intuitive. I don't know where you got that.

Why not credit LISP and use something like:

{lambda (args) { body}}


Thanks,
HT

Anonymous said...

Hi Neal,

I am very pleased about the syntax of closures for the simple case, thanks for explaining. +1 from me!

aehrenr said...

Define closures in the sense of Mr. Bloch. They should be simplification of the syntax of anonymous inner classes. Instead of discussing whether to add enormous complexity to the language we should discuss how to clean up this calamitous generics with wildcards definition that drives away developers from Java. Java is an all-round language for the rest of us and has been an extremely handy straightforward tool. For everything else Sun should give first class support for Groovy and Scala in Netbeans and documentation. Then everyone has what he wants.

John M. said...

I have always felt that the language complexity introduced by generics was not worth the benefit. Generics solved a problem that I did not have with pre-generics versions of Java. The example cited in this blog shows why we need closures to simplify a problem with generics. It is too late to go back to remove generics, but additional language changes along the line of closures seems like the wrong way to go.

Axel Rauschmayer said...

I agree with Bloch's hesitation to add too many little new and potentially uncohesive new features to Java. On the other hand, one could add top-level functions to Java (see (6)) and while this would make Java more complex, this feature would have its own internal cohesiveness and should thus be easy to learn (examples of where this works are languages that mix classes and functions, such as Common Lisp, Python and Javascript).

Maybe it makes sense to separate concerns: I've ordered the following points by increasing complexity.

(1) Easy creation of code blocks: If that is all one wants then Bloch's proposal is the way to go, as it adds very little technical and conceptual burden to Java.

(2) In-place declaration of code block signatures: Sometimes a parameter type does not need a name. It would be nice if one weren't forced to give it a name then. If we go with Bloch's proposal, is having a few pre-defined generic interfaces (one for each arity?) enough here? Or do we need to introduce a new Java construct (similar to what Gafter does)?

(3) Looping: What is a minimal solution for loops? Just break and continue? The I still like the idea of throwing exceptions

(4) Non-local returns: Non-local returns seem to me to be orthogonal to loop support. They would even be useful on their own, as an *explicit* construct: every now and then it would be nice if a method I invoke could return a value for me.

(5) Exception type parameters: I haven't thought about this feature yet, but it seems to make sense on its own, too.

(6) First-class functions (as a counter-argument to Bloch's proposal): Shouldn't closures be something that has nothing to do with interfaces in the long run? Especially for performance reasons it feels strange that a closure is created as a heap instance. But with Gafter's proposal we are entering a completely new world. I don't think it makes sense to do this half-heartedly; in the long run there should also be top-level functions. Top level functions, especially in conjunction with multiple dispatch, neatly solve some single-dispatch design problems such as the visitor pattern or methods whose parameters are collaborators (then the method should belong to all collaborators at the same time, finding a single class for it is impossible). An example of the latter case are binary operators.

(7) Method literals: These are very complementary to closures and really needed for binding APIs. They are also proof that first-class functions make sense.

jvb said...

I was at javapolis this year where Josh gave his presentation about the "closures Controversy", In the presentation he refers to a presentation "the feel of java" by James Gosling (OOPSLA 1996). One of Josh's arguments is that the closures proposal from Neil and others contradicts this "feel of java" as James Gosling described it in his talk in 1996. What Josh didn't mention at javapolis is that James Gosling himself is one of the authors of the "BGGA (http://www.javac.info/closures-v05.html)" proposal that he believes to contradict with "the feel of java".

Does this mean that James Gosling thinks differently about how java should feel these days, or does it mean that Josh misunderstood what James Gosling meant with "the feel of java"?

Cow_woC said...

There is an active discussion going on here: http://www.javalobby.org/forums/thread.jspa?threadID=104586

Armin Ehrenreich makes a very good point:

"... imagine a language where you can define such control structures at will. This for sure would be completely unreadable, or so to say operator overloading on steroids"

The hidden danger of closures will result in Java code that is not readable by the lay developer. Not because of its syntax, but rather because of the fact that it lets you overload such control structures. In a sense, everyone will have their own version of the language which will make code harder to maintain across different developers.

I am personally in favor of a controlled form of operator overloading and closures. Let the community evaluate on a case-by-case basis when it makes sense to add these to the language and add it accordingly. I think the result will be a far cleaner language. That is, I would advocate adding operator overloading to class Complex if the majority of the community is in favor of it as opposed to adding unlimited operator overloading into the language. A similar example is the addition of the for-each operator to the language and (in the future) a resource-management mechanism.

marius said...

Although I'm totally in favor of closures and function types etc. Josh has some valid points in his presentation. But I find pretty hard to believe that BGGA will harm in any way the language itself.

Furthermore I find the new constructs very helpful as allowing them leads to LESS code and it makes so much sense if we're thinking into blocks of code that we want to parallelize for example.

Of course it will take some time to get use to such syntax (I'd love to see in an year or so Java puzzlers on closures :)..) but I still think it worth it. I can't help to not think of Scala which is a language that I like more and more but I'm not very optimistic in terms of when my company will adopt it ... or better yet when a good amount of my colleagues will show some interest in it.

I think simplicity is not necessary given by a "rigid" syntax as one can see that even with current form people still can write totally unreadable code. A lot of times we shoot ourselves in the foot and guess what ... it's not the language fault. Unreadable/un-maintainable code always existed regardless of the language and are we that concerned that adding closures/function types will make things worse? I doubt they will.

I am convinced that OOP + functional are a good fit (see Scala or even C# for some syntactic sugar) but that's arguable for average developers.

Cow_woC said...

jvb,

I don't know whether James Gosling thinks differently about how Java should feel these days but I think it is important for us to qualify exactly what "Java feel" actually means.

To me, "Java feel" means ease of development and readability above performance and kitchen-sink features. Specifically I believe this means Java omits features (with low power-to-weight ratio) by design, and the language is so simple that any developer can look at the code and immediately know what it does. This means that the syntax set has to be as small as possible and that features that alias the meaning of control-constructs are a bad thing. This is part of the reason that I believe that unlimited operator overloading and closures is a bad thing. You could solve their use-cases in a much more limited fashion which will be easier to learn and use and will be less open to abuse.

Also, I think it is fair to point out that James Gosling has since moved on to Groovy and his experiences there obviously makes him more likely to push Groovy features back into Java in the same way that a C++ programmer is likely to want to push C++ features into Java.

I think we can all agree that throughout recent history Joshua Bloch has been advocating simple/minimalistic design above all else. If you read his Effective Java book you will know what I mean. Now the question is whether you like this approach or not. I happen to like it.

Jacob said...

Don't worry if Java became confusingly complex. There is always C# to save our future.

Haha!! Threatened? :-)

--
_001101110011011100110111_

Ex-Borlander said...

Yikes! With Neal's proposal on adding closures to Java, I see the same mindset that destroyed C++ as a language that could be used by mainstream developers.

Closures in today's world are a "language geek" feature. Unless done extremely carefully and in a way that supports the various skill levels of developers, they end up being unusable and unsupportable by anything less than computer language savants. In their inherent obscurity and complexity, "language geek" style closures are about as anti-Java as you can get.

I believe Joshua's views represent the bulk of mainstream Java developers and existing development while Neil is pushing hard for something that is important to Neil as a personal accomplishment and maybe a few other "language geek" cohorts, most of them probably C++ hackers who have developed their own inscrutable C++ dialects.

If Java is going to have any sort of longevity, the people running the show have to _slow down_ and _continue to improve existing features_ (such as generics), including _reworking_ the language to incorporate what has been learned to date.

I see a terrible attitude towards Java these days -- glue on new features copped from other languages -- instead of focusing on the quality of the existing language features and solving real development problems with true innovation.

In other words, there is great value in making complex problems simple to address.

There is _no value_ in making complex problems complex to solve. That is anti-value because of the load on the overall language created by adding the complexity.

It is very important to _take time_ to consider big changes, not just hack stuff in from other languages (in an ugly way nonetheless). Java is not C#. Java is not Scala. Java is not Haskell. Java is not Lisp. And so on.

Another way of talking about it:

The lifetime of a ship is dramatically reduced if you sail the ship a long time without putting it into drydock and giving it a thorough overhaul.

Java has not gone into drydock. It is dying, opting out of its own future, because its internal systems have not had that overhaul. The learning from all that time at sea has not been applied to reworking and rebuilding the ship's systems. Welding a bunch of new appliances onto rusted pipes is not going to make it out at sea. Plugging in a bunch of new high-wattage toasters into old overloaded electrical systems is a recipe for disaster. And that is what is going on today.

Strong words, dying and all, but those with strong pattern-based predictive abilities will know exactly what I am talking about.

I have a fondness for Java. I really do hope the language (meaning the language, the VM, the runtime environment, etc) gets its time in drydock. And that the "language geeks" are kept in check, their energy channeled into genuine innovation, making complex problems simple to address. After all, that _is_ Java.

Anonymous said...

Neal,
How will the decision be made on what closure proposal will be selected (CICE, BGGA, Other)? I see a lot of heated debate but who will ultimately drive the decision of what gets into Java 7 (that is on the subject of closures)? Is there some sort of way that the average Joe can vote on this?

Thanks

Axel Rauschmayer said...

Wow. Lots of conservative voices here. Let's make one thing clear: in its current form, Java does have a lot of confusing features, but it also severely limits one's freedom to express oneself compared to (say) Common Lisp or Smalltalk. The next version of Javascript is everything Java should be (it has multiple dispatch, generators, optional typing , etc.) and I think Java has to keep up. In order to do that it needs some kind of closures (among other features, but this is the most urgent one) and it has to be in Java 7. Maybe I shouldn't worry and just wait for JS2 to be compiled to the JVM, but I like Java and would hate to see it die.

Edson said...

I think that the current proposal for Closures is incomplete; it can't support somewhat like .NET's LINQ because you can't get the abstract syntax tree of a closure.
You can't create methods that get the closure's AST and create SQL or LDAP queries for deferred evaluation, for instance.
I support BGGA but it must be completed by somewhat that allows implementing code in LINQ style.

ashishwave said...

Closures are the most needed features of Java. Rather i will say, that heredoc (multiline strings) are also the most needed ones(just think of pain of creating SQL strings by adding a lot of adding java variables .

Anyway, i think that those who say, that -- "Java should not add new things, bcoz everything which could be invented in java has been invented, and adding/changing features in java further should stop. Bcoz learning those features will have a curve for the old used-to programmers", that way there was no need for java itself in first place, everybody was coding in C++/Visual Basic. Every needed feature at that time was available in those langs, so why was java needed in first place; After all programmers had to learn the new syntax (of java) at that time.
--- i think this is a very shoddy logic .

anyway, closures etc are very powerful features , which saves you a lot from too much badly written code in many places in java. In many places using closures, cleans up the code a lot.
bye :-)
Ashish Ranjan
ashishwave@yahoo.com

Macneil said...

Let me take back my previous comment about function types "looking better". That's not always the case, as Josh's talk points out, we could have examples like "joinedCounters" (slide 30). (Thanks for including the link to the slides!)

But I think the CICE proposal could gain from including the ideas based on your work. I talk about this more on my blog post (linked to from here). It would really help the CICE proposal out if it
included your notion for inferred generic type parameters together with the short "simple expression as a return statement" syntax.

I believe you could then add the non-local return and breaks to the CICE proposal by making it shorthand for throwing anonymous exception types and catching them in the innermost enclosing context. These operations could be made transparent by requiring the use of the "throw" keyword for the non-local versions: "throw return" and "throw break".

Awdir said...

Java's uptake grew quickly because it was adopted by major institutions: colleges, businesses, etc. They chose Java because it was a relatively simply language that could handle a large number of problem domains relatively well (though it may not be suited perfectly for any or all those domains).

With each new feature, Java grows in complexity. A time will come when many will say about Java what was said about C++, "it's too complex."

I'm in agreement is Ex-borlander that more focus should be directed at improving current features, increasing VM performance, etc.

Luc Duponcheel said...

Hello,

if you are interested in code examples, then it might be a good idea to go to my blog. I just started the blog.

The intention is to show how to do pure functional programming using java closures.

I hope to introduce the topic in a 'gentle enough' way, if not, all suggestions for improvement are welcome.

thx

http://lucdup.blogspot.com/

Luc Duponcheel

javierbds said...

I have always found "mixed paradigm" programming to be quite difficult, I'd rather mix languages (where every different language is used with a clear purpose and advantage) than many different styles in a language ... Maybe my context switching hw is not up to par ... I'm pretty sure I am an average Joe, and my proudness is that I do not f#ck up (pardon my french) as big as others ...

Luc Duponcheel said...

Mixed-Paradigm programming is, indeed, not the easiest thing programmers are confronted with. Especially choosing between them is not always easy.

The question whether or not to do mixed-paradigm programming using one language or different languages is somewhat orthogonal to the question of using mixed-paradigm programming or not.

So what are the criteria to choose between one or the other?

Working with different languages probably has the benefit of writing more elegant code (each language offering syntax that is highly tailored towards a specific paradigm), but, switching to another language from within a given language is not always very elegant either (e.g. using JNI from within Java). Furthermore I'm not sure that, when using different languages, you're not going to be confronted with subtle differences in operational semantics. For example: does Java multithreaded programming work well with Haskell multithreaded programming?

So, imho, the challenge is to offer extra programming paradigms within one language in a way that programming using them is sufficiently elegant, and not programming using them implies not being confronted with them in any way.

Luc

Axel Rauschmayer said...

Where does the idea come from that closures in Java lead to a badly integrated multi-paradigm language? First, languages such as Smalltalk, Javascript and Ruby *all* have closures and nobody would say that these languages are functional or multi-paradigm. Second, even if a feature feels functional, it can fit very well into a different paradigm. Case in point: Java's immutable strings. Finally, I do agree that closures make the construct of a function more explicit in Java, but on one hand Java already badly disguises functions as static methods, on the other hand, Stephen Colebourne's ideas [1] do a reasonable job of continuing the disguise (if that is your thing).

Closures are going to make many things much easier. The most obvious example for me has always been iterating over a tree: Letting the iteration code invoke a closure is much simpler than implementing an iterator [2].

[1] http://www.jroller.com/scolebourne/entry/closures_comparing_the_core_of
[2] http://gafter.blogspot.com/2007/07/internal-versus-external-iterators.html

Joe B said...

I can respect all the work that has been put into BGGA. However, the main concern I have is that it could allow anyone to circumvent the JCP. I prefer new language constructs to be reviewed and approved by the community. One of the great strengths of Java is that programmers can easily move from one project to another. BGGA will weaken Java because individuals will create their own dialects. This will increase the time it takes a new developer to become familiar with an existing project, as well as learn the language itself. Does anyone remember what Microsoft did with C++ Macros in COM and MFC? I am concerned that something similar would happen to Java because of BGGA.

Neal Gafter said...

@JoeB: APIs such as withLock are library APIs, not language extensions. Do you support allowing programmers to define their own classes? Doesn't that circumvent the JCP too?

kirillkh said...

I was shocked to see such an important figure in Java as Joshua Bloch resorting to such dirty tricks. I find most of his arguments, together with code examples, twisted in one way or another. Talk half an hour of anonymous classes and then show the audience a code sample with an anonymous function - it must be a great surprise, when they think it has anonymous class semantics! In reality, programmers that RTFM before touching code will quickly learn to see the difference, it's no rocket science.

No less was I shocked to hear of his concerns for "Programmer Portability", which really means dispensability, and the dangers caused by allowing mere coders to implement their own control structures. It feels almost like these concerns are the true reasons for his opposition to BGGA, and most of the technical arguments are just bluff.

Josh Bloch needs to realize that many smarter and more ambitious programmers don't want to be production line workers, and if you insist on treating them like that, they'll go elsewhere. Look at .NET and its massive shift towards functional-style programming and start thinking.

Cow_woC said...

I give up. When it comes to closures, everyone seems to hear what they want to hear.

I think it is unfair to say that closures is all bad, but I also think it is unfair to say that none of the points that Joshua brought up hold any value. You can interpret "programmer portability" however you want but he was essentially talking about code readability. Can you honestly say this is a bad thing? It's one thing to challenge whether closures are readable or not. It's another thing to threaten to leave Java for .NET because we are advocating such "crazy" ideas as readability. If you find .NET more conductive to your point or view by all means please join their crowd.

Axel Rauschmayer said...

Has anyone read Miguel de Icaza's comments on Java closures [1]? I wonder what he means. C# is going overboard with all kinds of features, but learning from C# users which of these features work and which don't sounds like a good idea.

[1] tirania.org/blog/archive/2007/Dec-28.html
(last paragraph)

kirillkh said...

cow_woc, he dedicated a vast amount of his talk to trying to convince the audience that higher-order functions are too hard for Java programmers. This is also the context for his "portability" word.

Regarding readability, it is a function of semantics and syntax. While I accept the semantics fully, I think the syntax can use some improvement.

Ideally, per my taste the definition would look like ML:
fun a b c -> c*a/b
and the signature would use the usual arrow syntax in parentheses:
(T1 -> T2 -> R)

More Java-like variant (declaration is pretty similar to BGGA, but I do prefer parentheses and "->"):
(fun a, b, c -> c*a/b)
(T1, T2 -> R)

But even with its current syntax, I'd pick it over CICE.

However, I must say that, in my opinion, even full BGGA closures are too little too late. Where are lightweight variant, tuple, record, list data types with recursion and everything? Pattern matching? Partial application? Tail recursion? And, most importantly, libraries suited for higher-order functions - where's my foldr() and map() in java.util.List? I won't even mention type inference, continuations, lazy evaluation, multiple dispatch and meta-programming.

But this doesn't mean that I don't want closures in Java - for me, as for many other people, Java is the day job, and higher-order functions have a great potential to make it much easier.

Cow_woC said...

Heh, sounds to me like you should be using Perl. Seriously, Java language features only get added if they provide great benefit to a very large segment of the community. Many of the features you requested are specific to small niches and by no means would help the majority of developers.

Why should we have to foot the bill (both in terms of development time and conceptual weight) for these things?

Back in my dad's time people used to use Fortran for Engineering mathematics because they found it well suited for that. Other languages were used by other domains. I know it sucks having to use a different language but I don't think it makes sense to add the kitchen sink to any language like you are proposing. Perl is that kind of language and see how it turned out. Next people will be asking for Java to support dynamic typing (in fact, certain people already have).

Go wild doing your own thing on the Java platform, but please avoid these kinds of changes to Java the language. Nothing prevents you from picking up OpenJDK and releasing a Java variant for the mathematical domain (just change the name of the language).

kirillkh said...

Thanks for your advice, but Perl doesn't have the better half of these features. These are traditional functional language features, Perl is not one.

Now, I didn't request to add them all to Java - at this point that's impossible, and the minimalist culture that has been built around Java would not let it happen. Listing them was supposed to show, how little of the true functional features Java is getting with closures and how little "harm" they can do to it, considering there are prospering languages (much more readable and elegant than the usual Java bloat, BTW) that have ALL these features and more. There are also advances in the OOP world that Java is missing, but that's a different story.

I'm not going to argue about the productivity of these things here, as it is irrelevant, but they are certainly not only good for math or any specific niche - much as closures, other features listed exist to improve productivity and maintenance and reduce the development time in general case. They were previously mostly used by scientists, because scientists, as opposed to your average Java programmer cliche (which I hope is incorrect, in the first place), are not afraid to learn new things. But as the field matures, many of these things become a commodity, which means that if any specific language doesn't have tuples in 2008, then it's either dead or designed for especially dumb people (and soon it will be over: you can't ride on marketing and PHB acceptance all the time).

So: all these things provide a "great benefit to a very large segment of the community", in fact they benefit all of it (unless any significant part of it is comprised of idiots that, given a tool, will do the wrong thing with it). Don't believe me? Here, read this paper from 80's:
http://www.math.chalmers.se/~rjmh/Papers/whyfp.html

The real questions are how compatible certain proposed feature is with existing language features and applications, how to include it without introducing significant pitfalls or gray areas and how hard will it be to transform the Java culture to accept it. And yes - Java culture is something that exists because of the supply, not demand. There was one culture in 2002 and a different culture in 2007 - generics was added, and guess what - suddenly everyone knows and preaches them. While I don't believe this can also happen to lazy evaluation (or else PHBs will cry murder and jump out of the window), everyone seems to be up to the closures hype at the moment, so it should be the right time to introduce the better form of them.

David said...

I've read blogs and tutorial about closures. And I don't get it. It pushes readability to an extremely low level. Even the simple example above is difficult to me.

I've been programming in Java for 5 years full-time. I don't feel the need for function types. They are a complex short-coded solution for complex problems. And a basic rule when you deal with a complex algorithms is "make it verbose". To increase simplicity and readability.

Though I think Automatic Resource Management blocks would be really cool. Simple solution for a simple problem.

Aakash said...

Hi, noob here :P. I tried understanding how to get the prototype to work, but couldn't. I have searched a lot and the instructions are kinda vague. I have never done such a thing before.

I work on Windows, and I have managed to extract the folder with the bin / lib / test directories.

Can anyone give me a detailed step-by-step instruction how I should proceed?

Thanks,
Aakash