Closures Prototype Update and Extension Methods
Closures Prototype Update
The Closures for Java prototype now allows a closure to access mutated local variables from an enclosing scope. You can download the prototype here. You can also download the sources for the rewritten parts of Doug Lea's fork-join library, ported to use function types. It is a good example of how APIs can be affected by these language changes. Personally, I find the API simplifications to be quite compelling. If you're on the fence about function types, I recommend you have a look. Any feedback you may have is most welcome!
I mentioned previously that I'm working on a number of smaller language features, which hopefully will be considered for JDK7. For now, I'd like to talk about just one of them.
Extension Methods
Once an API has been published, you can't add methods to it without breaking backward compatibility. That's because implementations of the old version of the interface don't provide an implementation of any new methods. You can use abstract classes instead of interfaces, and only add concrete methods in the future. Unfortunately, that limits you to single inheritance.
One way API designers work around this limitation is to add new
functionality to an interface by writing static utility functions. For
example, java.util.Collection.sort acts as an extension
of java.util.List. But such methods are less convenient
to use, and code written using them is more difficult to read.
Extension methods enable programmers to provide additional methods that clients of an interface can elect use as if they are members of the interface. Todd Millstain's Expanders are the most full-featured version of this feature. The simplest version of this feature that I advocate would be to enable statically-imported methods to be used as if members of their first argument type. For example:
import static java.util.Collections.sort;
...
List<String> list = ...;
list.sort();
Extension methods are completely orthogonal to closures, but they enable a number of typical functional-style programming patterns to be expressed more directly in Java using extension methods that accept closures.
32 comments:
I'm strongly in favor of extension methods but not wild about this implementation, because I dislike static imports. (In fact, I never use them, even for EasyMock.) This is because they look like they might be implemented in either the current class or one of its superclasses.
So, rather than cluttering the namespace further, I'd rather have a separate way to import extension methods so that sort() shows up only as an extension method on its first argument.
Neal, can you please elaborate why list.sort() is better then sort(list), which is already working with static imports.
A couple of questions (mostly)
(1)
It sounds like your plan would be to not require the extension method to mark itself as such. So that I could, for example, have:
import static java.lang.System.getProperty;
...
"foo".getProperty();
Even though System.getProperty(String) really isn't intended as an extension method.
Are you leaning towards the "let them write bad code if they want" path, or is there some other reason for not wanting to have to mark (annotate?) extension methods?
(2)
With the introduction of extension methods, I can see the overloading rules, and the potential confusions (a.k.a. bugs) stemming from those rules getting quite complex.
Any thoughts about how extension methods and the overloading rules would work?
Hi Neal--
I take it that the prototype does not include support for extension methods? At least, the example doesn't compile on my end. I like the idea, though--I remember coming across this in Nice with multimethods, though there you could also define methods in scope (not just statically imported) and dispatch to them--e.g.
void foo(String s) {
System.out.println("foo: " + s);
}
String msg = "bar";
msg.foo();
(example derived from this article)
Your proposal has the advantage that we can ship extension methods packaged in utility classes with static methods--seems like a nice way to package extensions and ship them around without burdening everyone with the larger API.
Patrick
Hi Neal,
Are you considering list comprehensions? Like in python
I find that using python list comprehensions leads to very readable code. Moreso than map/filter/reduce.
Do you think that something similar could be implemented for Java easily?
Thanks,
Anders
Wonderful. That would greatly assist readability of Anders Norås's LINQ clone, Quaere.
This seems to be very similar to Objective C categories
Apple's category docs
I like you proposal to use static methods because it makes clear and intuitive the two limitations that other syntaxes can hide:
* you can't add new instance variables
* you can't refer to 'this' or 'super' per se
One of the problems with Categories is loading them can be a bit unpredictable. If you load two Categories with the same method the Objective C runtime picks a winner. In this case it would seem the 'import static' declaration for two methods with the same name and first argument type should be a compile error?
What if I do import static org.foo.Bar.* and org.bar.Baz.* and two methods conflict? Flag a compile error only if the ambiguous method is actually used?
What if a static method would hide an existing instance method on a class? Would that be allowed? One of the issues with Categories is there's no clean way to so the equivalent of 'super.foo()'
If hiding in allowed in your proposal, it seems the following would work naturally as an equivalent to being able to call super.foo() without needing any new syntax?
package baz;
public class Extension {
public static doIt(Bar b) {
b.doIt();
//now do some cool additional stuff...
}
}
import static bar.Extension.doIt;
Foo someFoo = ...
someFoo.doIt();
-> the extension version of doIt() is called, but it chooses to call the original version, like a 'super' call. Correct?
I have to agree with Eugene. The benefit seems to be nothing more than adding a new way to say the same thing, without saving keystrokes or any other benefit. Thereby simply complicating the language for no real gain.
list.sort() in this case is more misleading than beneficial. If list really doesn't have a sort() method don't make it look like it does! sort(list) works just fine and doesn't add this confusion.
Please also consider JavaGI.
Yeh, that what happends when C programmers come to Java ;). It really reminds me Objective C categories which I personally find confusing. How would you controll those extension methods? What if you have two different libraries which try to introduce similar (signatures are equal) methods to the same interface? Once those extensions are introduced they may be taken into use but as they are not part of API who can garanty that they are always provided?
-1. Hate it!
Don't make extra ways to do something that's already possible.
list.sort() == sort(list)
I'm just wondering how is accessing a mutable variable outside a closure from a closure implemented. Is it "boxed" in a mutable container like this:
int v = 0;
{ => int} postInc = { => v++};
// actualy means:
class intContainer {
int value = 0;
}
final intContainer v = new intContainer();
{ => int} postInc = { => v.value++};
What about synchronization issues? Is volatile honored?
"%1$s like it, %1$s think".format(Person.this)
Neal,
Is it your intention to allow or deny this feature on arrays (where the first argument of the statically imported method is some array type)?
Bruce
Neal, I have a comment regarding code generation for mutated local context. You generate synthetic for boxed local context. All seems fine, but I still see the programmer expecting a life time for his local variable being equal to function invocation. In case the closure is used asynchronously you keep a reference to local context in a closure, that could be claimed a memory leak. Would it be better to nullify the field when the function returns and set a flag. And all accesses from a closure should check the flag and throw exception in case the function frame is already not there?
Eugene.
Neal, I have a comment regarding code generation for mutated local context. You generate synthetic class for boxed local context. All seems fine, but I still see the programmer expecting a life time for his local variable being equal to function invocation. In case the closure is used asynchronously you keep a reference to local context in a closure, that could be claimed a memory leak. Would it be better to nullify the field when the function returns and set a flag. And all accesses from a closure should check the flag and throw exception in case the function frame is already not there?
Eugene.
Declaration-site extension methods.
I like the idea, but I think that what Peter suggests is probably a safer way.
I have a question regarding generics though - if I defined a method like Sorter.sort(List<Integer> intList) and imported it, would it mean that someList.sort() is available for
List<Integer> someList...
but not for
List<String> someList...?
I also think that list compreensions generators and filters would be ace, and more readable than the map, reduce, filter equivalents.
Neil, can you tell me your reasons not to use the natural java syntax for method signature in closures.
I think you directed me to read Tennet Correspondence principle, but for me that just reinforces my opinion.
a f(); call is a call to the function.
a void f(){} is a signature
a closure( void f() ) is a closure function that accepts a function signature.
a closure( void f(){} ); is a closure call with a created closure
and a closure( f ); is a closure call with a function reference (implicit above).
@paulo:
Re "a f(); call is a call to the function."
That violates the variable vs. method namespace separation that is part of Java's name lookup rules.
Re: "a void f(){} is a signature"
I don't know what you mean by signature. If you mean type, then what is the "f" there in the middle of the type? Types can be used in contexts in which they need not be named (for example, base clauses and type arguments).
Your suggestion doesn't provide a way to distinguish a method declaration from a field that contains a closure. They must be different as they have different scoping rules; only the variable can be reassigned.
Neil, would it be possible to add some syntactic sugar for having some way of passing a method into a place a closure is used?
Like
{int,String,int,int -> boolean} matcher = #regionMatches;
which should translate to sth equivalent to
{int,String,int,int -> boolean} matcher = { int i1,String s2, int i3, int i4 -> this.regionMatches(i1,s2,i3,i4) };
Does list.sort1() throw NullPointerException in case sort1(list) doesn't throw it? Seems confused. I dont sure I'll use this feature.
If I could pass a method (with the same signature as the closure) where a closure is needed, I would use that feature a lot.
I assumed this feature was never considered because in Axel's example you can write regionMatches as follows:
{int, String, int, int => boolean } regionMatches =
{
int i1,String s2, int i3, int i4 =>
// code for region matches over here
return true;
};
But I think that having a regionMatches() method is more natural.
Passing a method would also be usefull in locations where the compiler expects a single method interface: in those locations you can already pass a closure(closure conversion) so with the proposed feature you could then also provide a method.
E.g. if you have a method void foo() in your class, you could then write:
new Thread(#foo).start()
Hi everyone,
Methinks that making an import like
import static java.util.Collections.sort;
always having the side effect that on a
List list = ...;
a supposedly late-bound call like
list.sort();
will call a static helper method will lead to confusion if such a method exists and is thus hidden.
And we can always have such a method in a future version even if right now we haven't.
I'd suggest making the extension explicit via a type alias:
importalias MyList for java.util.List java.util.Collections.sort;
or (for all applicable static methods in Collections, i.e. those having a java.util.List - including subtypes - as first argument):
importalias MyList for java.util.List java.util.Collections.*;
This should allow to cast any List to MyList (and vice versa) e.g.:
MyList list = new ArrayList();
i.e. "externally" MyList is regarded to be equivalent to List while in the source doing the importalias the extension method can be called with:
list.sort();
but on something not declared as MyList, e.g.
List l2 = list;
the call
l2.sort();
will call the dynamically bound sort() method of List or fail to compile if none is declared there. What do you say? Of course "importalias" is just a suggest and also we might dispense with the ".*" in "java.util.Collections.*" if that seems nicer. Note that this includes an aliasing feature (which some people say Java is sorely lacking) if we allow the shortcut:
importalias MyList for java.util.List;
Of course you should never misuse this like e.g.:
importalias ArrayList for java.util.LinkedList ;-)
I concur with comments at http://weblogs.java.net/blog/forax/archive/2007/11/java_7_extensio.html.
The only point I can see to extension methods as proposed is to obscure the location of methods and lead to troubleshooting and code reading nightmares. I don't believe I'm overstating the case.
Sure one might wish sort() was a method of list, but it is not and to pretend otherwise just makes it harder to understand what's really going on.
At the URL noted above there's a proposal for static methods on interfaces that seems *far* more attractive in spirit. Personally I think his ".do." and @ExtensionMethod stuff goes too far -- I think simply allowing static methods on interfaces suffices.
If that can't be done for name conflict, etc, reasons, then I maintain we're still better off without extension methods. I can hear junior and senior developers alike asking me "where the #!@$#$ is this implemented?!?" all day long with extension methods.
Hi jess,
you are certainly right with:
"Sure one might wish sort() was a method of list, but it is not and to pretend otherwise just makes it harder to understand what's really going on."
If I wanted a List class with a sort but had to use instances of normal list classes, I could always wrap them in e.g. a new class SortableList which just delegates to the wrapped list object. Such a wrapper would not "pretend otherwise" since it has its own class name. And my suggest for an "importalias" above allows for the same effect while optimizing away the wrapper instance. Wouldn't that be acceptable for (almost ;-) everyone?
Perhaps it would be more readable if import of extension method was different from a regular static import, e.g.
import static java.util.Collection.sort extends java.util.List;
I completely agree with Yardena, it would be confusing to make a "normal" static import (because you only want to import constants or static methods to your class namespace) and see some new methods you didn't know about in some classes.
I think it is a great idea to import extension methods with this syntax (import static {static_class} extends {extended_class_or_classes}).
Another benefit of this syntax is that you can take a look at the imports part of your source code and see instantly which of the classes have been extended (which makes the code clearer also).
While I agree with the idea of a visibly different syntax for these imports, I think that "extends" would still be confusing it a little: It is NOT an extension, it has nothing to do with true polymorphism and dynamic binding. Imo it is more like an alias or wrapper.
re: list.sort()
I like the idea immensely, but it's gotta be in all lists. If Sun just added sort() to List, it would break every list out there. If they also added sort() to AbstractList, it would break any List not derived from AbstractList. So, they gotta add it to List. (actually, Collection).
How about doing that, but adding a default implementation as well? e.g. something like:
@defaultsto Collections.sort(this)
public void sort();
or, IMO cleaner but a bit weird to have actual code in an interface
public void sort() {
Collections.sort(this);
}
All Lists will still work, users can override sort() if they want to (or already have).
Just wanted to crosslink
http://www.artima.com/forums/flat.jsp?forum=106&thread=220783
where Howard Lovatt gave an IMHO nice approach to extension methods which seems simple to use but avoids the ambiguities
(I also think it would not have to be restricted to static imports... see my comment there)
@Morgan I really think adding a method to an interface while giving a default implementation for those implementors which don't have one of would open a very nice way of migration. But that is more declaration site extension methods.
Discussion of that mostly happening here at Peter Ahé's proposal: http://digital-sushi.org/entry/declaration-site-extension-methods/
Hey, I really enjoyed your post. I'm a .Net guy myself, and we have extension methods already for C#. I noticed a lot of your commenters are extension method detractors.
I've been writing a lot of extension methods lately.
I can tell you from experience that extension methods make life so much better. I wish they had syntax to make extension properties too. I actually have a few implementations
you could look at. I'm curious what you think about the syntax from a Java programmers point of view. Here they are:
Extension Method to Copy One Stream to Another
IsEmpty and IsNull Extension Methods on the String Class
Extension Method to Get Custom Attributes with Reflection
I generally like the idea. A note about ambiguity, use static keyword as a primary expression to access statically modified/imported members.
import static java.util.Collections.sort;
import static java.util.Random.sort;
...
List list = ...;
list.static.sort(); // can't use this one when more than one static import conflicts
list.sort(); //resolves to local implementation
static.sort(list);
list.static.Collections.sort();
list.static.Random.sort();
Collections.sort(list);
static void sort(List list) {
//local implementation.
}
Post a Comment