Monday, November 13, 2006

Closures Esoterica: completion transparency

The Closures for Java draft specifiation (currently at v0.3) is the product of a lot of work. Everything in the spec has been discussed and scrutinized before being placed into the specification, and is there for a reason. From your point of view, you have seen snapshots of our specification, with very little explanation of why things are the way they are. Changes from one revision to another may seem arbitrary. I am not writing detailed notes on the progress and evolution of the specification and its prototype, because there is too much to explain and too little time would be left for me to work on the specification and prototype. I am, after all, working on this on my own time. I suspect few people would care for that level of detail anyway. Much of the work on the specification takes place during face-to-face meetings, after which I update the specification to reflect our decisions. Some issues get resolved through email discussions. We've been keeping an eye on the comments on this blog, various message boards, mailing lists, and elsewhere for input, and meeting with interested folks, and that has been a very useful part of the process. Thank you for your help!

I'm preparing the next revision of the specification, and we just resolved an issue by email. Unless you know what the issue is, the change in the upcoming version of the specification will appear totally arbitrary. Because the issue was resolved by email, I have a nice record of the problem and its resolution. Here is an edited excerpt of the the start of our discussion:

We'd like programmers to be able to define their own control constructs. One thing that would make this easier would be if programmer-defined control constructs act like built-in ones for the purposes of handling reachability of statements (and "can complete normally"). That's why we added the bottom type java.lang.Unreachable to the specification. But just having that isn't enough. Watch as I try to define a withLock method that is completion transparent.

First, ignoring reachability, the library writer might define

<throws E> void withLock(Lock lock, {=>void throws E} block) throws E { ... }

this achieves exception transparency, but not completion transparency. If the block returns a value, this would work:

<T, throws E> T withLock(Lock lock, {=>T throws E} block) throws E { ... }

this works because a closure that can't complete normally can be converted, via the closure conversion, to {=>Unreachable}. In practice, any specific invocation of withLock will either have a block that doesn't return anything (i.e. =>void) or can't complete normally (i.e. =>Unreachable}. You might think, therefore, that the library writer need only write these two overloaded methods to achieve completion transparency, and let overload resolution take care of the rest.

Unfortunately it isn't that simple. With these two methods, an invocation of withLock using a closure that can't complete normally can be an invocation of either of these methods. That's because the closure conversion can convert a closure that results in Unreachable to an interface whose method returns void. Since both are applicable, the compiler must select the most specific. Neither of these methods is more specific than the other (the closures are unrelated, given our scheme of mapping function types to interfaces), so the invocation is ambiguous.

I don't propose that we mess with the specification for "most specific". I'm afraid that would be a disaster, though maybe you can reassure me that it isn't. Instead, I propose that the closure conversion be allowed to convert a closure that results in void to an interface whose method's return type is java.lang.Void. The generated code would naturally return a null value. Then the library writer would write only the second version, above, and it would work both for the void-returning case and the Unreachable-returning case. I think being able to write control abstractions as a single method (instead of two overloadings) is a significant advantage. Additionally, this API is more flexible because it can be used by programmers to pass a value back through from the closure to the caller.

We discussed this issue and found the proposed resolution our best option. The next version of the proposal will include in the closure conversion a provision allowing conversion of a closure that returns void to an interface type whose method returns java.lang.Void. You can see hints in this email thread that the syntax of a function type has changed slightly (the throws clause has moved inside the curly braces), and that a function type is now specified to be an interface type, rather than having its own type rules. The specification is getting simpler, which is definitely a move in the right direction!

5 comments:

mic said...

Looking Forward to see the 0.4v

forax said...

Just a question, is there a reason to use curly braces '{' and '}' in a closure type and not parenthesis '(',')' or better '[',']' square bracket (or what you want unlike parenthesis and curly braces).

[int,int=>int] f={int x,int y=>x+y }

For me '{' means block of codes and not type.

The thing that disturb me is the fact that curly braces are used
to specify the type and the code of the closure.

It remember me the C syntax for the pointer
int *p=*k;
beurk !!

I think it makes the proposed syntax not easy to understand for a beginner.

Rémi

Neal Gafter said...

Rémi- I expect people to read

{int,int=>int}

as "a block that takes two ints and has an int result." The parallelism between the function type syntax and closure syntax is intentional. I expect the parallelism to make it easier to learn rather than harder.

Anonymous said...

> It remember me the C syntax for the pointer

int *p=*k;

> beurk !!

The C syntax is

int *p = &k;

that's not parallel.
The proposed syntax is parallel.

Alex Popescu (aka the_mindstorm) said...

A couple of question related to the readability aspects of the spec:

1/ is there a common super type for all closures?

2/ if so, would it be possible, considering also the new introduced TypeParameter, to have a better syntax for the FormalParamsDecl? (namely withLock(Lock lock, {=>T throws E}) maybe something like withLock(Lock lock, Closure<T, throws E>)... but I am not sure what can be done about the return type :-) ).

3/ By looking at multi-exceptions:
Locks.<throws IOException|NumberFormatException>withLock(lock, {=>
System.out.println("hello");
});
I am wondering if this will work with static imports? If so, isn't it looking extremely ugly?

4/Most of the things I've mentioned are about the exception transparency feature. Most probably, you have already discussed this, but I haven't found any references: wouldn't be possible to wrap all exceptions thrown by the closure code in a RuntimeException? (well, I confess I am not pretty sure about this one, because even if it would simplify the life of the API writer, it would lead to uglier code on client code)

bests,

./alex
--
.w( the_mindstorm )p.