Thursday, August 26, 2010

A couple of comments on Defender Methods

Brian Goetz has posted version 3 of his proposal "Defender Methods", which is a way of adding methods to existing interfaces without breaking binary compatibility. Generally speaking, I think the idea is sound but I think there are some problems with the proposal in its current form. I would normally post my comments on the proposal to the lambda-dev mailing list, which ensures that any IP embedded in my comments are formally submitted to Oracle's ownership. However, Oracle's recent lawsuit against Google has made it clear that, even though I am a contributor to openjdk7, I do not have a license to Oracle's patents that are necessarily infringed by the use of the openjdk7 source base. This is a very confusing position for the organizer of an open-source effort to take. Rather than continuing to contribute IP directly to the project, I'll post my comments here and contribute them to Oracle once it is clear that I've been granted a license to the patents necessary to use openjdk7.

The latest version of the proposal has added a section on "Binary compatibility" (section 11), but that section fails to address the one binary compatibility issue raised previously: what are the binary compatibility implications of changing the default for a defender method? Since the defender method's default is an implementation detail, such a change should be binary (and source) compatible, but the resolution mechanism (section 3) makes it binary incompatible. The proposal doesn't provide compile-time semantics for invoking an extension method or compile-time rules for when a given class declaration is legal. At the very least there should be rules to ensure that sources that compile cleanly together do not trigger a VM error at runtime due to an ambiguous method invocation. The most natural change of the compile-time rules to match the given runtime resolution rules would make changing the defender method's default a source incompatible change as well (because it could introduce an ambiguity). I believe this undermines the justification for having defender methods out-of-line (section 2). Moreover, the separation forces API designers to create vestigial public API elements (the static extension methods).

These issues can be addressed by (a) having defender method bodies written inline, and (b) rejecting non-abstract class declarations where some defender method does not have a unique default.

I suspect the proposed implementation mechanisms (sections 7 and 8) will be far too expensive compared to existing virtual method calls. For the Interface.super calls (section 7), my suggestion is to extend the VM specification of invokespecial.


Ken B said...

You need to work with the Scala developers to identify what the binary compatibility problems are (they had these problems in Scala 2.7 with their implementation of traits, which are implemented an awful lot like defender methods) and how they solved them (if they have), and what kind of JVM support would benefit them as well.

AlBlue said...

Scala fails to be binary compatible with itself upon each release. Fortunately the designers of the Java language and runtime are not so myopic, hence the discussion here.

Fatih Coşkun said...

Neal, it's good to see you have not given up on Java, yet. It gets harder and harder to believe in the future of Java :-(

Ken B said...

@AlBlue: the Java developers are very myopic. That's why they introduced inner classes and SAMs in JDK 1.1 instead of introducing closures then, and that's why they're trying to introduce ARM blocks and closures in the same release (instead of taking a cue from Ruby, Scala, and Groovy which implement the ARM functionality in libraries using closures). In fact, I think that they're being myopic their approach to defender methods. Rather acknowledging the inevitable Java paradigm shift that's going to accompany their availability (by defining defender methods inline), the JDK devs are going out of their way to prevent the paradigm shift as much as possible (by forcing developers to add the boilerplate of another class definition to hold static methods that will be used to service the defender methods).

Scala's developers are very smart people who have done a brilliant job of creating a language that avoids paradigm limitations as much as possible. The binary compatibility issues stem from JVM limitations. Since JVM improvements for defender methods are on the table here, I hope that a method can be chosen that would eliminate limitations that Scala developers face.