In this installment of The Open Road, Elliotte Rusty Harold looks at how JSR 294 superpackages, intended for inclusion in Java SE 7, will allow for the creation of hierarchical packages, allowing you to organize your code more cleanly and effectively.

First though, here's an update on the status of the OpenJDK project. The latest JDK 7 release is b24, released December 4, 2007, which is no different from b23, except that b24 was built from Mercurial sources.

There are now several significant OpenJDK projects, including the OpenJDK 6 project, a backport of the evolving GPL JDK 7 codebase to the JDK 6 spec. The OpenJDK 6 project just released b06. As described in Joe Darcy's blog, imaging classes have been moved from closed to open, JAX-WS 2.1 has been included, and the SNMP portion of the build has been modified to not fail when binary plugs are absent. Several other improvements are included.

Now, on to JDK 7 and our first feature preview: superpackages. Here's Elliotte.

Since Java 1.0, packages have offered a convenient means of organizing code into different namespaces at the source code level. Classes in the same package can see non-public, non-protected, and non-private methods and fields in other classes in the same package that classes in other packages cannot see.

Unfortunately, there's no standard adjective for members that have no explicit access specifier. The most common, though by no means standard, name for these fields and methods seems to be "default" access. They're also sometimes referred to as "package protected" or "package private." The Java Language Specification uses "default (package)," and that is what I'll use in this article.

How Packages Affect Visibility

For example, consider the three classes in Examples 1, 2, and 3. A cat can make a dog bark and a dog can make a cat meow, but neither a cat nor a dog can get the bacon out of the refrigerator. The refrigerator can make a dog salivate, but it can't make either animal speak. This is because Cat and Dog are in the same package as each other but different packages than Refrigerator, while the speak() and getBacon() methods have default (package) access.

package com.elharo.animals;

public class Dog {

    void speak() {
       System.out.println("bark"); 
    }

    public void salivate() {
        System.out.println("drool");
    }
    
}

package com.elharo.animals;

public class Cat {

    void speak() {
       System.out.println("meow"); 
       
    }
    
}

package org.cafeaulait.kitchen;

public class Refrigerator {

    String getBacon() {
      return "bacon";   
    }
    
}

Hierarchical Names, Flat Namespaces

The package naming convention is hierarchical, but the namespace division isn't. The package namespace structure is flat. A class in java.util.zip has no more access to the internals of java.util.HashMap than does a class in org.apache.xerces. Even if the Refrigerator class had been in the com.elharo.animals.appliances subpackage, it still wouldn't have been able to make the dog bark or the cat meow. Despite the apparent hierarchy of the packages, there's no actual hierarchy in the access protection.

Ninety percent or more of the time, this doesn't matter all that much, but occasionally there are problems. For example, the JDOM XML API put its core model classes like Element and Attribute in org.jdom and its parser classes in org.jdom.input. This meant the builder classes that read XML documents and created objects from them could only use the model classes' public API. However, this API duplicated checks the underlying XML parser had already performed, thus doubling the workload to build the in-memory representation of a document. The core model classes could have provided special, non-verifying methods that did not duplicate the work. However, because the builder classes were in separate packages, these methods would have to be public. Then anyone could use them to bypass the checks, not just the known safe code from the input package.

Another common problem arises when writing unit tests. Test code often needs to see parts of a class that the general public isn't allowed to access. Sometimes test classes want to directly test non-public methods. Other times they want to inject dependencies. Usually you'd think test code belongs in a separate package. However, many programmers put their tests in the same package as the tested code precisely because they want to be able to access the non-public parts. Alternatively, some programmers make methods public just so they can be tested, even if that pollutes the published API. Neither approach feels especially palatable.

Workarounds

There are two common solutions to these problems in Java 6 and earlier. The first is to make methods public that really shouldn't be, and then document them as "non-published" and "for internal use only." Of course, we know how much programmers love to read documentation, and no developer anywhere would ever use a method marked internal, and now that I've moved out of Brooklyn, I've got a bridge for sale at the end of Flatbush Avenue, cheap!

The other alternative is to place all related classes in the same package. That's a little safer, and it's usually what I do--for instance, in my JDOM alternative library XOM, the input and output classes are in the same package as the core classes precisely so they can see each others' internal parts--but this loses the benefits of organizing large code bases hierarchically.

Look! Up in the sky! It's a Bird! It's a Plane! No, It's Superpackage!

JSR 294 proposes to improve the situation in Java 7 by introducing superpackages. A superpackage is a new construct that, like an interface or a class, has its own .java file. This .java file lives in the usual file system hierarchy and is always named super-package.java. For example, a superpackage for com.elharo.pets would be defined by the file named super-package.java in a com/elharo/pets directory. This file would contain a list of all the members of the superpackages and exported classes, as shown in Example 4.

superpackage com.elharo.pets {

    // member packages
    member package com.elharo.pets;

    // member superpackages
    member superpackage com.elharo.pets.avian, com.elharo.pets.aquatic;

    // exported classes
    export com.elharo.pets.Dog;

   // exported superpackages
    export superpackage com.elharo.pets.aquatic;

}

Once the superpackage is created, only exported classes are visible outside the superpackage. For example, the com.elharo.pets.Dog class is visible, but the com.elharo.pets.Cat class is not. Classes in the com.elharo.pets.aquatic package are visible, but classes in com.elharo.pets.avian are not.

Inside the superpackage, everything is the same as before. However, if a class is placed in a superpackage and is accessed from outside the superpackage, everything changes. By default, all access from outside the superpackage is blocked. It's as if all the classes inside the superpackage had suddenly become private. The superpackage barrier forms an event horizon from which information can only escape if it's explicitly exported.

Note the difference between how packages are created and how superpackages are created: packages are created internally when classes declare themselves to be members of those packages. Superpackages are created externally in a separate file.

Exporting and Hiding Methods

What has this bought us? Remember the issue is usually that you have classes and methods you want to expose to other members of the superpackage but not to the general public. For classes, the problem is solved: make them public but don't export them from the superpackage. For methods, the answer is a little trickier and requires an additional level of indirection. In brief, you have to tag the methods you want to publish within the superpackage with default (package) access, just like you do now. Then you define a new accessor class in the same package that delegates to the default (package) methods. You make the methods in this class public, but you don't export the class from the superpackage.

For example, let's see how we'd use this to provide a test class the ability to inject a set of mock system properties into a class. Begin by assuming we have a class set up as in Example 5.

package com.elharo.examples;

import java.util.Properties;

public class Stubborn {

    private Properties props = System.getProperties();
    
    // rest of class...
    
    
}

You want to test the stubborn class with various values for the different system properties. However, that's going to be hard to do because some of these properties, such as java.version and java.vendor, are fixed by the virtual machine you test with. They are not under your direct control.

To allow the test case in the com.elharo.examples.test package to change the system properties, you have to add an additional setter method for the properties as shown in Example 6.

package com.elharo.examples;

import java.util.Properties;

public class Stubborn {

    private Properties props = System.getProperties();
    
    // public dependency injection method
    public void setProperties(Properties props) {
        this.props = props;
    }
    
    // rest of class...
    
}

Now the test code can inject the test properties directly into the class. Unfortunately, so can everyone else, and that can be both a security risk and a source of bugs. It also makes the API needlessly complex and larger than it should be.

Instead, make the setProperties() method default (package) as shown in Example 7.

package com.elharo.examples;

import java.util.Properties;

public class Stubborn {

    private Properties props = System.getProperties();
    
    // no longer public
    void setProperties(Properties props) {
        this.props = props;
    }

    // rest of class...
    
    
}

Then add a new public class in the same package that can see the non-public setProperties() method, as shown in Example 8. The test class in the subpackage com.elharo.examples.test can use this class to create a Stubborn object with custom configured properties rather than invoking the constructor directly. Note that this is not the only way you could arrange this; there are several other possibilities, but this seemed simplest.

package com.elharo.examples;

import java.util.Properties;

public class StubbornBuilder {

    public static Stubborn build(Properties props) {
        Stubborn s = new Stubborn();
        s.setProperties(props);
        return s;
    }
    
}

Finally, define the superpackage for com.elharo.examples as shown in Example 9. Notice that com.elharo.examples.test is a member, but is not exported. StubbornBuilder has not been exported either. This means that to all the classes outside the superpackage, they effectively don't exist. Presumably, there won't even be JavaDoc for these classes.

superpackage com.elharo.examples {

    // member packages
    member package com.elharo.examples;

    // member superpackages
    member superpackage com.elharo.examples.test;

    // exported classes
    export com.elharo.examples.Stubborn;

}

Voila! We now have a public API that exists only in the superpackage. The compiler and the virtual machine will enforce this. This does mean that neither source nor byte code that uses these features will be processable with Java 6 and earlier tools. However there are enough other changes coming in Java 7 that this was likely to be true with or without superpackages.

Summing Up

Superpackages are one of the better ideas being floated for Java 7. They address a real pain point for many developers. However, they don't introduce a lot of confusing new syntax, and if you don't need them you can fairly safely ignore them.

The one change I'd really like to see is better support for working at the method level rather than the class level. I'd like to be able to tag individual methods as exported or not, rather than working with entire classes at a time. The spec is still in early draft review, so time remains for changes to be made and the syntax to be improved, and I'm told a new access specifier to accomplish this is indeed under consideration for the next draft.

Nonetheless, even in its current imperfect state, superpackages are a significant addition to the Java programmer's toolbox.

Resources

• JSR 294: Improved Modularity Support in the Java Programming Language
• The Open Road column on java.net

Elliotte Rusty Harold is the author of numerous books including Java I/O, Java Network Programming, and the upcoming Refactoring HTML.