Skip to main content

Improve on Javadocs with ashkelon

August 26, 2004

{cs.r.title}










Contents
A Database of Docs
More Than a Collection of Loosely
Connected APIs
A Formal Definition for an API
A
Different User Interface?
Final
Thoughts

Sun's Javadocs provide us with automatically generated, hyperlinked documentation of fields, methods, classes, interfaces, and inner classes. They are helpful in many ways, but you will soon develop a wish list that includes the ability to search for programming elements, the ability to access multiple APIs from one location, the ability to cross-reference relationships across APIs. The list goes on.

This article describes ashkelon, an open source documentation system for Java that leverages Sun's Javadoc parser.
When developing a UI for, say, 50 APIs, 500 packages, 10,000 classes, and 100,000 methods, how do you present mounds of information in a readily accessible and intelligible way?
Patrick Chan's solution was to produce the Java Almanac, containing indices and cross-references to programming elements. But there are drawbacks to paper forms of referencing; for one thing, the Java Almanac covers only J2SE. And it's long and heavy.

The handful of techniques used repeatedly in ashkelon's UI to address this problem are:

  1. Hide information on the page using CSS display and visibility properties, and toggle those properties dynamically (using various DHTML client-side "widgets" such as tabs, collapsible trees, and filtered tables).

  2. Color code, for more natural filtering of information on the page.

  3. Minimize scrolling.

  4. Provide many ways to access the same information: via an index, browsing or searching, temporal navigation back to an element recently previewed, via a cross-reference, or another type of programmatic relationship such as a throws relationship.

A Database of Docs

One component of ashkelon plugs in to Javadoc as a doclet to populate a relational database with Javadocs. The corresponding database schema for the Java language consists of tables with names such as member, classtype, package, author, field, parameter, method, and thrownexception. The schema can be updated and grow to include changes such as those introduced with J2SE 5.0 and aspects.

So what can you do with a database schema filled with Javadocs? For one thing, with ashkelon you can populate multiple APIs into a single repository. You can do this incrementally, too. That is, you don't have to add all APIs in one run to "get your links." Let's look at how we might use ashkelon to populate an API such as Hibernate into the database. Invoke the following command from the command line, in much the same way as you would use the Javadoc tool to produce HTML documentation:

~$ ashkelon add @hibernate.xml

To add another API (say dom4j), you do this:

~$ ashkelon add @dom4j.xml

Now we can ask ashkelon to list the APIs in its database:

~$ ashkelon list

which, with my local copy of ashkelon running on my notebook computer today, produces:

Ashkelon: API: Ashkelon
Ashkelon: API: Hibernate
Ashkelon: API: J2SE
Ashkelon: API: JTidy
Ashkelon: API: JUnit
Ashkelon: API: JiBX
Ashkelon: API: Servlet 2.3 & JSP 1.2
Ashkelon: API: dom4j

I happen to be running my local copy of ashkelon against PostgreSQL. The jdocs.org site chose to run ashkelon with MySQL. (I've also recently added support for mckoidb, a lightweight database written in Java.)

ashkelon's design decision to use a database gives it intimate knowledge of the code. It enables it to answer intelligent questions, such as: "list all methods that return objects of a certain type," "list all classes that implement a certain interface," "list all direct subclasses of a certain class," and so on.

More Than a Collection of Loosely Connected APIs

ashkelon makes a major effort to truly integrate the multiple APIs that reside in its repository.

Each time you add an API, a cross-referencing step takes place after the "store" step to update inter-API references between programming elements. Here's an example. Assume you already have the W3C's DOM API populated and then decide to add Hibernate. Look up this Hibernate method:

public org.w3c.dom.Document toGenericDOM()
           throws net.sf.hibernate.HibernateException

Using ashkelon's web-based UI, you'd find that the return type (org.w3c.dom.Document) is hyperlinked.

Inter-API integration can also be seen in the cross-reference section of ashkelon's UI. All classes and interfaces in the ashkelon system have a cross-reference tab, shown in Figure 1.

Figure 1. GUI view of cross-linking
Figure 1. GUI view of cross-linking

For example, you can ask ashkelon to list all methods that return a org.w3c.dom.Document. The results, shown in Figure 2, include methods from various APIs: hibernate, dom4j, javax.xml, org.w3c.tidy, and of course, org.w3c.xml.

Figure 2. Search for occurrences of a class
Figure 2. Search for occurrences of a class

A third place where this integration is evident is in search results. A class search for "*Writer" will produce a listing that includes these classes or interfaces: org.jibx.runtime.IXMLWriter, org.dom4j.io.XMLWriter, javax.swing.text.html.HTMLWriter, java.io.Writer, and javax.sql.RowSetWriter.

A Formal Definition for an API

Digging a little deeper, let's return to that add command. We had to feed it an XML file: hibernate.xml or dom4j.xml. What's that all about?
With the original Javadoc tool, you typically feed it the name of a
file that contains the listing of packages that you want to document.
Being a multi-API repository, ashkelon needs to invent one more level
up the containment hierarchy: the API.

These XML files are nothing more than information about the project you want to document. Those of you familiar with Maven are also familiar with its definition of a project XML file, known as the Project Object Model (POM). As of version 0.8.7, ashkelon understands the Maven POM file format and can use these project.xml files as input for populating APIs into its database repository. Here's how one populates dom4j's API using its project.xml file:

~$ ashkelon add @project.xml

This makes the population of APIs trivial and effortless for projects that use Maven. Alternatively, ashkelon defines its own XML file format for APIs. Below are the contents of dom4j.xml.

<?xml version="1.0" ?>
<api>
<name>dom4j</name>
<summarydescription>The flexible XML framework for Java</summarydescription>
<description>dom4j is an easy to use, open source
library for working with XML, XPath and XSLT on the Java
platform using the Java Collections Framework and with
full support for DOM, SAX and JAXP.</description>
<publisher>SourceForge</publisher>
<download_url>http://www.dom4j.org/</download_url>
<release_date>2004-05-12T08:00:00</release_date>
<version>1.4</version>
<package>org.dom4j</package>
<package>org.dom4j.bean</package>
<package>org.dom4j.datatype</package>
<package>org.dom4j.dom</package>
<package>org.dom4j.dtd</package>
<package>org.dom4j.io</package>
<package>org.dom4j.io.aelfred</package>
<package>org.dom4j.persistence</package>
<package>org.dom4j.persistence.nativ</package>
<package>org.dom4j.rule</package>
<package>org.dom4j.rule.pattern</package>
<package>org.dom4j.swing</package>
<package>org.dom4j.tree</package>
<package>org.dom4j.util</package>
<package>org.dom4j.xpath</package>
<package>org.dom4j.xpp</package>
</api>

There's no need to write this XML file by hand. For one thing, ashkelon comes "out of the box" with approximately 32 pre-written API XML files for various popular open source projects including Ant, J2SE, Hibernate, dom4j, and many more. Secondly, ashkelon comes with a script, called apixml, that will automatically produce the XML file, directly from a Javadoc package-list file:

~/devel/dom4j-1.4/doc/javadoc $ apixml package-list

which produces this output (to stdout):

<?xml version="1.0" ?>
<api>
<name></name>
<summarydescription></summarydescription>
<description></description>
<publisher></publisher>
<download_url></download_url>
<release_date>2001-07-03T08:00:00.000</release_date>
<version></version>
<package>org.dom4j</package>
<package>org.dom4j.bean</package>
<package>org.dom4j.datatype</package>
<package>org.dom4j.dom</package>
<package>org.dom4j.dtd</package>
<package>org.dom4j.io</package>
<package>org.dom4j.io.aelfred</package>
<package>org.dom4j.persistence</package>
<package>org.dom4j.persistence.nativ</package>
<package>org.dom4j.rule</package>
<package>org.dom4j.rule.pattern</package>
<package>org.dom4j.swing</package>
<package>org.dom4j.tree</package>
<package>org.dom4j.util</package>
<package>org.dom4j.xpath</package>
<package>org.dom4j.xpp</package>
</api>

Finally, fill in the missing pieces (API name, description, publisher, etc.), which you can look up on the project's home page.







A Different User Interface?

There have been two user interfaces for Javadocs. The first was
released with JDK1.0 and used images (then in fashion) for page section
headers, such as "Classes" and "Interfaces." The veterans among you will
recall the convex yellow underlines adorning these titles. The second
UI was released with JDK1.1 seven years ago. That is, by large, the
same UI we use today.

I developed ashkelon's DHTML user interface to address a number of usability issues that personally impeded my productivity as a Java developer.

  • Problem 1: Too much scrolling

    To get from the top of java.awt.Component
    to the bottom, you must "page down" 117 times. Contrast this with
    ashkelon's UI, which uses tabbed panes to lay out the information
    horizontally across the page, thus minimizing scrolling. Furthermore,
    ashkelon scrolls only those blocks of information that need to scroll
    (not the entire page), so you don't lose sight of header, footer, and
    sidebar sections that are the means of navigating to other pages in the
    system.

    Figure 3. Compact layout.
    Figure 3. Compact layout
  • Problem 2: Monochromatic The class listing in Figure 4
    below uses color and style to indicate whether a list item is an interface, an
    exception, an abstract class, or a concrete class. The mind can digest
    style and color information much faster than it can distinguish among
    keywords.

    Figure 4. Color cues
    Figure 4. Color cues

    Similar cues within a class method listing, as shown in Figure 5,
    help you identify static methods (in bold), deprecated methods (with
    strike-through), and abstract methods (italicized). How many times have
    you made use of a deprecated method, only to discover after
    implementation that you didn't read the Javadocs carefully enough?

    Figure 5. Method styling
    Figure 5. Method styling
  • Problem 3: Static ashkelon's UI is built dynamically using
    JSPs. Yes, it also is dynamic in other ways: it sports a number of
    client-side, DHTML features (as in a dynamically collapsible
    inheritance tree, or dynamically filtered tables). But that's not the
    point. With a dynamic (as in JSP or PHP) site, you can build a user
    interface that is more flexible. You can implement search forms (direct
    access to information). Figure 6 shows the results for searching for
    all classes whose names begin with String. ashkelon also provides advanced search forms for either classes or methods.

    Figure 6. Search Results
    Figure 6. Search Results

    With a dynamic system, one can build a navigation trail to allow temporal access to information. A sample navigation trail is displayed in Figure 7, just below the menu bar (note that the color coding applies also to elements within the navigation trail).

    Figure 7. Navigation trails
    Figure 7. Navigation trails

    The schema can be extended to produce views that don't exist in
    Javadocs. Figure 8 is one example of how you might display author
    information.

    Figure 8. Author information
    Figure 8. Author information

    One can produce statistical information about code using SQL group by
    statements. Figure 9 shows the results of a query that lists all of the
    packages in alphabetical order. Next to each package is the number of
    classes it contains, with all packages with more than fifty classes
    highlighted in yellow.

    Figure 9. Displaying statistical information
    Figure 9. Displaying statistical information

Final Thoughts

ashkelon can benefit an individual developer, a team, and even the
Java community.
The individual developer can run ashkelon on his or her local box to
reference multiple APIs locally. More interestingly, the team can use
ashkelon to look up both internally developed code and APIs that the
team is using on their current project. The ashkelon database can be updated
nightly with the latest snapshot of the code from CVS so that everyone
has a reference to the latest code. Lastly, a public web site hosting a
large number of APIs can create some terrific economies, and at the same
time capture community contributions and improve itself. I've long wanted to see something like Patrick Chan's examplets married with Javadocs. In the early days, ashkelon was called dbdoc and was featured in the java.sun.com article
"
dbdoc: Persistence Pays."
A public instance of ashkelon ran at dbdoc.org a few years ago, and
could be resurrected. We would need hosting as well as volunteers to
maintain the site, populate APIs, field users' requests for new APIs,
and so on.

There are so many ways you can get involved. Adopt ashkelon in your work environment. Download ashkelon
and, if you can, please
donate
to the project and/or join the project. With a half-dozen talented
developers, this project might have what it takes to become an Apache
project. If you're a graphic designer, you could help significantly by
producing a formal project logo or graphics to enhance the
ashkelon UI. If you don't have time to become a developer, please
participate in the discussions. Join the ashkelon-users or
ashkelon-devs mailing lists.

Eitan Suez is a programmer living and working in Austin, Texas. He has been programming in Java since 1995 and is a certified Java programmer.
Related Topics >> Programming   |