Standing in the Field @ Valley Forge

Standing in the Field

Notes from SJS Application Server Field Engineering

« Previous month (Dec 2004) | Main | Next month (Feb 2005) »
Monday January 31, 2005
Annotations (Footnote)

So I have a couple of websites I maintain as side projects. The one I've been spending the most time recently is my Mom's fabric arts page. Because of some of the precise layout that my Mom wanted, I used XHTML and CSS to layout the pages. Which is great: CSS is easier to do than table based layout and greatly reduces the page size. But browser compatibility has haunted me somewhat.

The nastiest example so far was that my Mom updated a couple of the pages to make a couple of minor tweaks. But once she made the changes the pages no longer rendered correctly in Internet Explorer. I was trying to troubleshoot the problem, but couldn't find anything wrong with the code. It was the exact same CSS style code as all of the other pages.

It turns out that Outlook Express had added an HTML comment to the top of the HTML page when my Mom saved some versions of the page we had been emailing back and forth. I hadn't noticed it because it was just a comment. But as HTML gurus know, browsers look at the first line of the HTML to determine the DOCTYPE and sometimes adjust their rendering based on that DOCTYPE. IE was rendering most of the pages correctly because it was using a strict XHTML mode. But with the comment at the top it must have fallen back to a quirks mode that didn't render the page correctly.

This triggered one of my longtime frustrations: comments that affect behavior. In everyone's first computer language class they are taught that comments are for increasing code readability but do not affect the functionality of the application. Shortly thereafter, however, we start learning all of the little exceptions to that rule. UNIX scripts declare their interpretive shell in comments. Java embedds its API documentation in comments. HTML embeds style sheet and script information in comments.

This has always bothered me. It seems a dangerous practice that violates the "contract" between developers and their code. This IE troubleshooting was a perfect example: I completely ignored the comment because my mind just automatically ignored it. And Dreamweaver helped me ignore it by greying it out. Both Dreamweaver and I were making the bad assumption that comments don't affect the layout. I've learned to just accept this practice because we've needed these hacks. JavaDoc is a great feature of Java even if it violates my aesthetics about comments.

Metadata, however, has the chance to clean up some of these hacks, at least in the Java world. It gives a way to let tools interact with source code without having to hide the interaction from the compiler using comments. HAll of the examples above (JavaDoc, shell interpreters, HTML styles/scripts, and HTML DOCTYPES) are really examples of application metadata. Now that Java has a declarative method for Metadata, hopefully we can move forward with cleaner code.

(2005-01-31 12:02:28.0) Permalink


Monday January 10, 2005
Travel: Problems, Small Airports, MacWorld SF

This week I'm out at Sun's corporate offices in California. A bunch of us software folks are getting some advanced training and some face time with the product engineering folks. This is a good thing, especially considering the new versions of Java Enterprise System and Sun Java System Application Server around the corner.

Unfortunately, I live in an area with relatively poor access to airports. I can drive east an hour and a half to a major airport or I can drive west an hour and a half to a smaller airport. Driving to the major airport is tough during rush hour and is susceptible to all kinds of traffic, parking, and security delays. The smaller airport, Harrisburg International, is a much more pleasant drive, the service is great, and it's generally just easier getting in and out. They even promote themselves as the "antidote to the big airport".

So, when the flight schedules work out, I choose to fly out of the smaller airport.

Now I'm a pretty seasoned business traveler. I think that I'm pretty savvy about the whole process. I pack light. I always have my iPod so that I can kill time. But most importantly I try to keep a positive attitude about the whole experience. Because when you travel frequently, you are going to run into problems sooner or later. My cardinal rule for travel is to be nice to gate agents and to trust them to help you. I've had lots of issues when traveling, some of them my own fault, but the airlines have always done the best the could for me. (Everyone should have to watch the TV show Airline before getting upset with a gate agent.)

So when I ran late on Sunday and missed my flight I was optimistic that the airline would be able to help me out. Unfortunately, however, being at a smaller airport does limit your choices significantly. Harrisburg only has a dozen or so gates, and they are divided between a lot of airlines. Combine that with a flood of people heading to San Francisco for MacWorld and some bad weather on the west coast and I was pretty much out of luck. I'm stunned at how crazy San Francisco is over MacWorld. The flights are packed. Changing car and hotel arrangements has been near impossible.

Anyway it's been a stressful day and a half getting here. I finally made it to my hotel room. I guess the lesson to learn from this experience is to know when the system has a small tolerance for failure and to plan accordingly. I suppose the same thing could be said about software system design. A skilled system architect should be able to know in advance where to the stress points of a system will be and to put plans in place to reduce those risks.

That's a big stretch to try and create an analogy, but you'll have to forgive me. It's a been a long day.

(2005-01-10 23:42:08.0) Permalink


Thursday January 06, 2005
Annotations (Part 2)

I really have to stop taking on big blog articles like this. They always take a lot longer to write than I expect. It's been a month since the last article in this series, but here we go with part two on my annotations preso, converted to blog format. Sorry it's so long, I really should have broken it into smaller posts. But I got on a roll and since I'm a month late I figured I might as well get it done.

Last article was just a basic introduction to the concept of Metadata. We introduced annotations, the reason behind introducing them into Java, the basic syntax for using annotations, and the built-in annotations.

Metadata (JSR 175) in Java 5.0, Part Two

Custom Annotations

Now that we understand the basics of metadata, we can now take a look at the more interesting topic of how to create our own metadata. The pre-built annotations have a certain amount of value, but creating your own annotations is where metadata gets interesting. (At least until EJB 3.0 and J2EE 1.5, which will have lots of time saving new annotations.)

Why would we want to create a new annotation?

There are several articles on the web about how to add markers to code and how to use reflection to allow code to inspect itself. I'd recommend starting with the annotations section of the language guide. It walks through creating a new marker interface @Preliminary and creating a single valued interface of @Copyright. It also creates a @Test marker interface and then shows how to use reflection to determine if that marker exists for a given class.

I'm not going to take a lot of time detail those simple examples again here. In summary, creating new annotations is a lot like creating new interfaces. The major difference (besides the @ symbol) is that you have to be conscious of the meta annotations such as @Retention and @Target. And there are some new methods and in the java.lang.reflect package such as isAnnotationpresent and getDeclaredAnnotations that allow you to detect and manipulate annotations. Here's a quick example of a single value custom annotation with a default value.

package AnnotationDemo;
import java.lang.annotation.*;

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.TYPE)
public @interface Maintainer { 
    String value() default "unknown";
}
And here is a little command line app to parse for those annotations: 
package AnnotationDemo;

//This class accepts a class and returns the value of the @maintainer
//annotation. (If the targeted class has one.)

@Maintainer(value="ogren")
public class SingleValueDemo {
    public static String findMaintainer(String className) {
        String maintainer;
        try {
            Maintainer notation = (Maintainer)
               Class.forName(className).getAnnotation(Maintainer.class);
            if (notation!=null) {
                maintainer="Maintainer is " + notation.value();
            }
            else {
                maintainer="Class exists but is not annotated with maintainer.";
            }
        } catch (ClassNotFoundException e) {
            maintainer= "No such class found";
        }
        return maintainer;
    }
    
    public static void main(String[] args) {
        if (args[0]!=null) {
            System.out.println(findMaintainer(args[0]));
        } else
            System.out.println("Usage: SingleValueDemo classname");
    }
}

But if you want to change the compiler behavior, the most interesting part of annotations in my opinion, you have to venture into the world of apt and annotation factories.

Apt / Annotation Factories

apt is a javac replacement that finds and executes annotation processors. Yes, this means that you have to change your build chain if you want to use these new features. If you use javac directly it will not perform all of your custom metadata enhanced behavior.

Annotation Factories implement a simple interface with three methods: getProcessorFor, supportedAnnotationTypes, and supportedOptions. These three methods allow the apt tool to determine which factories are interested in which annotation types and to obtain a AnnotationProcessor object for a given annotation type.

Here is the bulk of the code from the annotation factory I use in my demo: (for the reasons I explained a while ago, I'm not going post all of the code.)

public class PooledFactoryApf implements AnnotationProcessorFactory {
   private static final String FQTAGNAME = "AnnotationDemo.PooledFactory";
   //(This commented line is how you would declare the annotations you support.
   //So that this demo can watch all
   //annotations, however, this factory tells apt we process all ("*") annotations.
   // private static final Collection supportedAnnotations =
   //    Collections.unmodifiableCollection(Arrays.asList("AnnotationDemo.PooledFactory"));
   private static final Collection supportedAnnotations =
      Collections.unmodifiableCollection(Arrays.asList("*"));
   private static final Collection supportedOptions = emptySet();
    
   public Collection supportedAnnotationTypes() {
      return supportedAnnotations;
   }
   
   public Collection supportedOptions() {
      return supportedOptions;
   }

   public AnnotationProcessorFor(
      Set atds, AnnotationProcessorEnvironment env) {
         return new PooledFactoryProcessor(env);
   }
}

The code is pretty self explanatory. You return collections of Strings defining the annotations and options you support. You can use wildcards in those Strings to support ranges of items. My particular demo accepts all annotations (just for demo purposes) and no options. And when asked for a AnnotationProcessor the factory blindly returns the PooledFactoryProcessor which I define below, regardless of what the annotation is or what is going on in the environment.

AnnotationProcessor

The AnnotationProcessor is the place where we we actually place our customized behavior. AnnotationProcessors support a simple interface with only one method: process(). Once the apt tool has obtained the correct AnnotationProcessor from the factory, it will call the process method. Once we receive this call to the process method we can use the environment (which we received during construction) to search for instances the annotation we are looking for and perform our custom behavior.

For my demo application I developed an annotation that would automagically generate the code to implement a factory pattern classes it decorates. In theory, this could be used to automatically make any class a pooled resource, although I built this as a proof of concept and didn't actually add the logic to implement pooling. Here is the declaration for my annotation:

@Retention(RetentionPolicy.SOURCE)
@Target(ElementType.TYPE)
public @interface PooledFactory {
    int poolSize();    
}

As you can see it is a single valued annotation that marks types (classes and interfaces) and is discarded after compilation. The annotation is discarded after compilation because its job generating the factory code is complete after compile time. Another thing to note is that legally this annotation could be placed on an interface since the annotation syntax considers both classes and interfaces to be types. My demo does throw an error at compile time if an interface is marked with @PooledFactory, but it will not show up as syntactically invalid in an IDE. The following is a simple illustration of how the @PooledFactory annotation might be used:

@PooledFactory(poolSize=4)
public class ExpensiveClass {
    ExpensiveClass() {
        System.out.println(
        "For some reason, I take a long time to create so I am being pooled.");
    }
    public void someMethod() {
        System.out.println("Some business method is executing");
    }
}
Clients would then access an instance via the automatically generated factory class: 
ExpensiveClass myClass = ExpensiveClassFactory.getInstance();
myClass.someMethod();
ExpensiveClassFactory.returnInstance(myClass);

To me, this is why annotations are exciting. We have just extended Java with a new piece of declarative functionality. If we are using a profiler and discover a bottleneck creating ExpensiveClass objects we can attempt a fix with three lines of code (one to add the annotation, one to change the allocation, and one to return the object to the pool). We also get the added benefits of code readability and code reuse.

So let's implement the code generation in the AnnotationProcessor. The first step is to receive the process() call from apt and find all of the classes that are marked with our annotation:

private final AnnotationProcessorEnvironment env;
private final AnnotationTypeDeclaration myType;

PooledFactoryProcessor(AnnotationProcessorEnvironment env) {
   this.env = env;
   this.myType = (AnnotationTypeDeclaration)
      env.getTypeDeclaration("AnnotationDemo.PooledFactory");
}


public void process() {
   Collection annotatedClasses = 
      env.getDeclarationsAnnotatedWith(myType);
   for (Declaration decl : annotatedClasses) {
      if (decl instanceof ClassDeclaration)
         //this method creates the new source file
         createFactorySource((ClassDeclaration) decl);
      else
         env.getMessager().printWarning(
            "An interface was marked with @PooledFactory");
   }
}

In the constructor we save the environment we receive from the factory and save the AnnotationTypeDeclaration we are looking for. Once in the process method all we have to do is use the com.sun.mirror.apt.AnnotationProcessorEnvironment to get a list of types that implement our annotation and then iterate over that Collection (using the new foreach style of for loop). As mentioned before we have to check to make sure that our types are classes and then we can execute our custom method (below) to generate the new java source. Notice how that we can print a compiler warning via the AnnotationProcessorEnvironment object. We could use the same technique to implement annotations that perform syntax checking or analysis.

The createFactorySource method is were we actually generate the new source file for the factory class. This demo uses a very simplistic method for doing so, shown below:

void createFactorySource(ClassDeclaration decl) {
   //Determine new class name
   String realClass = decl.getSimpleName();
   String newClassName = realClass + "Factory";
   String newQualName = decl.getQualifiedName() + "Factory";
            
   System.out.println("Attempting to create factory :" + newQualName);
        
   try {
      PrintWriter out = env.getFiler().createSourceFile(newQualName);
      out.println("//autogenerated by PooledFactoryProcessor");
      out.println("package " + decl.getPackage() + ";");
      out.println("public class " + newClassName +" { ");
      out.println("public static " + realClass +
         " getInstance() { return new " + realClass + "(); }");
      out.println("public static void returnInstance( " + realClass +
         " old) {  }");
      out.println("}");
   } catch (java.io.IOException e) {
       System.out.println("Factory already exists or cannot be created");
   }
}

This class is a bit of a hack, but it shows the basics of source code generation. We figure out what class name (and qualified class name) to use and then use the AnnotationProcessorEnvironment to get a PrintWriter leading to a new source file. (The new source file ends up getting created in a temp directory that can be specified using the -s apt argument.) We then write out the new code to PrintWriter and apt handles the rest. It will automatically pass the new code through the AnnotationFactory again since we might have annotations in our generated code. After that recursive step completes it will then use javac to compile all of the code, including our generated code. At that point ExpensiveClassFactory exists just as if it had been coded by hand.

Obviously this is just a little demo hack and doesn't do any real pooling. Real world code generation takes more than six println statements. But it illustrates the basics of modifying compile time behavior and its a lot more fun than the little reflection samples I see in most annotation tutorials. The same techniques cold be used to do all kinds of powerful compile time behavior.

In the next an final chapter, I'll share some my closing thoughts about metadata. I'll talk about the pros and cons of apt and the future of annoations, such as how metadata will be used in J2EE 1.5 and EJB 3.0.

(2005-01-06 06:47:40.0) Permalink


Wednesday January 05, 2005
"SPARC ecache failures"

Hmmm. Was looking at the referrer logs and saw that someone reached my blog by doing a Google search on "sparc ecache failure".I mention ecache failures obliquely in my post on why I'm blogging as an example of bad communication from Sun's past. But didn't think I had enough information to show up on a Google search.

So I ran the search myself in Google to see what happens. This blog was the #1 hit! How strange. So I did a little more investigation. I used Google to see who was linked to that article and found that there aren't any external links to that article. So how does an article with no external links and only two of the three keywords rise to the top of Google? Google PageRank is a quirky beast.

(2005-01-05 06:59:01.0) Permalink Comments [2]


Tuesday January 04, 2005
Podcasting
audacity screen capture

I had a half finished post on podcasting drafted on MarsEdit (which I've been using to post since Roller 1.0 went live). But I wanted to finish my second post on Java annotations before I spent any time posting about iPods. But now Tim Bray has posted an excellent article on podcasting and I have to put in my two cents.

I admit to playing around with podcasts recently. Gio had been talking about them a month ago, some of the topics on IT conversations had been really interesting, and NetNewsWire added it as a feature in its 2.0 beta. Which made downloading podcasts pretty much point and click. So with all of this, you'd think I'd be a big podcast fan. I have a Mac. I have an iPod. I'm usually up for the latest new gizmo.

But as both Adam Curry and Tim point out, you can't really be listening to a podcast while you do anything else mentally engaging. Unlike music, podcasts really don't function too well as "background sound". So, for me, that limits listening to podcasts to plane and train rides. And when I do have dedicated "iPod time", I'm not sure I want to use that precious downtime listening to an amateur audio show.

Plus, like Tim, I have a bias towards text. You can read at your own pace depending on your interest and mood. You can skip around an article, skimming past stuff you aren't interested in reading. You can search it and index it for keywords. Plain text is just efficient and versatile.

But podcasting is still interesting to me as a phenomenon, even if I'm not going to be an early adopter. Because it's amazing how easy that podcasting makes audio production. Back in 1975 creating an amateur newsletter or brochure required expensive equipment and produced poor results. (Remember mimeographs?) Home production of audio or video wasn't really feasible at all except for rudimentary Super8 splicing. The desktop computer revolution has changed that. In 2005 a low end home computer, with moderately priced software, is easily capable of doing semi-professional print or audio production. And a mid range computer is capable of doing basic video production.

Amateurs are publishing blogs that are taken just as seriously as professional productions. To the point where the line between amateur and professional is very blurry.

Film students are creating films that are reaching professional quality with budgets under $100,000.

Mac Geeks are creating their own iPod ads that are arguably more professional looking than the real thing.

And now we are seeing the emergence of the semi-pro home audio production studio. I found this description of a semi-pro audio production studio fascinating.

I'm not the first to have noticed this phenomenon. It's been called the "prosumer" trend. (Consumers that are also producers of content. Apparently the term was coined by Alvin Toffler in 1980.) But with the rise of blogging and podcasting in 2004 it really makes me wonder what the media will be like in 2020. What will the next generation be able to create when they grow up with iMovie, Garage Band, iPodder, and TypePad? And what will the home studio be capable of producing when everyone has a desktop Hero server?

I can't say for sure, but it will be nice having a few episodes of IT conversations saved up on my iPod for my next flight.

(2005-01-04 20:51:50.0) Permalink

Comments back on
Turning comments back on. Hopefully the new 1.0 upgrade will protect me from the spammers. (2005-01-04 15:08:07.0) Permalink