Ales Novak's old blog

Motivation

We had a customer visit a few weeks ago, where customer wanted to see development process for a web application emphasizing master-detail view of data. We did not have quite a good answer back then so I decided to build such an application to see how difficult it is.

Choosing Technology

I have already played with Ajax so I decided to use it for the detail view. Roman was talking briefly about NetBeans support of Google Web Toolkit during the visit, so I started to look at that. The rough idea is that you write Java code which is then compiled to Javascript. After seeing the generated code I decided to move on. Not because it does not work - I just could not read it and that meant no go for me.
Next tech was jMaki. There is a nice video here, with example which is sort of master-detail. jMaki got green light :-)
My application is a bit extended example from here. That was my starting point.
The detail data are stored in a new table called COMPANY_DETAILS. The table is simple one - just ID and an address. I generated an entity class for the table repeating steps from the original application.

The bad

The original app, uses a table which does not store the original company Id. I did not want to waste time to change that so I used company name as a unique attribute. Provider of the detail data is a new jsp called detailsData.jsp. It expects that it is called with an URL parameter companyName. The jsp is one big example of bad practices, it is left as an exercise to the reader to find them.

<%@ page import="java.util.*" %>
<%@ page import="server.CompanyDetails"%>
<%@ page import="server.Company"%>
<%@ page import="javax.persistence.*" %>
<%@ page import="java.lang.reflect.*" %>

<%
    String compName = request.getParameter("companyName");
    EntityManagerFactory emf = Persistence.createEntityManagerFactory("jmaki-jpaPU");
    EntityManager em = emf.createEntityManager();

    Company company = (Company) em.createNamedQuery("Company.findByCompanyname").
        setParameter("companyname", compName).getResultList().get(0);
    CompanyDetails details = (CompanyDetails) em.createNamedQuery("CompanyDetails.findById").
        setParameter("id", company.getId()).getResultList().get(0);

     if (details != null) {
        out.println(" ");
        out.print("{ companyId : '" + details.getId() + "', " +
            "address : '" + details.getAddress() + "' } ");
        //out.println();
        out.println(" ");
    }
%>

Publish

I was attracted to jMaki mainly by its approach to events - publish/subscribe. So if I click on the master table, how do I get the right data, that is company name (well that is not quite the right data), and post the data to get the detail?
I needed to add to glue.js following:

function getCompanyDetail(company) {
    var encodedName = encodeURIComponent(company);
    var url = jmaki.webRoot + "/detailsData.jsp?companyName=" + encodedName;
    jmaki.doAjax({url: url, callback: function(req) { var _req=req; postProcessCompDetail(_req);}});

    function postProcessCompDetail(req) {
        jmaki.log("process? : " + req.status);
        if (req.readyState == 4) {
            if (req.status == 200) {
                            var response = eval("(" + req.responseText + ")");
                            //jmaki.publish("/yahoo/dataTable/detail", req.responseText);
                            jmaki.publish("/yahoo/dataTable/detail/addRow", {
                                 value : response
                             });
            }
        }
    }
}


jmaki.subscribe("/yahoo/dataTable/on*", function(args) {
    if (args.widgetId == 'master') {
        var masterWidget = jmaki.getWidget('master');
        var sids = masterWidget.dataTable.getSelectedRows();
        var company = masterWidget.dataTable.getRecordSet().getRecord(sids[0]).getData('companyName');
        //jmaki.log("targetId : onSelect request from: " + args.targetId);
        //jmaki.publish('/yahoo/dataTable/detail', company);
        getCompanyDetail(company);
        jmaki.log("targetId : onSelect request from: " + args.targetId);
    }
}); 

Unfortunately this piece of code took me several hours to complete. First, I did not see in the docs that there is something like dataTable variable in the widget - I had to examine javascript source for the widget, so I was able to find which row is selected. Second, even from sources I was not able to find out that there is that getRecordSet() function. I had to read Yahoo developer documentation. As I was in a place without internet connection, my attempts were constantly failing until reading the docs.
My first attempts were focused on using args.targetId a masterWidget.getIdMapping(), but those attempts really did not go anywhere - I just did not know if it should have work or if I found the right way how to get data.
I had also problems with lack of typechecking in Javascript. E.g. sids were displaying as e.g. 0, or 1 in debug prints, but getRecord just returned null or nothing, driving me crazy, because I thought that I passed string there and it did not work. It turned out that sids[0] is the answer.
getCompany detail is more or less copy/paste from some other example on the jMaki web site. From the docs I thought that just publishing req.responseText is enough. Again, it silently did not work, again problem with types, where API expects a map like object, not just string.

Subscribe

The detail table needs to get updated when Ajax call finishes. Here we go (from glue.js):

jmaki.subscribe("/yahoo/dataTable/detail*", function(args) {
        jmaki.getWidget('detail').clear();
        jmaki.getWidget('detail').addRow(args);
});

The page

The page is very simple, containing something like:

  <div id="content" style="height:400px">
    <a:widget name="yahoo.dataTable" service="data.jsp" id="master" 
        publish="/jmaki/master/onClick" args="rowSingleSelect=true"/>
    <br/>
    <a:widget name="yahoo.dataTable" id="detail" subscribe="/yahoo/dataTable/detail/"
      value="{columns : [
        { label : 'Company Id', id : 'companyId'},
        { label :'Address', id : 'address'}
        ],
      rows: []}"
    />
  </div>

It does not look like subscribe for the detail table does anything useful, I guess I am using bad format or something, so I update the table not by sending right messages to the table but by using the API.

The application

The application is one where master contains more data than detail :-)

If I click on the last entry, the detail table is automatically refreshed with new data:

I did it in approximately four months. I was convinced that resisting hunger is difficult. It is not, unless I am tired. In such a case my will is not strong enough. That ruled out sport activities - running, bicycle :-).

Another lesson learned was that it was good to plan on eating, for instance it is easier to follow plan such as I will eat something at noon, then two hours later an apple, then after another two hours a piece of pastry and turnip-cabbage.

I followed common advice to eat food that does not cause sharp rise of blood sugar followed by sharp drop, which leads to hunger. I started to eat salads at noon and I can confirm that it works, i.e. I will not be starving three four hours later, I will be just dreaming ... :-) anyway, I can stand it.

My own invention is that drinking beer actually helps. I mean when having one beer in the evening I have found it much easier not to eat though I was used to eat even after 8 p. m.

Another objective was to consume less cholesterol rich food. Eggs were easy to stop to eat though I really like them. Avoiding, or I should say eating less, meat proved to be more challenging because I did not know what to eat instead of it. We have have a lot of meat based foods. National habit. I have found however some alternatives over time such as beans with sauce, usually chili sauce, anything containing jalapenos, and Indian vegetarian food.

I created a simple IdM monitoring facility. The application has a Swing GUI, it connects to IdM through Apache/Jakarta HttpClient library. Application advertises itself only by an icon in the system tray. Currently the application monitors WorkItems. WorkItems data are reported for the currently logged in user as well as the total number of all WorkItems. There is also a possibility to view statistics about password activities. That includes password reset and password change. Data are presented as 3D charts using JFreeChart library.

Once the application is started, you will see a dialog asking for credentials first.

As you see, https is supported *). Url and user name are saved using Preferences API, so they popup next time the GUI is displayed.

After successful connect you can see data about WorkItems, you can also display data about password activity (last 5 days, this is hardcoded).

What caught me by surprise was the ability of JFreeChart library to scale data. If there is only one password reset, but 250 password changes, you cannot see values for password reset very well. What is supported, is that you select an area on the image. The image is then scaled:

On the IdM side, the hardest thing was probably finding out which APIs to use. What regards WorkItems I reused code from a former project, but password data - well LighthouseContext.countObject(Type.LOG, ...) is your friend.

The project took surprisingly short time to develope. HTTP communication is basically cut and pasted example from the HttpClient library. Connect dialog GUI was made in about 5 minutes using NetBeans form editor (Matisse). Password charts code is also greatly reused from an example. Finally, presenting the application in system tray is based on another example from JDIC project. I reused jsp serving data about WorkItems from a former project. I had to write a new jsp presenting password data. Rest of code is event handling, parsing data, bunch of requestFocus() calls. There are also some fancy features such as notifications of connectivity problems and automatic reconnect.

*) The only thing to make SSL work, you need to import the certificate to a truststore. I did something like:

/usr/sfw/bin/certutil -L -n s1as -d . -a

to export server's public certificate from Sun application server.

Then the certificate must be imported to JDK so that the certificate is trusted so the HTTPS connection can be successfully established.

C:\Program Files\Java\jdk1.6.0\jre\lib\security>keytool -import -file cert.pem -keystore cacerts
Enter keystore password:  changeit
xxx
Trust this certificate? [no]:  yes
Certificate was added to keystore

I made a small demo of how can Ajax be used in IdM. This was my first Ajax attempt, so the demo is not exactly rocket science. I used book Ajax Hacks By Bruce W. Perry, mostly first chapter :-). I also found a Javascript timer function somewhere on the internet. All in all the javascript part was not tough.

On the IdM side, I had to modify several jsps. headStart.jsp for starting the timer onLoad, bodyStart.jsp for embedding an HTML element, which is later modified by javascript. I also created an ApprovalProvider.jsp. It turned out to be the most difficult part of this exercise - finding right methods for searching WorkItems in the repository and for getting currently logged in user. What I mean by difficult is that it took maybe several hours to find right API methods.

All in all, I spent about 4+4+2 evening hours on that.

Picture here:

That red circle shows a field which is updated every 10 seconds without the full page reload, so if an approval is required by an activity in another browser, it suddenly pops up.

I thought that it was first Ajax in IdM - not at all, a guy from engineering told me that there already is some kind of ajaxized dashboard.

I have just finished reading the book. I wonder if anybody in Istanbul found that there is such amount of hüzün as described in the book.

As we have state (public) TV channels, each owner of a TV set must pay regular fee. Now there is a discussion if computers connected to the internet should be subject of fees as well. Germany will take the route. Sounds like you may be breaking the law when travelling to Germany and connecting to the internet and not paying fees.

This is just great. If applied here as well, Czech TV can just sit and collect money. They do not need to be competetive at all, just look for what else could be subject of fees. DVD/video players? They also have their own metric of success - it is not the number of people watching Czech TV (as any sane human being would thought) but rather it is ability to protect and enhance cultural heritage whatever it means.

What is next? Fees for breathing? We already pay fees for fod - it is called VAT.

The popular serie will be broadcasted in Middle East countries. It seems that offending parts will be avoided however. Now the list of those parts look appropriate for Germany 1933 - 1945 or medieval Europe maybe. Jews/Indian characters are to be avoided as well as representatives of foreign religions. Cool. What is next?

Our friends at Intel released Paxville. Paxville rhymes with paskvil (pasquinade). It is a joke only in Czech I guess. The poor CPU is however "joke" regardless of your nationality.

Here we go with my slides from our Java Forum from September. It is in English, describing various Sun and non-Sun technologies - application server, web services, ant, Hibernate, JMX, LDAP. There schematic code samples and a lot of screenshots. I guess I should add some comments. Maybe later.

Sometimes you just have to sit back and think what the programmer was thinking? I am not sure how the following thing happened, I suspect that somebody did not want to brought in API changes. Anyway, to give you some background - the customer application worked at a "developer workstation" or under light load, thus passing tests in staging environment, but was failing silently, in production environment. The net result was corrupted data. Until somebody inspected data, realizing they were not consistent, there was not any sign that anything went wrong. Here we go.

We start with two methods, writeToDataStore and writeToFileBasedStore:

        public void writeToDataStore(String data, String msg) {
                ...
                writeToFileBasedStore(data, msg);
                ...
        }

        void writeToFileBasedStore(String data, String msg) {
                ...
        }

What happened probably was that somebody has to add third parameter to writeToDataStore, since this was an API method it cannot be just replaced. I can only speculate why they chose not to overload the method, i.e. add writeToDataStore(String data, String msg, String moreData), they were probably afraid of bloated APIs so they came up with the solution, which is here:

        public static Hashtable table = new Hashtable();

        ...
        table.put("datakey", moreData); //*
        writeToDataStore(data, msg);
        ...

        void writeToFileBasedStore(String data, String msg) {
                String moreData = (String) table.get("datakey");
                ...
        }

You should see intent of the developer. However, what he really wrote was code working only in single threaded environment. When deployed to multithreaded environment, e.g. application server, the whole thing just break. You can start to think of your favourite scenario of a race condition. For start, what happens when multiple threads reach and complete line marked with //* simultaneously? Then they all enter writeToDataStore. The last thread's moreData wins and overwrites everything else.

What is the solution for the problem? Well, you can add overloaded, three parameter long version of the method, then four, five, ... long versions as new requirements arise. You can put the critical section in synchronized block, which you should definitely avoid. You can use thread local Hashtable, which is not nice either. I personally would add something like writeToDataStore(Record r) to API - remember OOP basics? Particularly encapsulation?

Last week I saw internals two of our products - DSAME 5.1 (pretty old), Creator 2EA (pretty new). Look at some DSAME exception handling code:

        try {
            connection = openConnection(); // possible SQL error here
            statement = createStatement(); // and here
        } finally {
            statement.close();
            connection.close();
        }

Comments are mine. We looked for a database problem but all we saw was some kind of WRITE_ERROR, it turned out to be caused by a NullPointerException which was masked again by swallowing the exception and printing out only message (which is none for NullPointerException). Bad bad. The NullPointerException was of course caused by some SQL exception and by ill-constructed finally statement.

On the other hand see following:

        try {
            ...
        } catch (Exception e) {
            throw new RuntimeException(e);
        }

This is much better, original exception is preserved - message and stack trace.

Sun acquired Forte for some $400 M some 5 - 6 years ago. Products are gone or in sustaining mode, people left, were fired. Sales were never good. Why? One guess because the software was not based on Java thus making it difficult for sales force to say our story is Java, but we offer Forte for integration. Not having dedicated sales force did not help either. By the way, our hardawre sales were sky high so nobody really cared.

Cobalt did not work in my opinion because Sun sales were not (and are not) interested in selling $400 pieces. Given that they have millions as their target, you can see why. Partners maybe could deliver on that but then again, I see our partner model as a bit broken at least here - but when I ask other people from neighbouring countries, they say the same.

Then we go for StorageTek, unlike Cobalt, this is expensive stuff. Thus chance that it will be accepted by our sales force as relevant for their quotas are high. In this regard, we could sell e.g. aeroplanes, except that we are a bit more experienced in data center :-)

Finally SeeBeyond - I believe that Waveset aquisition was/is successful. Product itself is great, based on Java so no problem here. I really like it. Price is built to be interesting for sales force, customers need it. I wish we buy more companies like Waveset. It seems that SeeBeyond is similar in this regard - Java, price, quality, customer needs.

Radim and Kamila were borned on Tuesday 5th July.

    public int getMaxSleepTime(Date time) {
        Calendar cal = Calendar.getInstance();
        cal.set(2005, 6, 5, 15, 45); // month is 0 based
        
        Date date = cal.getTime();
        int hours = date.before(time) ? 2  : 10;
        return hours;
    }
    

Just look at SeeBeyond. Another link is at yahoo news. So now we have the best integration software on the planet.