Head in the Cloud: Project Caroline musings, facts, viewpoints

I have two excuses for having fallen somewhat silent over the past month or so neither of which invokes the myth of the Medusa. The first is that this being the middle of the season as far as motorsports is concerned, I've been spending far too much time in the service of  spending less of it on a per lap basis. But more to the point, I have failed to be more energetic on these pages, as most of my fuel budget has been invested in the first of a series of rewrites of CWS (Caroline Web Server). While outwardly perhaps the changes appear somewhat minimalist, the innards, somewhat akin to cresting the uphill at Lime Rock, have been significantly displaced.

As pC begins to take shape as a commercially viable platform, most of the team is rethinking much of what we've learned and accomplished over the past two years. We are aware that this might be our last opportunity to get it right, for once we go commercial, much of pC's interfaces and structure will be far less malleable, for better or worse. One of the key issues that has been wonking us for a while now is how, where and when to make both the platform and DE layers less Java-centric. Were this 1979, our approach would likely have been to supply bindings for whatever languages we could think of. If this were the fall of 1996, perhaps we might have been seduced by the siren's song of IDLs and Corba. But in today's world, it seems far more reasonable to navigate these waters by opting for the simplest, most neutral representation one might find, and matey, that has to be JSON. 

While the platform might be going in a slightly different direction (more on that as it develops), in the case of CWS, we choose to obliterate the current Jeri/RMI implementation in favor of the more ReSTfull landscape afforded by JSON over HTTPS. As the user view of CWS appears to be a single package, in reality, as it both uploads, installs and initially configures the on-grid payload as well as providing the command interface for controlling the farm, its serves two disjoint functions. The installer/agentserver launcher portion of CWS is accomplished purely though webdav and platform API communications, so it remained largely intact. But on the control side, everything has changed and hopefully for the better.  How many non-Java environment will wish to control Java servlet deployments is certainly questionable, but just to plant a little seed, the way CWS is being reorganized going forward, it hopefully wouldn't take a whole lot of effort to run something other than Glassfish as the web server instance. Besides, Glassfish isnt just for Java anymore, what with its Ruby and PHP support.

Being a devotee of simplicity, largely due to my somewhat limited mental capacity, once I wrapped my head around the notion of a document driven API, I found working with JSON to be quite enjoyable. In the end, what had been a dozen or more Java methods boiled down to three main URLs and documents.

The control document informs the CWS agent server to update the current state of the server farm per the contents of the following JSON document:

   {"serverCount" : 0 | 1-32 | "0", "1-32" | "auto",
    "minServers" : 1-32,
    "maxServers" : 1-32      
   }

This simple document handles all the control features for the farm. The min and max server fields are only used if the serverCount attribute is set to "auto" in which case the CWS agentserver will monitor load and flex within the min/max server count specified. Otherwise, set the serverCount to non-zero and the farm runs with the specified number of servers, set it to zero and all the server instances vanish. Cool, huh? Commands can be issued to the agent server at the appropriate external ip address or DNS host binding at port 9464, which enables any on or off grid non-Java application the ability to flex the farm through simple documents set over HTTPS. A GET retrieves the last issued command to the far, while a PUT will set the farm state to the requested configuration. A DELETE will destroy the farm entirely.

The info documents presents a view of the last state captured state of the farm. The contents of the JSON document appear as follows:
{"port":80,
    "monitoringInfo":{"serverCount":"2",
                      "upTime":"0",
                      "suggestedServerCount":"0",
                      "intensity":"0",
                      "averageRequestTime":"0.0",
                      "arrivalRate":"0.0",
                      "serviceRate":"0.0",
                      "projectedIntensity":"0",
                      "queuingProbability":"0"},
    "serverCount":3,
    "autoFlexing":true,
    "state":"Running",
    "maxServers":16,
    "minServers":1,
    "networkName":"cwsfarmNetwork",
    "requestedCount":1,
    "extAddrName":"cwsfarmExtIpAddr",
    "farmName":"cwsfarm",
    "version":"0.3.0"
   }

The deploy document is used to deploy, undeploy and list applications associated with the server farm. Applications are deployed by providing a URL which contains the target name of the war to be deployed. The contents of the war are transmitted as multipart formdata. To deploy or replace an application, a POST is issued to /deploy/my.war where my.war is the application to be installed. The contents of the formdata is unwound onto grid and then submitted to the farm for deployment. To undeploy an application, a DELETE is issued to   /deploy/mycontextpath where the mycontextpath is that of the application to be undeployed. A GET command of /deploy returns a list of the deployed war files (in the next release this will be the changed to reflect the deployed context paths).

One of the cool details of the implementation is that the agentserver's uses the Java internal lightweight http server, which hopefully means little appreciable bloat to the memory footprint. In fact, document retrieval via a brower is very, very fast, so performance looks on par, if not better than the previous methodology. On grid the documents are unencrypted, as we employ a the platform's L7 Virtual Service to handle the encryption/decryption from off grid clients, so requests from on grid within the same network can be done in plain text. So while the Java API still is in there and available for those whom Java remains their first love, we've now exposed a mechanism by which others can play too. Not a golden fleece perhaps, but a pleasant voyage none the less.