Andi Egloff's Weblog

Comet Basics

There are some very good detailed technical explanations of Comet already, but it seems there is some room for a basic discussion of what it is about, what it is good for – and how it jives with AJAX.

Comet is a "made up" term - so unlike AJAX (Asynchronous JavaScript and XML) this did not originate as an acronym.

If we view AJAX as the ability to do "invisible HTTP requests" behind the scenes via javascript, coupled with some dynamic HTML - then Comet it primarily an extension or variation on how the communication over HTTP is done.

Essentially it allows to "push" or "stream" data to the web client via standard HTTP GETs instead of for example polling for updates at regular intervals.

How can it do this, considering the browser always has to initiate requests?

Well, first imagine that you do an HTTP GET request as with any old AJAX framework. The server gets the request, but instead of responding immediately with nothing if it doesn't have data, it then waits for data to arrive - for example from an event mechanism, JMS or any other asynchronous source. Only once it gets data will it actually write data into the response for the HTTP GET. It also typically does not finish the HTTP GET request at that point, but will continue to wait for further data - and write more to the response when it arrives ("stream").

So that is what is meant when people say Comet will "always keep a connection open" - it may re-establish a connection by sending another HTTP GET request when a previous one is closed for various reasons. This way there is always a "channel" already open for the server to send data to the web client.

At this point your gut may tell you - open connections and waiting sound terribly expensive - does any of this scale? Is it bad for the network or server?

Done right and used right this can be a very efficient way to send events to web clients; in fact, it can save a lot of unnecessary “polling” requests. Not only can it be more efficient, updates will get to the clients quicker (lower latency) than when polling.

The open connection itself primarily costs the server a socket it needs to allocate per client – with associated (“kernel”) memory. What you do *not* want to happen is for the server to block for example a thread per request when it waits for data to arrive – that would be Comet done in a fashion that will not scale well. Instead, use a non-blocking server – for example one that uses Java NIO non-blocking sockets and can deal with the request and the response separately, in an asynchronous fashion.

There are several products on the market already that do this efficiently; and it is moving from more specialized event driven solutions to the main stream web servers too.

For example, my Québécois pal Jeanfrancois just recently added a Comet extension and example to the NIO based http connector “Grizzly” used in the GlassFish application server. This is actually based on the same asynchronous extensions he had added during our work to support non-blocking NIO sockets for Java Business Integration (JBI) (which we presented at a BOF at this year’s JavaOne).

Unlike JBI, some technologies such as Servlets currently have a little catching up to do until such asynchronous, event driven responses are dealt with in a standardized way and fit well.

There is plenty of controversy about the term Comet too; some like to call it “hanging get” – I don’t like that term too much as the “hanging” sounds too much like blocking to me. Others call it the "slow load technique" ("slow" might be a bit misleading) or "long lived HTTP". And if have this feeling that something is odd about the term Comet - yes, Ajax and Comet are both names of cleaning products. Whatever it is called, a Comet style interaction for AJAX is a worthy concept to name. I probably picture more of a meteor shower in considering when and how the data will flow - but hey that’s just me :)

As with AJAX, Comet can be over-done and abused. Care has to be taken to only transfer relevant data at rates that can be digested by the target platform.

For more details also have a look at Alex Russell's original Comet term proposal or Greg Wilkin's white paper on Ajax, Comet and Jetty which does a very nice job of explaining why you want a server that fits this style of interaction well.

Posted at 11:08PM Oct 02, 2006 by Andreas Egloff in Software Architecture  |  Comments[1]