In this post , I briefly try to push how XMPP can be used as a infrastructure - and not just as 'yet another chat protocol'.
It is not as descriptive or authoritative as I would like it to be ... but here goes nothing.

The abstract from RFC 3920 says :
"This memo defines the core features of the Extensible Messaging and Presence Protocol (XMPP), a protocol for streaming Extensible Markup Language (XML) elements in order to exchange structured information in close to real time between any two network endpoints. While XMPP provides a generalized, extensible framework for exchanging XML data, it is used mainly for the purpose of building instant messaging and presence applications that meet the requirements of RFC 2779."

For the purpose of this discussion , let us take a small subset of what XMPP provides which allows us this :

1. As specified in section 4 of 3920 , an xml stream between two entities which allows for exchange of structured xml stanza's at realtime.
2. A binding to tcp and defines how to secure and authenticate the stream.
3. Defines the top level stanzas allowed in this xml stream. These model request-response , broadcast and one way meps.
4. Server federation , routing rules , multiple resources per node built into the protocol.

Let us consider the implications of these.
We have an authenticated and secure stream between two entities - so this asserts the client identity.
This explictly identifies an entity within , not only its server - but if federated , across the XMPP universe of federated servers ! This guarantees that any stanza orginating anywhere in this grand federation of XMPP servers will be unambigously routed to this specific client endpoint
Defines an xml stream ... most data transfer protocols have already moved , or are moving towards standardizing on usage of xml : so XMPP provides a native transport mechanism for these.
The MEP's supported allows us both synchronous communication and asynchronous notification - in either direction: hence both push and pull models are natively supported.
The protocol itself allows for presence notifications through a publish subscribe model.

Now , the XMPP community has come up with a variety of protocol extensions which have been standardized as JEP's.
A few notable ones which are relevent for this discussion are :

• A flexible discovery mechanism : allowing various clients/servers to interoperate. It helps either side to discover the supported subset and allows protocol extension without breaking compatibility.
• Multi User Chat - though it defines how to set up a 'chat room' : it can be used or modeled to multicast message's to a subset of interested entities.
  
• A powerful publish subscribe extension for XMPP.
• Advanced message processing allows specifying additional delivery semantics for messages.
• HTTP Binding - allowing XMPP to be bound to HTTP. (More on this later)

All this taken together allows for providing solutions for some challenging problems.
We could use xmpp as a middleware for realtime communication and use it for both synchronous and asynchronous notifications. Some examples :

1. Pushing information to interested entities - like stock quotes to customers or brokers , etc.
2. A whole bunch of publish subscribe usecases can be solved.
3. Multicast stanza's as messages to interested entities.
   
4. Allows us to write presence aware applications - the entity in question need not really be another 'person' ! Also , use of CAPS and PeP will definitely make this world of applications much more interesting !
   

Now , add JEP 124 (httpbind) into this picture and things become really interesting.
Httpbind essentially can be considered as way to tunnel a xml stream based protocol on top of http : no , not the old CONNECT hack that most implementations use , nor a dumb frequest poll mechanism.
As an analogy , you can consider it to be similar to how you would simulate a persistent stream on top of UDP - only it is HTTP here instead of UDP.
The extension takes care of retransmissions , packet loss , request/response id , etc and makes allowances for HTTP characteristics .

Hence , all the power of XMPP is now available over HTTP.
So be it a rich client application or a rich internet application - both can talk XMPP and harness its power !
And with it , allows all usecases of using XMPP as a realtime messaging middleware available to all developers/users on most platforms - be it a RIA or a thick client.

XMPP - coming to a browser near you

Here is a report from CNET News talking about Sun regaining market share - and more importantly , gaining revenue while all other major vendors lost !

Just got back yesterday from Portland after the XMPP interop event ... it was a really cool event.
We had 6 completely different servers interoperating without too much of a problem - including when some funky multi-byte testcases were tried in S2S which stpeter dreamed up There were some hiccups when TLS was enabled with SASL External / dialback ... but nothing too serious which could not be fixed.
The achievments of the event are pretty incredible - there is no other protocol on instant messaging which can possibly boast of achieving something like this - most dont even have an implementation of size more than 1 !

While the testing was going on , in parallel we had some very productive discussions , including to clarify on aspects of the RFC's and JEP's : what was debated upon would get translated into changes or discussions at standards-jig pretty soon !
Portland looked "pretty" - from what little I saw , and the event venue itself allowed for a real nice view of the downtown.
I could not attend OSCON unfortunately ...

Now to explain what is checked in ... the main purpose of the whole project

Essentially a simple framework for authentication has been checked in. Let us take a look at the Classes involved.
There is a AuthenticationManager in net.java.dev.xmpp.client.core , which handles the actual session creation and management.

In the collab api , you create a collab session using a CollaborationSessionFactory - which delegates the actual session creation to an instance of CollaborationSessionProvider.
The default CollaborationSessionProvider is org.netbeans.lib.collab.xmpp.XMPPSessionProvider - this allows us to create a direct socket based session to a XMPP server.
Other CollaborationSessionProvider that can be used are :

1. org.netbeans.lib.collab.xmpp.XMPPSecureSessionProvider - This provider allows you to use legacy SSL to the server. This is a deprecated jabber protocol where the server always listens on SSL mode : like what you have in http/https case.
2. org.netbeans.lib.collab.xmpp.XMPPComponentSessionProvider - This allows you to create a 'trusted' component session : please refer to JEP 0114 for technical info.
3. org.netbeans.lib.collab.xmpp.ProxySessionProvider - This provider allows you to create a XMPP session tunneling through HTTPS or SOCKS proxy. The actual proxy to talk to is picked up from the service URL.
4. org.netbeans.lib.collab.xmpp.XMPPSecureComponentSessionProvider - allows creation of component session when server is in legacy SSL mode.
5. org.netbeans.lib.collab.xmpp.httpbind.HTTPBindSessionProvider - allows creation of a HTTP based XMPP session as per JEP 0124. The connection related info is picked up from the service URL.
   

To use a non-default provider , you will just need to specify the classname as the parameter to CollaborationSessionFactory constructor (in our case , constructor of AuthenticationManager) ! Rest of the details are taken care of by the collab api - and it exposes a uniform behaviour irrespective of the underlying transport : even when the underlying transport is not a dedicated socket stream like in httpbind case !
Ok , almost ... there are a couple of little details a user still has to take care of .... let us finish those of too.

If in the process of session establishment , an untrusted certificate is provided as part of TLS , the decision whether to accept the certificate or not depends on what the provided CollaborationSessionListener says.
That is , if the listener is an instance of org.netbeans.lib.collab.SecureSessionListener , then the corresponding "onX509Certificate" method is invoked ... else the certificate is treated as untrusted and session terminated.

If the service URL points to the form "host:port" , the collab api uses that directly as the destination XMPP server/port.
In case it is of the form "domain" , it tries two things one after the other.

1. It tries to do a dns query for the specified "domain" to try to find out if there is a xmpp service registered for it. If yes , it uses that as the destination server. This way , the actual xmpp server hosting a domain could be decoupled from the domain itself. Refer here for more details.
2. If (1) is not the case , the api treats "domain" as XMPP host with default port of "5222".

Well , not always - service URL is overloaded to mean something specific in case of two providers ... as explained below.

ProxySessionProvider and HTTPBindSessionProvider use the service URL as a way to obtain required parameters.
In all the other providers , the service URL is just the destination "host : port" (or domain in case you want to do a dynamic lookup of the XMPP host servicing that domain).
In these two providers , the semantics are different , let us see how :

• ProxySessionProvider

As of now , the service URL is expected to be in the form :
<protocol>://<proxy host>:<proxy port>?(name=value)(&name=value)*

The protocol can be one of "socks" or "http"/"https" : socks will tunnel through a socks proxy , while http or https will tunnel using http CONNECT.
The query names currently handled are :

1. "keepalive" : After authentication succeeds , the client will send whitespace pings to the server after each keepalive interval : this is so that intermediate proxies do not terminate the connection assuming it is inactive.
2. "authname" , "password" : The authentication credentials to talk to the proxy.
3. "service" : The 'actual service url' !
4. "usessl" : Whether to use legacy SSL to talk to the XMPP server after tunneling through the proxy.

So an example service URL would be of the form :
https://myproxyhost:8080?keepalive=120&authname=test&password=pass&service=xmpp.sun.com&usessl=false

• HTTPBindSessionProvider

The service URL in the case of this provider is the actual URL used to get in touch with the httpbind gateway ... except for the fact that the query parameters are removed.
The actual query parameters are defined in org.netbeans.lib.collab.xmpp.httpbind.HTTPBindConstants ... and allows for a wide range of customisations. The only mandatory query parameter is the "to" parameter.
This specifies the domain the user wants to log into : and should be serviced by the specified httpbind connection manager.

Another important point related to httpbind is related to proxy support.
The default two ConnectionProvider's for httpbind do not support proxies directly.
But , they use URLConnection to get to the httpbind gateway , so you can specify the appropriate proxy using the corresponding java properties and expect them to be used.
The main reason why they do not support it is 'cos the 1.4 java.net.* code does not provide a way to explictly specify a proxy... as and when the api moves completely to 1.5 , this implementation will get revised.
Ofcourse , you can always write and provide a custom ConnectionProvider in the service url which picks up the proxies and uses it !
Please note that "proxytype" and "proxyhostport" is expected to be provided even if you explictly set the java proxy vairables ... this is so that is the collab api's move to 1.5 and start using the java.net.Proxy class , the client code should not need to be modified.

So a sample HTTPBindSessionProvider service URL would be : https://my_gateway_host:port/codebase/httpbind?max_buffered_bytes=1000000&max_buffered_packets=1000&to=sun.com&proxytype=http&proxyhostport=myproxyhost:port


Other than specific customisation's , there is nothing much else that needs to be taken care of !


[Technorati Tag: XMPP]
[Technorati Tag: Sun IM api]
[Technorati Tag: xmpp-im-client]

First off , I took much more time than I anticipated before I could finally make some checkin's to xmpp-im-client.
Without giving excuses , I will just mention work as the reason

Ok , now that we have cleared that , what has been checked in ?

1. Very basic skeleton of the project - how it would be structured , how it will use external api , etc.
2. The first few pieces of code ! (More on this below).

The basic skeleton of the package structure , build structure , etc has been checked in - if you are part of the project (if you are not , what are you waiting for !) , feel free to comment about it in case you have any reservations or suggestions.

A few classes have been checked in :

1. A session manager which takes care of creating session's and handles reconnections as needed. This is envisioned to be common across UI modules.
2. A basic simple CLI based UI has been checked in - as of now , it does nothing other than create a XMPP session using the collab api (using the wrapper described in prev bullet).

So session creation is done .... almost.
What else can be done ?

1. Change the sessionprovider passed to AuthenticationManager to try out different providers : legacy ssl , httpbind , proxy support , etc !
2. If required , implement org.netbeans.lib.collab.AuthenticationListener also in CliSessionListener - this will allow you to select and control SASL based authentication.
3. For more advanced SASL support - like custom client SASL modules , along with (2) above use <AuthenticationManager>.getFactory().getCollaborationSessionProvider().registerProvider( <SASLClientProviderFactory> ) !
   

Though the code checked in looks deceptively simple , it handles almost everything required for session creation - including reconnection when connection gets dropped.
The next functionality that we target will be just reusimg the CollaborationSession that we created using the above to build on to add more complex functionality !

Check out the discussion forum where I have added some blurb on how to test this.

So here to happy IM'ing !!



[Technorati Tag: XMPP]
[Technorati Tag: Sun IM api]
[Technorati Tag: xmpp-im-client]

Initial idea was that I will just forge ahead with design/coding and add stuff/refactor things as they develop.
When I saw that a small number of other developers have joined in the effort , decided to slow things a bit and atleast discuss the design before going ahead. The smattering of snippets on my workspace will remain there until we come up with some basic agreement.
This is the right time to join in case you want to influence the project from the start !
The unexpected delays have been , as I explained before , due to some deadlines at work - which have been cleared : full speed ahead !

I have also created a new Category "XMPP_IM_Client" in my blog under which I will post discussions and posts on this topic.


[Technorati Tag: XMPP]
[Technorati Tag: Sun IM api]
[Technorati Tag: xmpp-im-client]

Unfortunately , caught up with beta build for JES5 ... so that means not much time to blog and even less to work on xmpp-im-client
I checked in a barebones structure there - no code , just a build file which pulls the required dependency.

Will get to posting some actual code this weekend ... so will keep all posted !
In the meantime , feel free to join the project like others have done already

Have you seen Matisse ? Most probably you have ! It is an award winning GUI builder which comes with netbeans.
It is one kickass GUI builder ... and even a person like me can create a UI using it

Now , eclipse folks have long since been clamouring for an equivalent ... and looks like they are finally getting their wish !
Matisse4MyEclipse is out ... hope it is as good as the netbeans version.

As previously promised , I plan to start off writing a client.
The focus will be more on exposing the api exposed and (very) less of the UI.
So , either this code can be used directly to write a more fancy UI layer on top , or it can be used as a tutorial on how to understand and use the api.
Primary focus will be on the latter : hence code will be as clean and readable as possible with copious meaningful comments   (Normally , both of these would be false for my code )
Where is the code going to be ? It will be here at dev.java.net - feel free to join the project !
I have just got the project approved and am a bit busy right now ... so the project is empty as of now - but expect action pretty soon !
The rate of progress would typically be not very fast and this blog will be in sync with it : as a new feature gets implemented , the related code and api would be discussed here.

Next entry : what are the dependencies of the api , how and where do we get them from , how to get and setup the api.


[Technorati Tag: XMPP]
[Technorati Tag: Sun IM api]
[Technorati Tag: xmpp-im-client]

  I was thinking up how to illustrate the various aspects of the IM api ... and rather than write up posts about each facet with info about them , my 'current' idea is this :
We will author a client - you and me together.
It wont be a fancy swing client : I am horrible at UI anyway
Instead of just writing dry posts with some code snippet , what I will do is actually build a new text based client from scratch using the IM api.
We will incrementally add functionality to it ... showcasing each aspect of it as we proceed.
It will not be exhaustive , obviously ... and you will be able to write far more fancier and powerful clients than what I will end up authoring here.
But the basic idea of how to use the IM api should be evident.

As I illustrated in the previous post , the client as such is going to be agnostic to the underlying communication mechanism ... so you can easily switch to using different session provider to use httpbind , tunnel through socks/https proxy ,  use legacy SSL mode , etc : just a couple of line changes.
The rest of the client would remain the same.

Most probably , I will create a new project in dev.java.net with view-all permission.
So let the series begin from next post !

  When I had bought Halflife 2 about  year or so back , the bundle had come along with Counterstrike Source.
Initial pains with the steam platform which Valve uses almost made me uninstall the whole thing ... 5 cd's for insallation and then another 5 hours to update (/decrypt/whatever they call it) ?!
Thankfully , I stuck with it and continued on with HL2.
Never got around to completing HL2 ... the stress on puzzle solving was irritatingly high and pretty soon I got bored and turned my attention to Counterstrike.
I have never looked back since - though the bots typically suck , the online gameplay is pretty awesome !

Considering my connection speeds at home , I am really impressed at the low latency and the gameplay expierence. You do get to meet lot of 'interesting characters' online - including damn good players , people who are pretty passionate about the game and the normal bunch of cheating losers , so-so players like me , and n00bs ... it is nice fun all in all !
So now , when I am not doing anything interesting , I am on CSS

Let us consider a simple client which will use IM api and talk XMPP directly to the server.
The only modification to this client to use HTTP would be in the session creation phase , so I will restrict the client to that part of the code.
If you want to take a look at a more 'full fledged' client , take a look at org.netbeans.lib.collab.tools.Shell.


The behavior of this code is as follows :

• Lookup the system property "org.netbeans.lib.collab.CollaborationSessionFactory" if specified , use that as the session provider factory , else.
• Fallback onto the default CollaborationSessionFactory delegates to a direct socket based XMPP stream : org.netbeans.lib.collab.xmpp.XMPPSessionProvider.

The parameters are :

1. Server url - this is an overloaded parameter. For a direct stream based XMPP connection , this will be of the form "host:port". For our HTTP case , it is slightly different - and I will detail it below.
2. The user and password specified are used to authenticate the user. In a later post , I will write about how to use SASL.
3. The CollaborationSessionListener specified is the default listener to dispatch events to. More interesting things can be done with subinterfaces of CollaborationSessionListener - more on that too later
   

For a direct xmpp case , the above will look like this :

A bare bone CustomCollaborationSessionListener will just implement the "public void onError(CollaborationException e);" without taking any action about them.

Thats it , your basic code will work now (the server specified above is the netbeans collab server - create a valid userid and give the code above a whirl !).

How to make this HTTP enabled ?
Just two changes :

• Specify the HTTP session provider class explicitly using :

1. CollaborationSessionFactory _factory = new CollaborationSessionFactory("com.sun.im.service.xmpp.httpbind.HTTPBindSessionProvider");
2. Or , set the env variable "org.netbeans.lib.collab.CollaborationSessionFactory" to "com.sun.im.service.xmpp.httpbind.HTTPBindSessionProvider" before creating the session factory.
   

• Change the server to point to a httpbind connection manager which is JEP124 compliant. Additional parameters can be specified to this url as query params : refer to HTTPBindConstants for list of actual parameters.
  

So , a JEP124 version of the initial code would be :


It is mandatory to specify the domain you want to talk to using the 'to' parameter. I am using two other parameters just to illustrate how you would go about customising more.
The rest of the code which manipulates and uses the returned _session is the same irrespective of whether it is direct XMPP or through HTTP.

Some caveats about the HTTP provider in collab codebase from the top of my head (ignore these if required) :

1. It does not support proxies directly , but just relies on URLConnection. So to use proxies , the user will need to explicitly set the "http.proxyHost" and "http.proxyPort" system properties. (*More on this below.)
   
2. It does not honor the timeout's set as part of the initial handshake. (** More below).
   

* This limitation exists since the im module is also used from java 1.4 VM's. The ability to specify a per connection proxy was introduced only in java 1.5 ...
How to fix this ?
It is very easy for a developer to write a custom impl of ConnectionProvider to pick up the "proxytype" and "proxyhostport" param from the service url and use it to connect to the httpbind gateway (If you want to go with default impl itself , just extend it and override the openConnection(URL) method).

** As part of the initial handshake with the gateway, the client and httpbind gateway arrive at timeout's for the request.
These are not honored at the client side 'cos of the same limitation above. Custom implementation can override and use the setReadTimeout() and setConnectionTimeout() methods are appropriate.


If there is a need , I could always write a simple implementation of ConnectionProvider for java 1.5 which gets over the above limitations.
What to expect next ?
What about how to use the basic client and start using TLS and SASL ?!
Yep , that should be fun - so watch out for next update soon !

In the recent IFR release of Sun Java System Instant Messaging , we shipped our first implementation of JEP 124 compliant client and gateway.

What does this provide ?

• Access to the Sun's XMPP server through HTTP.
• The IM client that we bundle with the product can now talk HTTP through any compliant connection manager to allow XMPP access.

  The httpbind gateway (as we call it) is a servlet deployed in a webcontainer which provides HTTP access to clients. You can consider it as a protocol gateway between compliant HTTP clients and our pure XMPP server - thats right , the server *does not* know anything about http : everything is encapsulated and managed at the gateway itself.

  The default client that we ship is built on top of our IM client api which is hosted in netbeans under the collab project. The actual protocol is abstracted away at the api level and the client does not deal with XMPP (or whatever is the underlying protocol). The api implementation which provides this client side access to XMPP through HTTP is available opensource , give it a whirl ! (Any and every help will be provided by me - drop me a note here )

What all does this allow for :

• Obvious first case : use our client and our httpbind gateway and talk over http through http proxies (not persistent CONNECT , but POST requests )
  

• Use a third party JEP 124 compliant client - either a AJAX client like JWChat , or more heavier cousins and talk via HTTP to httpbind enable XMPP access.
  

• Use the api and develop your own client - like what the collab folks have done with the collab UI within netbeans : and how do you make this HTTP enabled ? Just change the session provider - thats it ! (I know it works 'cos that is all I did to support this in our client )
  

• Use constrained clients - like from a mobile phone , etc : talk http and still be 'online' (subcase of 2 actually) ... internally for demo's , we do have a J2ME midlet which does just this.

  Ofcourse , this was just one of the main feature additions to IFR release - the most notable among them being server pooling for high availability and enhanced security through starttls and sasl support (finally 1.0 compliant !) among many others.

Next blog entry , I will try to give a rough idea about how to go about using this new code - some snippets are being promised You will see how simple it really is to dish out your own client , all the protocol heavylifting is already handled at the api level - so let loose your UI skills !



[Technorati Tag: XMPP]
[Technorati Tag: Sun IM api]
[Technorati Tag: xmpp-im-client]

A friend of mine forwarded this excellent preso by Roumen , give it a whirl !

Check out the server details page for Sun Java System Instant Messaging at jabber.org.
The feature score here will definitely be improved in coming releases