Andi Egloff's Weblog

Non-blocking NIO sockets in the JBI HTTP Binding Component (BC)

I received some questions about the HTTP BC (part of the Java EE SDK Tools Bundle and the project OpenESB) - and what the advantages are of it supporting the processing of request-response and one-way web service invocations without blocking threads.

So I will try to give a little background on the JBI architecture, illustrate what would happen if we did block threads - and last but not least show the advantages of supporting asychronous processing.

The JBI architecture is based on an asynchronous message bus that can be thought of as similar to in-memory JMS queues. The messages are XML based – and the processing of the request and sub-sequent response are completely asynchronous – they will be handled by different threads and there is no need to block a thread anywhere in between.

This allows for an extremely scalable system that is capable of handling a huge number of concurrent requests with a minimum of resources – in an extreme case where it might be tuned for throughput rather than response times for example it could be handling 10’000s of concurrent requests with under 10 threads - environment permitting of course.

Given this architecture, it would not fit well to use an HTTP connector that blocks threads - and doesn’t have an asynchronous API. Some of the implications this would have in the HTTP BC are:

Implications of blocking

Threading

* Threat of resource starvation and dead-lock (if there are recursive calls within services they can easily exhaust the thread pool – and stop the response from getting processed which might let others progress)

Scalability

* The large number of threads needed to scale at all will consume a lot of related memory which will limit the number of threads feasible
* Because the number of requests we can handle is limited by the number of threads we can efficiently use, we are limiting scalability a lot
* Even without taking efficiency and memory into account, the JVM will ultimately limit the number of threads possible at all - and hence concurrent requests

Performance

* More threads means more context switching, hurting performance
* Limiting the number of concurrent requests we can handle means we can’t always fully utilize available CPU or network bandwidth, i.e. we might be “waiting” for a response so we can free a thread and proceed with the next request

Fortunately this does not have to be. The HTTP BC uses the Grizzly NIO framework including the asynchronous request processing (ARP) extensions that is part of the Glassfish application server.

Not blocking threads makes for an excellent match with the JBI architecture and has these implications (Note that the HTTP BC in the appserver 9.0 currently only uses Grizzly on the server side):

Benefits of not blocking

Threading

* Because nothing blocks there is no threat of resource starvation or dead-lock
* As an example 3 threads in the BC could handle any number of concurrent requests (for higher response times we can choose to add more), e.g.
  thread 1 – Handle the incoming traffic (asynchronous Provider), normalize the message, send it to the JBI “queue” (normalized message router)
  thread 2 – Handle messages off the JBI “queue” (normalized message router), send requests via the asynchronous client API
  thread 3 – Handle responses (asynchronous client API listener), send them to the JBI “queue” (normalized message router)

Scalability

* Very limited number of threads, minimizes the memory footprint
* Because the number of concurrent requests we can process is not limited by the number of threads we can realistically use, we can scale to much larger number of concurrent requests

Performance

* Less context switching which might hurt performance
* Never “waiting” for responses to proceed means we can fully utilize available CPU and/or network bandwidth

Scalability and performance are some of the primary selling points for an integration platform - it makes sense for us to make use of the full potential of JBI.

Posted at 06:40PM Nov 28, 2006 by Andreas Egloff in SOA  |  Comments[0]