Scorpion's Corner
Tuesday Dec 19, 2006
Clustering in JBI Components 2
What’s in store for JBI?
Each server instance, whether it is standalone, DAS or clustered
will contain a JBI runtime. DAS will contain Facade mbeans that
will be communicating to the cluster instances or standalone
instances based on the target information.
Basic JBI runtime will not have any special capability to handle clustering. The components on top of it will be aware of EE clustering. So, it will be the component's responsibility to work in a clustered manner. For example, a BPEL process doing a correlation will need the messages to routed to the same instance. However this will be done in a way specific to the component. For example, BPEL engine deployment to a cluster will use a group ID to identify the cluster. This can be the cluster name. In case clustered environment once we deploy the ServiceAssembly(SA), the same will be replicated in all the Sever Instances in the cluster. This mandates that we need to have a mechanism of identifying what is the component that is servicing the request on the cluster. This will boil down to associating the Node Agent Name with the endpoint that is exposed if there is no identifying port as in case of HTTP/SOAP BC.
That said we need to thrash what are the scenarios for any
component and the what prompts the need for the load balancer for
the components and then the strategies for handling the same and
look for what Glassfish provides via API for that.
Scenarios:
1. Outbound communication from the BC to the external system.
a. In Only MEP: Since the message is destined for the external
system and the WSDL contains the external system information the
message can be spitted out without any issues since the message
comes out of the correlated BPEL instance.
b. In-Out MEP: In this case the external system sends either
ACK/NAK to the SA/SU instance in the cluster. This case needs to be
taken care of. This has a catch in view of HA/ or failover, unless
otherwise we have XA enabling with respect to the EIS we can not
guarantee the failover, but since this scenario ends up in deadlock
we can not guarantee the failover.
2. Inbound communication from the external system into the BC
a. In Only MEP: Since the message comes into the BC and is targeted
at a particular instance (endpoint) of the BC this has to be taken
care.
b. In – Out MEP: Since the message comes into the BC and is
targeted at a particular instance (endpoint) of the BC this has to
be taken care.
In both the above cases as in case of EJB container for having
failover support we mandate that the resource happens to be an
XAResouce and is transactional.
Need for Custom Request Router (Load Balancer): I rather do not want to call this as Load balancer rather I would like to call it a request router, because this appeals to me that this name would suite it most. If we go over the sequence of the events we can arrive at the need. This is what happens when we have the Binding Component is built and the same is deployed in the JBI container in a clustered scenario:
-
Deploy the Component on DAS
-
Component gets replicated in all the clustered server instances.
-
Component when installed will call Bootstrap / Lifecycle and
Deploy(ServiceUnitManager) in that order and the Deploy will take
care of the Service Unit's that are referenced by this component.
The component handles the inbound and outbound message requests
using either ExecutorService / or normal threads. -
When ServiceAssemblies gets deployed on the cluster they also get
replicated on all the server instances and the service units
contained there off will register the endpoints whether inbound or
outbound and start doing work. -
While the endpoints get registered there should be a way either for
the failover/recovery or for the stickiness that is required while
handling the request, we need to have unique way of identifying the
endpoint which has previously handled the request. In case of HTTP
LB the LB has a mechanism of logical-physical mapping which will
service the request. -
For this reason we can have the
ClusterName+NodeName+ServerInstanceName as a unique identifier
added for the current/existing endpoint for the BC. For this before
registering the endpoint the BC should check whether its cluster
aware and if yes do the way said above if not register the endpoint
the normal way. -
The JBI runtime has to provide for some hooks from with in the
AppServer, to the GMS module which will pass on the information
regarding the cluster and the health of the individual components
that are deployed on the container. I have discussed these classes
at the end of this document. -
Apart from that the component might need to send out notifications
regarding its own status on the cluster by NotificationEmitter or
some other mechanism that is outlined by the GSM. -
Component can use one of the strategies that are out lined below
for the load balancing and failover in the component specific way.
For example, the SOAP BC will use HTTP load balancer and the JMS BC
will use inbound JMS load balancing. -
From other components perspective, we might require to have a load
balancer for some BCs as in case of JDBC where in we have large
data, which is in bound or out bound HL7 picking up from a DB/file
system and sending out data to the external system. We need to
implement algorithms based on volume and size of the data that is
coming in. We might even have to give the option of selecting the
load balancing algorithm to the user.
Strategies for Binding Components:
1. Using JXTA / JGroups: Both these APIs are good enough for peer
to peer based communication and in fact AppServer has hooks to both
these APIs and uses JXTA as its protocol for GSM. These provide
APIs on the client side for the component to hook to the server.
Both models support virtual synchrony and dynamic finding of the
machines in the cluster etc. For load balancing here we can think
of other strategies based on the request that is coming and the
data that needs to be handled by way of peer to peer communication,
sending the request context from one peer to the other peer if
other has already the requested data.
2. Custom way: we can think of using single database for storing
the request information and client session with respect to each
protocol. The request information includes the key that uniquely
identifies the session for the req. and can carry on the work. This
can be combination of the cluster name, instance name and the
schema + table name in case of JDBC etc. We can think of lead time
here for catering to fail over where in if in the specified time if
the request is not catered to, another instance can set its key
take the request context and can proceed with handling the request.
There is a catch here, if the resource is participating in a
distributed transaction since one TM is already involved in
handling the transaction (and transactions are atomic), we might
need to wait till the Server instance comes up. But here too we can
have a mechanism by way of knowing that the said tuple is under a
transaction and emit a notification for the System Administrator to
take care of the same.
I will be discussing regarding the API support and other implementation details in my next post
Posted at 09:05AM Dec 19, 2006 by Narayana Rallabandi in SOA |
Comments:
