Derek's Blog: Water Hazzard

It's been a while since I last posted anything...I've been busy climbing Fuji (metaphorically, I physically climbed Fuji-san many years ago) the project name for OpenESB v3. Andi Egloff presented a demo in the Java One Technical General Session that displayed some of the features that we have been working for a little over the balance of this calendar year.

I worked in a bunch of areas:

• EIP infrastructure
• Change EIP execution from using interceptors to being light-weight components
• Allow EIP based routes to be treated as a normal JBI services
• Specific EIP instance implementation
• Tee - as basic as they get
• Scatter/Gather - much higher complexity level. Reused the Broadcast and Aggregate EIP constructs to get the desired affect.
• Content Based Routing - the first steps. Allows for regex, XPath and NormalizedMessage headers to be queried and selected. Allow the rule set to be dynamically changed by storing the rules is in a file. 
• Tracing/Monitoring infrastructure
• Instrumented the NormalizedMessageRouter to signal tracing events for the 4 basic primitives: before send, after send, before accept and after accept. Implemented a simple text based logger that has been used for simple debugging.
• Kirill Sorokin & Alexander Permyakov are using this to enhance the WebUI with message tracing/replay mechanism

We are still on a tight schedule to round out the feature set and complete backward compatibility with OpenESB v2.  Watch our progress here...

 

Fuji, the OpenESB V3 development playground, has hit its Milestone 2 release.

For a quick preview watch the screencast. For more a more in depth look, check out the source code and the Milestone 2 demo. 

Besides a few fixes and some design collaboration on interceptors my main contribution to this release is the Proxy bundle.

The Proxy bundle allows a component in one Fuji instance on one machine to communicate with another component in another Fuji instance on a different machine. The Normalized Message Router (NMR) only deals with a single JVM instance. Usually SOAP/HTTP is used to span machines. The Proxy bundle is a JBI component that extends the NMR bus across JVM's using a light weight protocol.

Shoal/GMS is the basis for the first transport implementation for the Proxy bundle. It provides group awareness and messaging.

More detailed information about the Proxy bundle can be found here.

Also, checkout a presentation I gave at recent OpenEsb Innovation Series

As you might have seen elsewhere, Glassfish ESB Milestone 2 is available for download.

Many features have been added and many fixes included.  Most important is to give it a spin and provide feedback, so that we can get any fixes/improvements in as we wind down towards the official release.

As an example, a recent fix/improvement that I participated in, which will make it into Milestone 3, enables End Point References (EPR) for callbacks in HTTP-BC.  This is a fix because it allows the HTTP-BC EPR usage to function without having to make a bunch of component changes (that would not have made Milestone 3) and an improvement because it gives component developers another choice in how to deal with EPRs.

The description of the problem and the solution can be found here.

GlassFish ESB: Open ESB with support!

The first glimmer of GlassFish ESB arrived this week. GlassFish ESB is positioned between the base OpenESB open source project and the various supported projects derived from OpenESB, like ESBSuite, MDM and CAPS. It contains a basic complement of components, and most importantly it includes support. This targets a common request and it allows a large percentage of applications to be built without resorting to a higher level product to get that all important support.

The announcement, in the form of a nice new web site, allows one to download the free first milestone build. Two future milestone builds are in the works with the final release product being available the first week of December. Download today, and give it a try. It could be a better solution to your problem.

In the near future, additional components will be released as add ons to GlassFish ESB. IEP will be the first out of the starting gate.

One of the more frequent questions surrounding JBI has been it's lack of a distributed solution. The current OpenESB kits extend this with support for operating in homogenous clustered Glassfish instances. My recent resurrection of the Proxy Binding component from hiatus will allow for explorations in heterogenous and transparent distribution.

Background

During the early work on JBI and development of OpenESB, the Proxy Binding idea was kindled as a potential solution for linking JBI instances. A prototype was designed to test out the feasibility of the idea. The prototype was also used to determine if any proposed JBI features would cause any issues with this architecture. For a while, it tracked the on going changes in the JBI specification.  The desire to support homogenous clustered Glassfish instances was considered a higher priority, and the Proxy Binding was place on hiatus.

Now that OpenESB exists and is the basis for JavaCaps, the desire to explore the distributed arena has returned. Father time, the JSR 208 specification designers, technological change has been kind to the Proxy Binding. The code still compiled, and almost ran, after almost 3 years. The features added to the specification leading up to its adoption will only require a small amount of work. The open sourcing of Shoal-GMS allowed for the replacement of a specialized, but raw,  home-grown group management solution with a generic robust and supported solution.

A Little Spring Cleaning

Once the parent Maven pom.xml was updated to include the Proxy Binding in the Open-ESB builds we were successfully back in the building business.

No jregress test suite existed for the Proxy Binding, but it did have a manual test suite. A quick run of the manual test suite immediately found some issues. Luckily it only took a day or two day to remember how things were put together and get ourselves limping again.

As advertised, using Shoal-GMS is very straight forward. The Shoal API is both simple and powerful and it worked out of box without any fiddling (even on a clustered GF instances which also uses Shoal.) When we need to get this running with JBI instances spanning a sub-net I'm sure it will get a little more complicated. It helped that it was replacing a similar type of technology, but I was still impressed.

A few features worked their way into the JSR. Service connections and interface connections ended up affecting the Proxy Binding. Service connection support was added as it was going to be needed for a Jregress suite.

The NMR portion of JBI has a feature/performance test suite. This is written as 2 communicating components, a source and a sink. This suite randomly generates ME's and randomly exercises all of the internal states an ME can take. By placing each test component in a different JBI instance the Proxy Binding components on each side can be exercised. Of course this did not run the first time. Surprisingly, given that the previous testing was spotty at best, only two issues needed fixing before this test suite ran correctly.

Test Drive

The Proxy Binding isn't currently (I'm working on that) packaged in the nightly builds. So if you want to try it out you will have to download and build the source.

The Proxy Binding component can be installed like any other component. No configuration is required (thats both good and bad.) One caveat, only endpoints created after the Proxy Binding starts will be  published to the confederated endpoint name space maintained by the collection of the Proxy Binding instances (I'll be fixing this shortly.)

There is a small amount of default logging. Shoal-GMS events, and Proxy Binding events are logged by default. This means a few lines when a Proxy Binding starts and a few lines when an endpoint is created somewhere. Also, when the Proxy Binding shuts down, it dumps a summary report like the following:

---

[#|2008-08-04T12:12:19.932-0600|INFO|sun-appserver9.1|SunProxyBinding.base|_Thre
adID=14;_ThreadName=httpWorkerThread-4950-0;|ProxyBinding state at 2008-08-04T12
:12:19.932-0600
  Event Processor
    Events         Sent(7)  Received(2)
    Instances Count(0)
    Wait Instances Count(0)
  Exchanges          Sent(0)  Received(1816)
  Messages           Sent(1822)  Received(3183)
  Bytes              Sent(130835)  Received(407701)
  Result             DONE(1594)  ERROR(222)  Faulted(92)
  Internal Exchanges Sent(6)  Received(6)
  Active Exchanges   Count(0)
  Remote Services Count(0)
  Local Services Count(2)
        InstanceId    (ESBTest.server)
          Service  (NMR-Test-Service2)
          Endpoint (NMR-Engine-Endpoint2)
        InstanceId    (ESBTest.server)
          Service  (NMR-Test-Service)
          Endpoint (NMR-Engine-Endpoint)
  Local Service Connections Count(0)
  Remote Service Connections Count(0)
  Active Statistics:
    Service (NMR-Test-Service2) Count(1255)
      Operation (operation1) Count(1255)
        BytesSent        M(68) X(141) T(90483.0) V(72.09)
        BytesReceived    M(173) X(250) T(282632.0) V(225.2)
        Duration-ms      M(0) X(170) V(9.83)) S(8.26)
      Total (1255)
      BytesSent        M(68) X(141) T(90483.0) V(72.09)
      BytesReceived    M(173) X(250) T(282632.0) V(225.2)
      Duration-ms      M(0) X(170) V(9.83)) S(8.26)
    Service (NMR-Test-Service) Count(561)
      Operation (STOP) Count(1)
        BytesSent        M(68) X(68) T(68.0) V(68.0)
        BytesReceived    M(166) X(166) T(166.0) V(166.0)
        Duration-ms      M(0) X(0) V(0.0)) S(0.0)
      Operation (operation1) Count(560)
        BytesSent        M(68) X(141) T(40284.0) V(71.93)
        BytesReceived    M(171) X(248) T(124903.0) V(223.04)
        Duration-ms      M(0) X(110) V(9.63)) S(8.53)
      Total (561)
      BytesSent        M(68) X(141) T(40352.0) V(71.92)
      BytesReceived    M(166) X(248) T(125069.0) V(222.93)
      Duration-ms      M(0) X(110) V(9.61)) S(8.53)

     Total (561)

      BytesSent        M(68) X(141) T(40352.0) V(71.92)
      BytesReceived    M(166) X(248) T(125069.0) V(222.93)
      Duration-ms      M(0) X(110) V(9.61)) S(8.53)
    Total (1816)
    BytesSent        M(68) X(141) T(130835.0) V(72.04)
    BytesReceived    M(166) X(250) T(407701.0) V(224.5)
    Duration-ms      M(0) X(170) V(9.76)) S(8.35)
|#]^M

---

It displays some performance counters and by default a breakdown of interactions by Service and by Operation. By default the number of bytes sent and received and the real time duration are reported for each Service/Operation combination, and totals are given by Operation and by Service. The cryptic 1-letter value designations decoder: M - maximum, X - minimum, T - total, V - average, S - standard deviation.

 To be cont'd...

Coming up with names for things, a title for by blog in this case, is not one of my stronger skills. I'm not sure how "Water Hazard" came to mind, but it quickly seemed to fit the bill.

1. I work on OpenESB which swims in the Aquarium.
2. I like to play golf. This is one of my local water hazard's.
   
3. I recently learned to surf. Plenty of water hazzards. The most famous being "Men in gray suits".