iep for Message Ordering...

It all Started with a Simple Email...

A colleague of mine asked me if I ever used the gapwindow or sequencing capabilities of our Intelligent Event Processor (iep). Well, I had to admit that no, but I jumped right on the occasion to tinker with something I have never used before.... He asked me because he needed it for the next day. Well, one day later, I'm one beer richer, ready to share my experience with iep...

Gap Window Operator

The Gap Window Operator takes what it finds to be missing from a stream based on the sequencing index and a grouping key (what it calls the partition). So, for that group, it will look at events in the stream that are missing sequenced events. For example, if S=sequencer, K=key (name, country), so we have Ev(Sequence,Name,Country)

Ev₁ = (2,"Serge","Canada")

Ev₂ = (1, "Jason","Norway")

The Gap Window Operator will see that Ev(1,"Serge","Canada") is missing, but that K("Jason", "Norway") doesn't miss any events. When Ev_X(1,"Serge","Canada) will be received, the output of the GapWindow Operator will be empty.

Sequencing Operator

The sequencing operator works on the events with the same structure. Hence, it will use a sequencing index and a grouping key (what it calls the partition) just the same. So for the same example, before Ev_X(1,"Serge","Canada) arrive, the output of this operator will only be Ev₂(1, "Jason","Norway"). Once the missing event arrives, the output will be: [ (1, "Jason","Norway"),(1,"Serge","Canada),(2,"Serge","Canada)]

Just Wink it...

If you would like, you can find a small movie showing the structure of what I did, including a sample of how it works (READ THE ADDENDUM!). You can also get the source code if you would like to play with it ourself. The zip file has the iep module, the composite application and the SOAP UI project file to test it.

Addendum

In the code sample, the output of the sequencing order is placed into an Attribute Based Window operator. This attribute base window serves as a grouping method that keeps a range of events, based on that value - the size as set in the customizer. After some discussions, we found that this operator is useless the use case we have for this sample. And the simpler model (see below) actually is better. So, if you're only looking at the movie and not reading this, you're missing this bit of information, sorry!

Posted at 10:30AM Feb 28, 2008 by sergeblais in OpenESB  |  Comments[0]