CQL in iep, a fishy example....

The following is a sample of a iep (Intelligent Event Processing) that can be done with the Open ESB iep Service Engine. The source code for this example can be found here. A wink movie about running the sample has also been created. Enjoy!

A Fishy CQL Example...

Imagine a stream, a very long river. For the sake of the discussion, assumes that these are salmons, and they swim in under the left bridge, and out under the right bridge. And you're fishing from the right bridge. If you would ask yourself how many fishes there is in the river between two bridges, you would need to define a window, which is the distance between the two bridges ) as the fishes can swim out and in of these boundaries. It is the same for any system that generates new information with passing time. For this, we will model the distance between the two bridges as between [-60, now].

We now need to figure out the information we can gather/use from the fishes. The more we know, the better are our chances. This will be our schema. It will be composed of the following tuple:

• fish specie (varchar 20)
• fish length (integer) in cm
• age (integer) in months
• weight (double) in grams
• sex (varchar 1) ('M/F')

So you would like to know what is your "prospect" is. How many fishes you have in the river so you can potentially catch. So, what do we have up until now?

• An inbound stream (the river).
• A sliding windows (the section of the river between the two bridges)
• The schema (the multi set)

Now, the only thing to figure out is the output format. In iep, the output format can be a table, a stream (batched or single) or a relation. What we want, is to now how many fishes there is, so for us a table as an output will be ok.

IEP Model, First Version

So, what do we have in this model?

• Our inbound stream, the river.
• BetweenBridged is a time based window. This get the fish swiming from the left bridge to the right bridge.
• The aggregate by sex and specie will give us the count of all (specie,sex) tuples we have in the stream in the last minute.
• The FishCountTable is the table output in which we can connect our sql statement.
• The FishCountNotification is a relation notification, that gives a notification whenever there is a new fish school arriving in the stream. By default, this is being written in the temporary space of the file system. You can change the wsdl if you would prefer another location.

With this, the fisherman will know what are his potential catch. This, gives us the basic on how an iep process works. But how about using some predator notification?

IEP Model, Second Version, Predator Notification

In nature, when prays sees a predator, they either flee or hide (or fight if it is too late). How about adding a predator notification for the fisherman, that while there a shark in the water, there is no fish that is going to bite ?

We can do this by adding a second stream. iep is event based. So we can't monitor for the presence of the shark, this is a state, not an event. But we can monitor for the shark [in,out] event.

So what is new here?

1. We added a direction to the fish tuple.
2. We have a steam filter that will keep only the event when the specie='shark'
3. From this, we keep only the last event, which is the most relevant to us.
4. Then we keep only the sharks that came in, (filter out the direction='out')
5. The result of which we put it in the SharkPresence table.

Now, this could be modified to handle multiple sharks. In which case, you would sum up the number of ins and outs and only send an event when the ins are greater than the outs.

Now what is the Applicability of This?

What we basically done here, is to keep the last of a stream of events, and produce another output with filtering on the data. In real life, this model could be used to :

• Monitor a water treatment plant for example. The "fishes" could be water contaminents, but of acceptable levels. A shark would be an event strong enough to shutdown the pipelines.
• For a production line, the detection of quality parameters that are out of norms.
• etc...

We're just starting to look at using iep at large in applications, and quite honestly, this is very promising. Now the thing to avoid is to start applying this even when it doesn't fit. The hammer and nail anti-pattern....

So what have we seen here?

• iep is a good fit for processing time based events.
• This can be used to filter events, and quickly provide notifications structures
• Interestingly, this has been done with minimal SQL knowledge and WSDL knowledge.

Posted at 03:30PM Feb 21, 2008 by sergeblais in OpenESB  |  Comments[0]