Andi Egloff's Weblog

On the road to the next generation of OpenESB - Fuji Milestone 5

We've been having fun (and been incredibly busy) with Project Fuji, and now we pushed out the download for Milestone 5! (M5 Detail Page)

So what's new since the last round of screencasts?

More Built-in enterprise integration patterns

• Scatter Gather
  
• Routing Slip
  
• Content-Based Routing (m4)
  
• Wire Tap (m4)

Aspects (configurable interceptors) in the context of a messaging layer, including out of the box

• Policy aspect
  
• Logging aspect (m4)

Also a couple of new service types we've started for

• email
  
• REST

As you might know from previous episodes, you can use Fuji with any text editor of choice plus Maven; but we also have plug-ins to enhance your experience with syntax highlighting and code completion

• Much enhanced Web tooling for service composition
  
• First "release" of Eclipse Tooling
  
• Enhanced NetBeans Tooling

To get an impression of what this looks like and what you can do with the web tooling have a look at the 2-part hello world example Kirill created.

Part 1: Building Fuji from scratch, simple "note taking" application example
Note that if you download the Milestone zip you can skip the check-out and build steps.

Part 2: Expanding the application, EIPs, tour of the web composition

We're still working on additional write-ups and showing off each cool feature as we race towards JavaOne, so keep your peepers peeled!

Posted at 11:48AM Apr 20, 2009 by Andreas Egloff in SOA  |  Comments[0]

Fuji Milestone 2 released - Screencast and Downloads

If you've been following the OpenESB community in regards to Project Fuji you know we've been busy adding a lot of powerful capabilities leveraging (amongst others) the benefits of OSGi, JBI and Maven. It's now time to show how these new capabilities can be used together.

To demo milestone 2 we have chosen a more traditional integration scenario (explained in the screencast below).

This should illustrate that Fuji is not just an "RSS piperator", but a very flexible way to link together any kind of system and protocol - and an easy way to define and link together services.

Have a look at the Fuji Milestone 2 Screencast to get an impression of the new capabilities.

Next, download milestone 2 and try it out yourself with the Fuji Milestone 2 step-by-step instructions

Note we couldn't fit all new features into one screencast and demo - so look out for further screencasts, particularly on interceptors and more ESB topology options. If you're wondering what is happening with the web based visual GUI, do not fret: that is coming along nicely too and you'll hear a lot more soon :)

Some of the highlights in this release (from the Milestone 2 page and the readme):

• Enhanced support for Enterprise Integration Patterns
  
  • Split
    
  • Aggregate
    
  • Message Filter
  
  
• New interceptor support which allows cross-cutting concerns such as
  logging, auditing, and security (to name a few) to be addressed in
  a modular fashion. Services can be enriched with additional
  functionality without changing the service implementation.

• Lots and lots of new NetBeans support to make developers more
  productive. Great new editing features for IFL and service
  configuration.

• Introduction of a "proxy" bundle which allows the ESB to extended
  through a distributed message bus.

• A new "Reactive Runtime" feature, where the platform detects
  application changes and automatically refreshes the deployed version
  of an application with updates.

• More service types
  
  • Database
    
  • SMTP
    
  • FTP
    
  • HL7
  
  
• IFL language enhancements
  
  • Nested route definitions
    
  • Support for inline and external configuration
    
  • Added EIP keywords: split, aggregate, filter

You may also want to check out our new and improved Fuji portal page to hopefully satisfy your curiosity with further background info, screen casts, joining the community, how to check out and build yourself etc.

So where do we go from here? The current plan (with the usual caveats) is to shoot for an early access release around JavaOne 2009, with a release in the second half of 2009. Obviously we will continue to publish milestones along the way to get early feedback from the community.

Let us know if you have feedback on Fuji!

Posted at 05:08PM Oct 21, 2008 by Andreas Egloff in SOA  |  Comments[0]

InformIT interview on Project Fuji and OpenESB

InformIT interviewed Keith and I and we talk about Project Fuji and OpenESB, including a whole slew of things such as OSGi, Maven, DSLs, EIP. It was an interesting format with no pre-set topic and a full blown professional production crew shooting this. I think we got into the swing of things after a while, have a look!

Project Fuji and Open ESB - Part 1

Project Fuji and Open ESB - Part 2

They actually shot this interview at JavaOne earlier this year, so it's interesting to see how it is still very relevant to Fuji Milestone 2 (which we're presenting next Wednesday 10/22/2008 in the OpenESB Innovation Series).

Posted at 04:41PM Oct 16, 2008 by Andreas Egloff in SOA  |  Comments[0]

Podcasts on Fuji and Open ESB from the Jazoon GlassFish Day

Alexis recorded the session I presented at the GlassFish day in Zuerich and split it into 2 GlassFish podcasts. Sound quality is a little crackly in the beginning, it gets a little better after that :)

Episode 13 covers Open ESB v2 and what we're doing with the current production version of this platform (which is included in Java CAPS 6 and will form the basis of GlassFish ESB v2 later this year)

Episode 14 gives a quick introduction to the future platform research we're doing with Project Fuji - including OSGi, JBI and domain specific languages (DSL)

And here is the presentation as a pdf: Open ESB v2, Open ESB.next and Project Fuji

Posted at 07:08PM Jul 15, 2008 by Andreas Egloff in SOA  |  Comments[0]

Open ESB joins the GlassFish community!

Open ESB and all its sub-projects are now under the GlassFish community! So what does that mean? It means we are sharing the same light weight governance model and strive to achieve or surpass the same level of transparency and community involvement in those projects. Come participate on our mailing lists or contribute to our components!

GlassFish, thanks for having us :)

Posted at 07:13PM Jun 09, 2008 by Andreas Egloff in SOA  |  Comments[0]

Fuji GUI sneak peek: Fingers too tired to type DSLs?
How about Enterprise Integration in a browser?

The last few weeks we've been showing how easy it can be to compose services and route messages to create integration applications with the domain specific language in Project Fuji, check out the original Fuji screencast if you haven't already.

The natural question that came up regularly after showing the demo was: are you working on any visual tooling on top of the DSL? Well, the answer is a resounding YES; one goal of keeping the artifacts simple to start with is that adding tooling on top is much simpler.

I've been working with Kirill in St. Petersburg to experiment with web based tooling for Fuji. So here is a little sneak peek screencast showing how to build the same scenario as in the Fuji screencast using visual tools in a browser.

Hot off the developer's desk, so no fancy voice overs yet :) but I think it speaks for itself.

The technologies used? It's just CSS layers and canvases, no plug-ins to install. We're still working on integrating this with the OSGi and JBI based Fuji runtime; so stay tuned! Also remember that whilst the demo shows an RSS feed, we have over 30 adapters and containers already that can be leveraged, ranging from legacy systems to modern protocols. Have ideas of how this can be improved? Join Project Fuji!

Does this mean that we're moving away from full blown IDEs? Not at all, for doing the heavy lifting of writing services a fully featured IDE is the tool of choice; for composing services however and maybe simple scripting a low overhead approach looks promising. Actually I would also like to push some boundaries here to see whether JWebPane could bring the two closer together...

Posted at 11:20PM May 29, 2008 by Andreas Egloff in SOA  |  Comments[3]

The power of JBI, OSGi and language oriented programming:
Screencast of Open ESB v3's Project Fuji

Wow! It's been a great week at JavaOne. I'm glad we're getting some great constructive feedback on what we're doing in Open ESB v3 with Project Fuji in making a services and integration platform much more productive and approachable.

Project Fuji is the new core integration stack at the heart of Open ESB v3 and the technlogy preview shows goodies such as a domain specific language (DSL) for integration, a JBI enabled OSGi runtime and simple yet powerful Maven enabled tooling.

We've been demoing the technology preview in our session, demo pods and to anyone who wanted to see it - and I have yet to run into someone who wasn't impressed with how quick and easy it is. Better yet, some excited souls are starting to see and think about what else could make it even more powerful.

You can see the demo for yourself in the screencast Keith recorded during JavaOne - he's using his best announcer voice :)

In a few minutes it shows how to poll from an RSS feed, run it through a JRuby filter, then send results to an instant messaging client as well as to a file.

Personally I find it very satisfying and empowering to be able to realize a scenario like this in minutes in a straight-forward fashion that is close to the way I think about the problem - rather than hours or days and potentially having to figure out how to make the tooling and technologies fit such an integration challenge.

Posted at 08:59PM May 11, 2008 by Andreas Egloff in SOA  |  Comments[6]

JavaOne 2008 - A Crazy Year for Java CAPS and Open ESB!

JavaOne 2008 is nearly upon and we've been very busy bees over the last year!

On top of the job as lead architect for Open ESB I also took on the job of lead architect for Sun Java CAPS 6 (composite application platform suite) which is slated for release this quarter.

This is Sun’s first commercial suite that includes the JBI based Open ESB runtime which brings the full power of this open source community and its partner ecosystem to our commercial offering.

The new service layer Open ESB adds to the suite is fully integrated with the rest of the suite, so it can fully leverage the new powerful annotated JSR 181 pojo model and a wide array of JCA resource adapters (RAs). It truly manages to deliver on an open, standards based platform that can take advantage of the rest of the Sun stack and its powerful features; it uses a standard Netbeans 6 and GlassFish v2 as the IDE and the integration server runtime respectively for example. All that innovation and we managed to make it evolutionary as well! Existing Java CAPS application will continue to run on the new platform.

So what else do we have in store? More Open JBI components are in the pipeline to be released on top of the upcoming release 6.

We’re also looking ahead much further and we’ll be showing off a technology preview of the core stack for Open ESB v3, called Project Fuji.

Stay tuned for a lot more content here and on the Fuji site by JavaOne and swing by our booths in the pavilion. For those interested in what we’re cooking up with Project Fuji I will be giving a technical session with Keith on Wednesday: See you there!

TS-6385 Integration Profile for GlassFish Project v3
Andreas Egloff, Keith Babo
Wednesday May 07, 09:30-10:30
Moscone Center, Esplanade 304/306

Posted at 11:57PM May 03, 2008 by Andreas Egloff in SOA  |  Comments[3]

Inside JBI part I: Clarifying Lifecycles

In this series I would like to give some insights into gotchas and topics that are not immediately obvious when reading the JBI 1.0 (JSR 208) specification. I'm very lucky that I can bug the spec leads Peter and Ron - and architects and leads like Keith Babo and Mark White who work on the RI and project OpenESB when I want to get some background information on JBI. I'd like to pass on some of this information that I have acquired in the last couple of years.

When stop isn't equals stop

In JBI the components and the service unit (SU) deployments both have their own lifecycle - and in just skimming the spec one might easily presume that stop in both has the same implications. It turns out, the service unit stop might be better described as "stop consuming" - that is right, after a SU stop it can still be provisioning services, it just isn't initiating new requests.

This is different than the component "stop", which means that the whole component should stop provisioning AND consuming services.

Here is another item to be aware of: the specification does not perscribe the interaction between the component and the SU lifecycles. The RI (and project OpenESB) does call the equivalently named lifecyle operation on all SUs when the component lifecycle operations are called (e.g. stop on all SUs when the component stop is called), but that is not explicit in the specification.

When implementing the component stop, you may come accross another interesting feature of the delivery channel: there is no way to re-start message delivery once it is closed, and we only get a new delivery channel in the component initialization.

This means you can not use the delivery channel close to implement the component stop: if the user does a stop and subsequent start you could not re-start delivery on this delivery channel, the user would have to completely shutdown the component first. This makes for an interresting time in implementing the component stop; none of these options are optimal and includes approches such as interrupting the deliverychannel or using the accept on the delivery channel with a regular time-out.

With this in mind, here is my wish-list for enhancements I would like to see in future specs:

• SU stop may benefit from a different name that makes it clear that only service consumption is stopped


• That the specification would explicitly declare the relationship between the component and SU lifecycle


• That the delivery channel would have a way to stop and re-start message delivery - separate from the ultimate shutdown

Tags: jbi esb openesb soa

Posted at 11:18PM Jan 04, 2007 by Andreas Egloff in SOA  |  Comments[2]

Non-blocking NIO sockets in the JBI HTTP Binding Component (BC)

I received some questions about the HTTP BC (part of the Java EE SDK Tools Bundle and the project OpenESB) - and what the advantages are of it supporting the processing of request-response and one-way web service invocations without blocking threads.

So I will try to give a little background on the JBI architecture, illustrate what would happen if we did block threads - and last but not least show the advantages of supporting asychronous processing.

The JBI architecture is based on an asynchronous message bus that can be thought of as similar to in-memory JMS queues. The messages are XML based – and the processing of the request and sub-sequent response are completely asynchronous – they will be handled by different threads and there is no need to block a thread anywhere in between.

This allows for an extremely scalable system that is capable of handling a huge number of concurrent requests with a minimum of resources – in an extreme case where it might be tuned for throughput rather than response times for example it could be handling 10’000s of concurrent requests with under 10 threads - environment permitting of course.

Given this architecture, it would not fit well to use an HTTP connector that blocks threads - and doesn’t have an asynchronous API. Some of the implications this would have in the HTTP BC are:

Implications of blocking

Threading

* Threat of resource starvation and dead-lock (if there are recursive calls within services they can easily exhaust the thread pool – and stop the response from getting processed which might let others progress)

Scalability

* The large number of threads needed to scale at all will consume a lot of related memory which will limit the number of threads feasible
* Because the number of requests we can handle is limited by the number of threads we can efficiently use, we are limiting scalability a lot
* Even without taking efficiency and memory into account, the JVM will ultimately limit the number of threads possible at all - and hence concurrent requests

Performance

* More threads means more context switching, hurting performance
* Limiting the number of concurrent requests we can handle means we can’t always fully utilize available CPU or network bandwidth, i.e. we might be “waiting” for a response so we can free a thread and proceed with the next request

Fortunately this does not have to be. The HTTP BC uses the Grizzly NIO framework including the asynchronous request processing (ARP) extensions that is part of the Glassfish application server.

Not blocking threads makes for an excellent match with the JBI architecture and has these implications (Note that the HTTP BC in the appserver 9.0 currently only uses Grizzly on the server side):

Benefits of not blocking

Threading

* Because nothing blocks there is no threat of resource starvation or dead-lock
* As an example 3 threads in the BC could handle any number of concurrent requests (for higher response times we can choose to add more), e.g.
  thread 1 – Handle the incoming traffic (asynchronous Provider), normalize the message, send it to the JBI “queue” (normalized message router)
  thread 2 – Handle messages off the JBI “queue” (normalized message router), send requests via the asynchronous client API
  thread 3 – Handle responses (asynchronous client API listener), send them to the JBI “queue” (normalized message router)

Scalability

* Very limited number of threads, minimizes the memory footprint
* Because the number of concurrent requests we can process is not limited by the number of threads we can realistically use, we can scale to much larger number of concurrent requests

Performance

* Less context switching which might hurt performance
* Never “waiting” for responses to proceed means we can fully utilize available CPU and/or network bandwidth

Scalability and performance are some of the primary selling points for an integration platform - it makes sense for us to make use of the full potential of JBI.

Posted at 06:40PM Nov 28, 2006 by Andreas Egloff in SOA  |  Comments[0]