Bernard Wheeler's Weblog

The various definitions of SOA and the discussions of the various benefits are not always tremendously helpful or illuminating when it comes to answer the question of “so what do I need to put in place to achieve one”. So that's what I've decided to start to cover over the coming days.

The usual disclaimer applies: this is my set of opinions, and there will be other opinions expressed elsewhere. However, they are based on 20+ years in the software industry, both on the supplier and consumer side, with multi-tier, distributed systems being a focus for just over 10 of those years (and counting).

There's a graphic that has becoming almost a de facto standard in SOA presentations. It shows a four-layer model, those four layers being (starting at the bottom) Data Layer, Service Layer, Process Layer and Presentation Layer, something like this:

Now having a layered model is nothing startling and it's been with us for years: 'monolithic', host-based computing fitted a single layer; client/server extended that into two layers, largely in conjunction with the advent of a 'standardised' way to access data – SQL; then there followed a wider push to three layers – data, business logic and presentation, driven originally by technologies such as DCE, CORBA, DCOM and the maturing of development disciplines around OO, patterns and the like.

So SOA introduces a couple of distinctions:

1. The “business logic” layer is further categorised into 'processes' (the sequencing and orchestration of actions to achieve something of measurable value) and 'services' (the encapsulation of business functionality into components that can be accessed in a consistent way). This separation of processes and services leads to the second distinction.

2. The layers should not stop at application boundaries: any and all applications ('processes') that wish to access a specific set of functionality should do so by using the same, shareable service

Examining the impact of the second distinction shows that the translation of functionality into services must also include aspects of security and self-protection, since the service cannot determine ahead of time who is attempting an access, and whether they are attempting to do so in a well-behaved manner. This is a point that I'll return to some time later.

Whilst I consider this four-layer model a reasonable start, I think it is still too simplistic and can actually encourage problems later.

To illustrate this, let's take the Data Layer. What goes into it? Is it just the actual corporate data itself? Is it an object Domain Model, so that the there's some functionality encapsulated in there? Or is it the same content that was in the three-layer model, such as object-relational mapping, Java Entity beans and their alternatives, etc.? And how are those those things accessed? Where do you draw the boundary between the Data Layer and the Service layer that's the next one up? And if you're using some external functionality, such as a credit rating service, does that fit in the Data Layer (because you're retrieving credit rating data), or in the Service Layer (because it's a “service”).

Therefore, when speaking with people about SOA I introduce this picture, but then I change some of the names (you'll see why) and I supplement it with two 'vertical' elements that interact with each of those four 'horizontal' layers, those two elements being:

1. A common, system- and platform-independent model for representing data that is transferred between and within the layers. By using this approach, each different source and sink of data, functionality and events that is added to the environment need only have one set of data translation functionality developed, i.e. from its 'native' form into the common model. This eliminates any inter-system dependencies based on data formats.

2. A common set of 'framework' services that will underpin the operation of the functionality in all of the layers, those framework services to include such things as auditing, exception/violation reporting, the collection of performance data and other operational metrics, the management of operational events and configuration parameters, etc.

So my picture looks like this:

So what I intend to do, is go through each of these six pieces in turn to outline what I consider to be some of the defining characteristics of each and why I consider them essential to the formation of a successful implementation of a service-oriented architecture.

But as a teaser, I shall explain that my Technical Platform layer is the layer that wraps all systems where there is an element of integration work to be performed, such as data transformation, connection into external and/or legacy systems, etc. So the access to a database and the access to a credit rating service would both go in the Technical Platform layer.

As it happens, SeeBeyond has a similar view of the SOA world, so I'm not alone here.

There's more to come – soon, I promise!

A picture of some daffodils from the garden:

Last year I bought my third Sony MiniDisk player, a MZ-NH700. A third? Well the first was a straight MD-LP (4 CDs on a single disk) Walkman, which led me to get a non-portable one to attach to my Bose setup at home (my wife didn't want “big black speakers messing up the decor” and since the Bose has small, white, ceiling-mountable speakers, it was an easy decision...).

The third came about because although my first MD Walkman had a PC connection, transfers were done in real-time. So putting 5 hours of music onto a disk would take 5 hours. Which was highly frustrating, because various bits of software (Real, the Sony SonicStage stuff, etc.) could rip CDs onto my PC a lot faster. So the PC connection wasn't really that useful in the end (since I haven't ventured into the iTunes^TM world yet and only listen to stuff I have on CD or vinyl).

The new MD is a lot smarter. For a start, it can cope with Sony's 1GB disks and secondly it actually has a useful USB connection and transfers are basically like putting stuff onto a flash drive (and the PC sees the 1GB disk as an additional drive if you want to transfer data/pictures and not just use it as a music device). And since the 1GB disks are now available through Amazon at 5 pounds a time, it's not an expensive option either.

Sony claim that you can put 45 CDs onto a single disk. Well, to test this out – and what the music quality is like at that level of compression, I've actually put 48 CDs and 2 CD singles on a disk. Admittedly, the disk lengths range from 33 minutes (Getz/Gilberto) to just under 1 hour 15 minutes (Bach: The Well-Tempered Clavier, Book II - 24 Preludes And Fugues, Disc 2), but that's still an impressive amount of music to fit on 5 pounds-worth of small, pretty rugged, portable media.

And the music quality? I listen using Sony's MDR-EX71 headphones and on the compression/sample rate to get that amount of music on the disk (48kbps ATRAC3plus) it's quite frankly astounding – bearing in mind that I'm getting towards the age when my hearing's not what it was and I'm not pretending to have 'professional reviewer quality audio' abilities, but I defy anyone to have a serious complaint about the sound that comes out of this portable device.

Another thing that really appeals – and steered me away from the iPod/MP3 player route is the battery life (60+ hours) and the fact that the player I (deliberately) chose takes standard 'AA' size batteries. So I don't have to cart around a charging cradle, power adapter, etc. when I'm travelling – I just take one spare battery and know that wherever I am in the world I'll be able to get some more in the nearest drugstore or supermarket.

So if you're in the “iPod or another approach” situation, I can highly recommend the Sony NetMD technology - even at the highest compression/lowest sample rates.

So what have I put on that disk (and I have others to fill too...)? Well I shan't list the full 50, but here's a selection of titles I can highly recommend:

• apart from the Getz/Gilberto, there's also Jazz Samba and Jazz Samba Encore, both Getz collaborations in the same vein

• Miles Davis' absolutely classic Kind of Blue and Birth of the Cool

• both of the books (4 CDs) of Bach's ground-breaking Well Tempered Clavier – originally written to prove that the newly invented piano could handle tunes in any key – its main innovation at the time over the tuning of keyboard instruments used until that point (and the first piece in the first book is possibly the most beautiful piece of music ever written). The recordings I have are performed by Jeno Jando

• Ludovico Einaudi's Le Onde – very soothing

• Lemonjelly's lemonjelly.ky and Lost Horizons – both obviously recent favourites in the BBC's factual programming section, as they feature repeatedly as backing music

• Massive Attack's 100^th Window – sadly overlooked in awards, but nevertheless superb

• the 'usual' fare of Keane's Hopes and Tears, and Coldplay's Parachutes and A Rush of Blood to the Head – 'usual' not meaning ordinary

• Zero7's When It Falls, and also their compilation for the AnotherLateNight series (there's also Fila Brazillia's similar offering on the disk)

• Radiohead's The Bends – I think their best work

• Muse's Absolution – just wonderful

That's it for now, although I may return to music topics at a later date, just to report on what I'm listening to as the days pass.

I was in an interesting discussion with Warren Buckley, CTO from PolarLake yesterday. He was going though an update on their technology and positioning with me and a few colleagues in one of Sun's London offices. What he said prompted a number of trains of thought about SOA implementations and some potential pitfalls that are so glaringly obvious as to be easily overlooked.

One of the repeating themes in discussions about 'effective SOA' is the matter of granularity of services: make it too fine-grained and you (or rather, the business) will have a hard time threading all the many bits together in the right sequence without forgetting to add in all the key steps; make it to coarse-grained and you'll find yourself creating a new set of services for each new application variation, because the services you have are too specific to enable re-use.

But we touched on something that I don't believe has had a sufficient airtime to date.

It is generally agreed among those that participate in such conversations that the services exposed to orchestration – and therefore the main 'components' used to build up composite applications are themselves going to be composite pieces of functionality. All well and good, but what paradigm do you use for the construction of those composite services? If the answer is that you want to employ re-usable services for that too, then there's a question that needs to be asked: what mechanism are you going to use to enforce the orchestration piece not to be able to use your lower-level services and only use the composite ones that you originally intended?

I'll use an example loosely based on a real situation: a finance organisation has built up to its current size by both organic growth and a number of acquisitions. As a result, some of its business units still effectively act as autonomous units IT-wise (and financial legislation dictates that to some extent this must remain so), but the institution as a whole wants to enable its customers to see and manage their overall portfolio through a single view. To handle this situation, the institution has rolled out the concept of service 'domains', where functionality available from a business unit is formally described at its boundaries and the internal workings of how that is achieved is at the discretion of each business unit. This matches the profile of a SOA approach, so it's so far, so good. Now the Home Loan group publishes a service at this boundary point for “get loan balance”, but they also have an “internal” service (say, “get loan balance from mainframe”) that actually goes to the specific system that holds the home loan data - in the “Data” layer, rather than the “Service” layer of the typical 4-layer SOA model. A developer has been transferred out of the Home Loan group into the main corporate group and thinks he'll use the service that minimises “service-hops” and goes straight for the data-layer service. Now that may not be what's intended, but after all, it's published as a service, and who's going to spot it?

Now in the OO world (to which SOA is often likened), there are distinctions of public, protected and private. And there are things called packages that may be equated onto the service 'domain' concept that the finance institution has implemented. But at the moment, equating is all that we can do. There is currently no mechanism to enforce this model. (There are many, many other qualifiers that can be used in OO and may have a useful equivalent in SOA, but that's not the topic of today's ramblings.)

I wondered (during the aforementioned discussion) whether this is something that will eventually work its way into the policy-related specs and technologies, but these still seem to be a little way off in terms of widespread consistent adoption and implementation. And on reflection, a better way to implement this kind of control is to (again) follow the lead and place the qualifier in the declaration – which in the case of a SOA implementation based on web services would mean putting it in the WSDL for each web service. But in my (admittedly far from encyclopedic) knowledge of WSDL, I don't recall equivalents of private, public and protected. And it's not a run-time permissions thing (so it's not WS-Security, SAML, or the like), it's a define/design-time 'exposure' thing...

So since WSDL doesn't help here, a way to achieve this (in my opinion) necessary enforcement, would be to start segmenting your service 'domains' so that they each had a separate registry with different look-up coverage. But that introduces the likelihood of data duplication of service definitions in more than one look-up registry, something that would be really nice to avoid. But at the moment this is likely to be the only realistic way of managing external access to services by third-party organisations (although PolarLake discussed a case where they have essentially implemented a reverse proxy for service access which would also be effective, and possibly more heavyweight).

So SOA – even a highly standards-based implementation – has the capability for creating just as much of a 'service spaghetti' as any other kind of software construction – and doesn't currently have several of the mechanisms found elsewhere to try to encourage and enforce good discipline. Exposing data-level, platform-specific functionality as services may follow the intent of SOA to the letter and will probably allow greater flexibility, isolation and 'agility' in being able to upgrade, replace or consolidate back-end systems without affecting your shiny new composite apps. But there needs to be a mechanism or procedure that checks and enforces that these lower-level services are not being called inappropriately, causing unforeseen 'hard wired' cross-links to appear where you thought they couldn't. But there isn't (to my knowledge) any such kind of automatic mechanism or procedure.

Which means that there's likely to be some very expensive almighty mess-ups made if the right level of discipline and forethought isn't consistently applied by the design and implementation teams. But it was always thus, and as has been said about SOA before: “the rules of physics still apply”.