« Strengthen and Prote... | Main | LinkedIn as a leadin... »
20060622 Thursday June 22, 2006

Extension and Versioning for Service Oriented Architectures

As I mentioned in a previous post here, my team has been struggling with some issues around extending and versioning of services. The problem manifests itself in having very incomplete and short sighted requirements for one application that in no way meet the requirements of the next application and then trying to force a common service that both applications use as a blackbox. There are a few problems that, in my opinion, are the root cause of this struggle: the lack of a versioning strategy for WSDL's and document schemas; our inability to understand how our clients are utilizing our services; and the technology used by our clients to integrate with our services which causes them to be tightly coupled to the service.

So we have no versioning strategy! What this has caused is that changes we make to services are next to impossible to get to be backwards compatible, we're no where near thinking about forwards compatibility..... An old W3C draft document talks in depth about how to create versionable xml documents. Some of the most useful pieces of this article discuss the need to begin to implement Must-Understand and Must-Ignore constraints in the XML documents and to design these documents in a way so that they have clear points of extension which will not break clients when new elements are added at the extension points. Also discussed, is the need to version documents at different level, sometimes even down to the element level. All of that is a lot to bite off at one time, so to begin with I believe that introduing Must-Understand and Must-Ignore constraints into the documents will make a huge step forward. Must Understand is essentially a Big Bang approach, the client is forced to understand new elements, if they don't understand an element, abort and fail. This would be used in situations when a new required element is being used. Must Ignore differs in that if the client doesn't understand a new element then they are allowed to ignore it without failing. This would be used in cases where new elements are optional, which is what we should attempt to default all new elements to. This goes the other way as well, if we are removing an element, then instead of completely removing it we can begin by making it optional with a Must-Ignore constraint. This would be similar to the beginning of deprecating a Java element. Eventually, as clients move to newer versions (and we understand what they are using), we are able to completely remove the unused element.

What makes our lack of a versioning strategy so painful is that we have no idea how clients are actually using the services. What elements, port types, attributes, etc are really being used, and which are essentially sitting idle? We do not know this answer. This is really the core of what I blogged about recently here. Basically we need to move to a Consumer Driven Contract model which essentially just means that we will document the direct use of the services to the element, atteibute and port type level. This simply gives us insight into how clients are utilizing our services.

Even if we intimately understand how clients are integrating with our services and have the world's best versioning strategy in place and follow it to the letter, our hands will still be tied due to the way that clients are integrating with web services. The typical (anti-)pattern I see for web service integration using Java, is to download the Web Services Developer Pack, then start by getting the WSDL and running wscompile on that to generate the client stubs. What this essentially does is pretty tightly couple the client to the particular WSDL. There are few changes that we are now able to make that will not require that client to re-run wscompile and re-generate the entire set of client stubs and likely the entire set of transfer objects that were created. Even if we modify the WSDL and the schemas in a compatible way it is highly likely that it will force a recompile for the clients. This violates the whole tenant of SOA which strives to be a loosely coupled grouping of services which any client can integrate with and the service can be modified in a compatible way without the knowledge of the clients. So how do we fix this? We have to begin moving client integrations to a less coupled integration mechanism. Tools like Schematron can help to decouple clients from services by looking only for the specific elements and attributes that the client needs in a document as opposed to consuming the entire document even though only part may be needed. That allows the service to change more fluidly without worry of breaking all clients.

So, to wrap this up, ya we have some issues that we're dealing with, but there are solutions to solve them. We just need to take a methodical approach to this, first putting a basic verisoning strategy in place, then understanding how current and new clients are utilizing our service, then we can address how those clients are integrating with us so that they are not so tightly coupled to our services making themselves brittle and unable to handle even simple modifications. Once we get these seemingly simple things in place, we can begin to move forward with truly reusable services that are able to evolve over time to better meet the needs of more consumers.

Links referenced in this article or where info was used from:

Posted by mikedav ( Jun 22 2006, 04:16:20 PM PDT ) [Listen] Permalink

Comments:

Post a Comment:

Comments are closed for this entry.