sapienti sat: Making SOA Ready for the Enterprise

Why policies do not scale

Most of the world seems to be governing by policy. And I can see why – it is simple, intuitive and easy to implement. However it has several serious limitations, which can be quite fatal for enterprise-grade SOA implementations once they go post-pilot: governing by policies does not scale and leads to nasty proliferation problems. By the former I do not mean the scalability in traditional IT sense: the ability to handle increased processing loads by throwing more hardware, but the structural scalability of a solution to match the scale of a problem. Let me give an example: enforcing two hour parking limit. On a free parking at the center a small historic town the traditional chalk mark on front left tire (every hour attendant makes a chalk mark at the same place on the same tire of each car, and if it already has two marks a ticket is issued or the towing company is called in) works just fine. But if you try to scale it to a huge multi-lot parking of a major attraction it breaks down for a number of reasons: it can take the attendant more then an hour to mark all cars; even if multiple attendants are hired they will spend all of their time marking, so it is hard to attribute marks to specific timestamps; question of the ownership of the marks and territories will arise, etc. So the approach will not work (result in cars being punished for staying less then the allowed limit and/or not being punished for exceeding it) or will require so many complications, additional protocols, conventions, algorithms and resolution mechanisms that making it work (and proving that it does) will amount to a Pyrrhic victory.

Now let us get back to SOA Governance and policies, but before we get to the point I’d like to make one last detour and talk about Enterprise Services. Since the word service has been used in so many contexts with so many different meanings and connotations, I need a more specific term to talk about governance. And in my typical manner I will use qualifiers: Enterprise Service is such a service that its name is familiar to the CEO and (s)he cares about what it does. According to this definition invoceCustormer and processOrder services fall into that category while getRecorsXfromCRMsystemY and validateSAMLToken do not. I would also stipulate that SOA Governance as a business-level activity applies primarily (if not exclusively) to Enterprise Services.

There seems to be a consensus amongst the SOA analysts and practitioners that a Fortune XXXX company if they go “100% SOA” will end up with one-to-two thousand Enterprise Services. It is also reasonable to assume that such a company will have couple of dozens of non-functional characteristics of services that the business cares about resulting in as many policy types. A simple multiplication yields up to 50 thousand policy-to-service applications, raising questions how someone can maintain control or even sanity when faced with that many decisions. Not to mention being able to answer questions like:
“which of the services that have financial implications and are exposed to business partners are Sarbanes-Oxley compliant?”
of for those in Europe:
“which of the services that deal with personally-identifiable information and are consumed across EU boundaries comply with European privacy laws?”.
As a result vendors keep building various “closed loop governance solutions” and “agents” that auto-discover services and auto-apply policies based on things like names, locations and regular expressions, etc thus completely defeating the purpose of Governance as a deliberate human oversight activity.

Another limitation of governance by policies is that it inevitably leads to proliferation which in my mind is the greatest threat to overall success of SOA and its ability to deliver on the original promises. The problem is that one can attach only one set of policies to a service without creating an ambiguity. Imagine a simple example: a search engine that wants to expose its capabilities as a Web service. They have a singe service find which they have to expose three times to support three different Service Consumption Scenarios:

1. One for developers and enthusiasts building a web site about their cat. There is no security or Quality of Service (QoS) guarantees, requests can be throttled and the usage is limited to 1000 invocations per 24 hour window.
2. For subscription clients wishing to outsource the search box on their web site another set of policies will require authentication (to prove subscription), this time impose no usage limits, still offer no defined QoS and requests can be throttled during the busy hour.
3. Finally for those for whom search is their core business the third version will provide premium service with pay-per-use pricing. This time the policies will require both authentication and authorization, provide specific QoS guarantees, implement usage metering to support billing, and guarantee that the requests will never be throttled.

This is the same service, running on the same server farms and they had to replicate it three times just to decorate it with different policy sets. That it not too intimidating by itself, however if we go back to the couple thousand of enterprise services, multiply it by dozens of Service Consumption Scenarios that likely exist in any sizable business and then throw in another multiplier for multiple versions of each service that have to be supported things will get really scary…

Luckily there is a simple solution that addresses both of these problems at the same time and facilitates role-based governance for a good measure – it is governance by contract, which brings us to the subject of today’s post, because it in turn leads to the notion of analysis-time governance as the missing third pillar of the SOA Governance triad, that includes:

• Identifying key Governance Aspects or non-functional characteristics of Enterprise Services that are of importance to the Business today;
• Identifying main Service Consumption Scenarios, such as a service with financial implications exposed to business partners that the business needs to deal with;
• For each of the Scenarios identifying a sub-set of aspects that are pertinent in this business context and defining appropriate business policies regarding these aspects;
• Defining, validating, reviewing and approving reusable Governance Contracts or complete sets of governance policies regarding the applicable aspects for every scenario;
• Applying the relevant contract to each service, that is to be consumed according to a particular Scenario;
• Auditing the use of services in each context and validating that the changing business needs continue to be met.

It seems that the analysts are also starting to recognize the need for Analysis-time Governance as the only way to achieve true compliance in the SOA domain allowing businesses to answer questions similar to the ones introduced at the beginning of this post. And clear separation of functional (operations, versions, schemas, messages and protocols) and non-functional (aspects, policies, contracts and scenarios) sides of enterprise services allows to build role-based governance solutions that support collaboration between SOA Developers and Business Analysts, while ensuring deconfliction and maintaining the separation of concerns. Cool, isn’t it?

Please bookmark with social media, your votes are noticed and appreciated: