John Crupi's Weblog

So there are business services ... Are we not just talking about another service sterotype ... You still need to define data services ... right? Top down vs bottom up-- top down business services -- bottom up data services? BTW ... John ... I fished in the google pond and found your deck a .pdf on eBay 3 -- it was excellent.

Posted by Eric Marts on prill 21, 2005 at 01:16 MD EDT #

John, Excellent posts. I cam to know you via the Software Factories book you wrote an excellent intro for. I am in contact with Jack and Keith and run what I believe is the largest DSL/MSFT community I know of on GotDotNet with 75 people here. I would love your opinion on what I am up to (I have a book coming out and this will likely make it in): I am experimenting with 'Business Expert/Programer Pairing for Intention Capture'. It is similar to pair programming but it is only done for the Unit Tests in Test Driven Development and you pair business analysts with programmers much like pair programming. The Business Analyst (BA) never drives (at least not yet) as this is not like Ward's FIT. This is at the Unit level and the Unit Environments are still code, no? I do have a short term solution and a longer term improved solution for allowing the BA to drive and switch just like normal Programmer Pairing right now.. Right now it is looking like InfoPath will be the short term tool for the creation of XML Schemas that would be used to generate the actual Test Cases using the target TDD Framework. We would also be able to reverse engineer our tests into an XML Schema format and I would look to create a standard for a XML Schema for the definition of Unit Tests. With .NET Integration InfoPath is a friendly enough environment (and there are similar Java tools I am sure that could use Reflection) as it empowers you to do quite a bit, as we have, with reflection access in say dropdowns to existing classes, methods, parameters, etc. in assertions that are created. So the flow would be pairing --> XML Schema Document --> TDD Test Fixture Generation --> In IDE Modifications as required --> Reverse Engineer into the same XML Schema Format using Reflection or simple text parsing Possible transformation of the XML Schema into HTML, XML, TXT, Etc. While with the BA we would not do as much test writing - code writing - test check - green - refactor - test - test writing. We would front load the testing and then implement when the BA runs off. My goal is to also use the XML Schema as the basis for DOCUMENTATION as many will agree; the TDD Unit Tests are a fantastic source of information for both the developers and increasingly the Business People (and your posts support this very, very well). Many have discussed a ‘comment free’ codeing style as a kind of panacea due to TDD and extensive refactoring to the point the code is a collection of invocations. That might be a fantastic goal to strive for and it is something I am trying. But I add comments when needed. I also want to make a point that the Unit Tests should follow similar quality quidelines. Convery literals to constants, use common sese. It's still code ad that would all come back over when you reverse engineerit back to the shared systen. I need to investigate the use of tools like Ndoc/JDoc as tools for the document generation, as well as other tools. We would likely need to add additional metadata (as comments) into the source to be read in as well as potential custom attributes (as examples). When a BA joins a pair doing TDD and Coding, the business analysts will only drive using the TDD Activities as necessary using the generation tool. He would be a 'third' in pair programming and the two developers would be encouraged to focus on developing their tests perhaps in a more front-loaded way then usual to not make the BA wait around while they implement. They would focus almost exclusively on the 'Service Oriented' Layer (AKA Business Services typically build using COM+, J2EE, .NET WSE 2.0, Remoting, MSMQ, MIME/SMIME, HTTP(S), Sockets, MQ-Series, and soon Indigo, etc. I believe the SOA interface should be almost completely understandable to a BA as you state so well. My thinking here is directly in line with yours: 1) You have no core business logic in your UI so there is really no need in my opinion to do TDD at the User Services Tier. This also encourages an good Object-Oriented Design (hopefully). They key is getting developers to not dump code in event handlers but to use UI Tier Page Controllers. 2) There will often be concurrent 'Bottom Up' and 'Top Down' activities happening in your architecture with your team’s pairs. For example, you may have a pair getting the Enterprise Library installed for .NET (and plugged in) and perhaps writing an adapter as your database is not one of the shipping adapters, while you have another pair defining core service APIs that you expose directly to your consumers. Obviously this needs to be managed in your iterations such that everything is done for a reason that can be tracked back to at least an index card somewhere. It' a bit like train tracks meeting. Your more Data Centric people working up and your more communicative, extroverted people working with the BAs working down. To be clear, we are not talking about sorting grids for example, or changing graph display options (that in my approach can be done using non-pairing techniques and prototypes (yes I am a 100% believer in the prototype - however I do not throw them away. Another story for another time. 3) In many cases it is perfectly acceptable for the BA to even NAME the SOA entry points you expose as these are arguably as much business as technology driven. I have even used a SINGLER entry point only and used the XML parameter to fully describe all that is to be done. This can be very powerful and help with security and other administrative issues. That is a call for your company. If I had to choose one approach I would go with the single SOA Entry point. Your flexibiliy is amaxing as new services can be added at will, you just need to ensure the chain of responsibility can find a handler for the message (which should be metadata defined and invoked at run-time in a 'late-bound' way). The technologist will have plenty of work at the layer blow the SOA Facade implementing the Mediators to handle collaborative work and of course much more. In the longer term my company is very seriously debating the construction and selling of a very comprehensive BA/Programmer Collaboration Tool (web based) that would allow for all kinds of interesting features like a 'screen white board' for drawing shared images, and of course the same level of intelligence using reflection on your assets. This tool would also offer an SOA layer ITSELF so you could build your own utilities as you like. We see Sharepoint WebParts, potential ActiveX Controls, Applets, CRM Integration, Software Factory Applications, Seamless Integration right into IDEs, and real-time visualization of any transformations that are made. I would love any feedback on your thoughts around this as a viable commercial software product. It would not be SYSTEM Test related (at least in the beginning) it would only help in defining the assertions for the SOA Entry Point Valuators (which is many cases could be XML Schemas themselves if our input is XML instead of parameters). Imagine we are using InfoPath. Developer: "I am still learning about the ins and outs of our rules around processing trade transactions. I am building the part of the system that will accept trade requests and I need to pair with you to understand all of the requirements in assessing that I will execute this work correctly. The power of us working together is BEFORE I EVEN START the coding of your system you can tell me if I have made any bad assertions or missed any. Those come from you anyway, as I don’t make the business rules, I just code them. So I like exposing a way for our customers to communicate with us through a 'channel' called TradeExecution." Business Analyst: "Actually that name is no good as it infers that we will always be able to do what they ask, as in 'execute their trade'. The reality is sometimes due to reasons we will cover the trade cannot be done. I would prefer if that was called 'RequestTradeExecution'. Developer: "That's cool with me.. " I believe (as does John in his Blog posts) that the SOA layer should be driven by BUSINESS semantics, not technical. Therefore it should be the BAs who have a hand in helping us define them. NOTE: We would not necessarily discuss with the BA that the 'RequestTradeExecution' message takes in a single String type which is a validated XML DOM with a corresponding series of XML Schemas that would be determined via a Chain of Responsibility to find the handler for the trade (be it an equity, mutual fund, individual bond, etc., and be it a sell, buy, and any of the various options around market orders, limit orders, 'all or nothing', 'Only Good Till End of Day', etc. This would likely be built using the Decorator pattern on a Master Abstract 'TradeRequest' class. As it has not been executed, and the actual execution can be quite different from the request, it makes sense to capture them separately. However we may find the trade is very similar to the request. You could possibly have a higher level parent called 'TradeInformation' with a branch for Requests and one for actual trades, with relationships tracing requests to their eventual executions (if they happen). This is where the developer will code with his CODING pair. The important think is he has his SOA Assertions and concepts straight. Another area of shared knowledge may be created: BA: "OK let me sit and use this computer for a bit. We obviously need to ensure the Customer exists in our system in a state that would allow the trade to happen. I see here you have a 'Customer' already listed so I will pick that and we must have the internal standard company contract signed before we can take orders. We have been lazy about that and we have to make this system really enforce this. Ah I see you have another dropdown called 'LegalPaperworkOnFile'. Is that the form 153a? It is? OK that is what we need. So I now will select that this need to be 'true' as I can only pick true or false. This system is pretty smart. (NOTE: Behind the scenes via reflection we saw that it was a simply property that was a bool so we could limit the values). So now I click this ADD button and ok, I see we added an Assertion that 'Assert.IsTrue(Customer.LegalPaperworkOnFile). Wait I want to fill in that field so I can explain why so if it fails they can see. I just click on this 'edit' button right? Ok. Now I type: 'In order for a Customer to trade with us we must have Form 153a on File. This is someting we maintain in our CRM system and I remember the field is called 'Legal Paperwork Has Been Recieved' so that must map to LegalPaperworkOnFile. I also will add: This is a mandate from our CEO that starting with the launch of this system no trades will be accepted without that paperwork. This is due to the federal 'know your customer' issues and some problems we have had understanding where some of our clients made their money from (like that Pizza Delivery Boy who was placing $25,000 trades. Then he gets arrested in a drug selling ring. Man we are so lucky they never tracked anything back to us. With this form we will always know. Developer: OK that will all get generated into the test so if it fails everything you just typed will show up. Do you have more assertions you would like to enter: B.A.: "Yes but I don’t see what I need here. There are certain securities that we do not allow to be traded on Margin due to their volatility. How can I see if this is a margin trade for one of those? They also change all the time so be aware of that: Developer: "Wow we missed that one. It was never written on the early planning sessions where we had you write on the index cards each thing the system must do. Then when we entered the iteration we asked you to just write up the items we were working on for these two weeks. But I never saw that." B.A.: "I totally forgot.. I will use the 'type in' feature to capture this and then you can do your magic" . The BA simply manually types in textboxes the information he needs for the assertions to occur. Developer: "Sounds Good" Those tests would be generated ‘commented out’ as we have uncovered the need for a collection type data structure to hold all the restricted Margin Stocks (globally cached) and we need to modify the XML The developer even looks at the prototype and THERE IT IS. ‘Margin Trade’. How did he miss it? Schema to add an attribute if the trade is a margin trade and the corresponding attribute (I like to be fairly wide grained in the results and calls for SOA so I would likely NOT create a property on the trade I would define it in metadata and use late binding. Then I can call 'Trade.GetDetails" and get back a structure with a lot of information rather then having to make many expensive Out of Process calls (nothing new there) Kind Regards, Damon Carr, CEO and Chief Technologist agilefactor www.agilefactor.com

Posted by Damon Carr on prill 24, 2005 at 10:17 PD EDT #

I agree with you that services should be coarse-grained. Regarding using business terms to describe services - in my opinion, technology services can also be offered. Suppose you have a general purpose optimization algorithm that serves various industry needs, could you not provide that as a service ? Or would that not be termed a typical SOA service? Would you necessarily have to have a "business" term to explain a service ?

Posted by Debnath Mukherjee on prill 05, 2006 at 05:11 PD EDT #