Lustratus LiteBytes

Archive for the ‘SOA’ Category

Just a short note to say that I’ve uploaded a new report to our web store at www.lustratus.com.

The report, entitled A Competitive Review of SOA Appliances focuses on Intel’s SOA Expressway, IBM’s DataPower range and Layer7’s SecureSpan SOA Appliance. In the report I compare and contrast the technical and strategic approaches each vendor takes to addressing the task of creating, managing, accelerating and securing a service oriented architecture using appliances.

The report can be found here.

Steve Craggs

I recently commented on a query in another blog about ESBs, and what they are in relation to SOA.

Since this is a subject that I continue to get asked every now and then, even though I have blogged on it a number of times before, I thought I would reproduce the response I gave.

It starts right at the ESB beginning, and concludes with a few old Lustratus paper references that I believe are still relevant in introducing the ESB and SOA concepts.

When the Enterprise Service Bus concept started off life in the mid 90s it was as an extension of a messaging pipe – that is, message based communications. Prior to the ESB, messages were sent with tools such as IBM’s MQSeries (now WebSphereMQ), Progress Software’s Sonic MQ and a range of others, particularly JMS (Java Messaging Service) implementations. Users quickly realized that more was needed than just the ability to send a message from A to B – value add capabilities were needed such as data transformation, message enrichment and dynamic routing of messages based on content or other circumstances. This resulted in the emergence of ‘message brokers’ – pieces of code that acted as a ‘hub’, where any required actions could be taken on in-flight messages. This is where the ‘hub and spoke’ concept that was the basis of the EAI market came from. Messages from A to B would go via the hub so that the required intelligence could be applied, with the A and B endpoints requiring little intelligence at all.

However, two things happened that caused the ESB concepts to emerge around 1996-97. Standards activity in the integration marketplace increased and took root, and users wanted to find ways to lower the entry price for integration – having to buy a hub was very expensive, particularly when connections were few in the early stages of integration development. There were also fairly groundless concerns about availability with the hub and spoke model due to the perceived single point of failure. As a result, the ESB emerged.

The ESB did two important things – it leveraged major standards, in particular the web services standards offering a standard way to invoke the messaging capabilities of the bus, and it adopted a ‘bus’ rather than ‘hub and spoke’ architecture which resulted in a much lower deployment cost, at least in the early stages of integration development. The bus concept involves placing more intelligence at each node, so that messages can flow A to B without going through the hub. In flight message processing happens at the individual nodes rather than at the hub.

So, an ESB is actually just a ‘smart’ communications pipe, providing not just a way to transfer the messages from A to B but also the in-flight capabilities (often called mediation services) required. In addition, this is all available under a layering of standards. This is why typically ESBs are used with web services invocations, and often utilize JMS servers for the actual transfer mechanism.

SOA is something much bigger. It is a way of architecting your IT programs around a service-oriented concept. The absolute key to this is that an SOA service relates to a BUSINESS piece of functionality as opposed to some programming activity. So, you do not have an SOA service to read a record from a file, but rather an SOA service to ‘Get Customer Details’ which internally will end up reading customer information from files and so on. The secondary characteristic is that this service must be able to be invoked from anywhere, with no requirement to know where the target service will actually run. Therefore, it is clear to see that SOA requires some sort of communications capability, and while this does not have to be an ESB, the ESB fits the role very well particularly with its affinity to standards such as web services.

There are a number of free white papers at www.lustratus.com that discuss this topic in more detail, particularly ‘SOAs and ESBs’, ‘What is an SOA Service’ and ‘The Year of ESB-ability’.

Steve

Software AG seems to be defying predictions and surprising the market at every turn.

Once seen as a sleepy European software house based largely around legacy system technologies, it has taken major strides to transform itself into a major global software industry player. Its acquisition of webMethods a few years ago surprised the market, with many analysts unconvinced that it could make a go of the move into integration / SOA middleware, but it has done a fair job of building some momentum by tying the webMethods portfolio up with its own CentraSite governance technology, providing service-oriented architecture (SOA) with integrated governance.

Then it once again shocked the market by snatching IDS Scheer, the well-known supplier of modelling tools, from under SAP’s nose. Given that the IDS Scheer technology is used by most of the major SOA suppliers across the world for modelling, and in particular is a key part of the SAP portfolio, this would appear to give Software AG lots of cross-sell opportunities across the two customer bases and throughout the SAP world.

Now it has announced its 2Q09 results, and they make pretty good reading ont he surface. A 9% increase in product revenues is particularly noteworthy give that so many companies are struggling to show any year-on-year growth in product sales. However, before getting too carried away it is worth delving a little deeper into the numbers. The product revenue numbers include maintenance as well as license sales. Licensesales actually fell, as with most other companies. Maintenance revenues jumped by 20% – does this mean that the company has built a much larger maintenance base, or is it actually a reflection of a more aggressive pricing policy? Then there is the split between the legacy business (ETS) and the SOA/BPM business(webMethods). License revenues in this segment were down 15% – not very encouraging since this is the strategic business unit. Also, it is noticeable that maintenance revenue in each segment increased by about 20%, suggesting that this rise does indeed reflect a price hike.

However, taking all this into consideration, Software AG is still looking to have moved forward substantially from a few years ago, and assuming the IDS Scheer acquisition goes through OK there should be lots of opportunities for the company. Of course, a cynic might point out that by adding IDS Scheer to the webMethods portfolio, the company has made itself a highly attractive acquisition target to someone – perhaps SAP?!

Steve

I was recently pointed to an article in Mainframe Executive magazine written by David Linthicum on the subject of “Mainframe SOA: When SOA Works/When SOA fails”.

I think the friend who suggested I read it was making mischief, knowing my views on the subject of SOA and guessing (correctly) that this article would wind me up.

In summary, the article says that SOA is a large and complex change to your core architecture and working practices and procedures, and that the success or failure is dictated by questions such as executive buy-in/resourcing/funding/skills, and not technology selection.

The truth about success with SOA is that it has little to do with the technology you want to drag into the enterprise to make SOA work, and more to do with the commitment to the architectural changes that need to occur

I have two problems with the opinions stated in this article. The first is to do with changing attitudes to SOA, and the second with the technology comments.

Let me first state that I am well aware that if a company wants to adopt an enterprise-wide SOA strategy designed to take maximum long-term benefit from this new way of leveraging IT investments, then this requires all ofthe areas brought up in the article to be addressed – skills, management buy-in, political will, funding and a strategic vision coupled with a tactical roadmap. I have no beef with any of this.

But I would contend that the world has changed from two years ago. The financial constraints all companies are experiencing have more or less forced the long-term strategic play onto the back burner for many. Some analysts actually like to claim that SOA is dead, a statement designed to be controversial enough to gain attention but to some extent grounded in the fact that a lot of companies are pulling back from the popular SOA-based business transformation strategies of the past. In fact, SOA is absolutely not dead, but it has changed. Companies are using SOA principles to implement more tactical projects designed to deliver immediate benefits, with the vague thought of one day pulling these projects together under a wider strategic, enterprise-wide SOA banner.

So, as an example, today a company might look at a particular business service such as ‘Create Customer’, or ‘Generate Invoice’, and decide to replace the 27 versions of the service that exist in its silos today with a single shared service. The company might decide to use SOA principles and tools to achieve this, but the planning horizon is definitely on the short term – deliver a new level of functionality that will benefit all users, and help to reduce ongoing cost of ownership. While it would have been valid a few years ago to counsel this company to deliver this as part of an overarching shift to an SOA-oriented style of operations, today most companies will say that although this sounds sensible, current circumstances dictate that focus must remain on the near term.

The other issue I have with this article is the suggestion that SOA success is little to do with the technology choice. Given that the topic here was not just SOA but mainframe SOA, I take particular exception to this. There are a wide range of SOA tools available, but in the mainframe arena the quality and coverage of the tools vary widely. For example, although many SOA tools claim mainframe support, this may in actuality simply be anMQ adapter ‘for getting at the mainframe’. Anyone taking this route is more than likely to fail with SOA, regardless of how well it has taken on the non-technical issues of SOA. Even for those SOA tools with specific mainframe support, some of these offer environments alien to mainframe developers, thereby causing considerable problems in terms of skills utilization. It is critical that whatever technology IS chosen, itcan be used by CICS or IMS-knowledgable folk as well as just disributed specialists. Then there is the question of how intuitive the tools are. Retraining costs can destroy an SOA project before it even gets going.

For anyone interested, there is a free Lustratus report on selecting mainframe SOA tools available from the Lustratus store. However, I can assure companies that, particularly for mainframe SOA, technology selection absolutely IS a key factor for success, and that while all the other transformational aspects of SOA are indeed key to longer term, enterprise-wide SOA there are still benefits to be gained with a more short-term view that is more appropriate in today’s economic climate.

Steve

I have just returned from a couple of weeks around and about, culminating in presenting at the Integration Consortium’s Global Integration Summit (GIS), where I presented the Lustratus ‘BPM Sweet Spots’ paper.

One message seemed to come out loud and clear from the conference – pragmatism is the watchword for 2009.

There were two other analyst presentations apart from the Lustratus one, and I was surprised to see that both presenters pitched a message along the lines of ‘you will never succeed with SOA/Integration/BPM unless you get all the strategic planning and modelling for your enterprise done first’, combined with a suggestion that the presenter was just the resource to ask for help! This contrasted sharply with my own presentation of choosing tactical targets for BPM rather than going for a strategic, enterprise-wide, fully modelled approach.

I was wondering if I had read the mood wrong in the marketplace, but then the eight or so user case studies all proved to be tactical strikes for specific business benefits rather than the more extensive strategic approach more common a year or so ago. It was nice to be vindicated – it looks like 2009 really IS the year of pragmatism and short-term practical considerations.

Steve

There has been a lot of chatter recently about measuring SOA ROI – take a look at Loraine Lawson’s recent blog for instance.

or Gartner’s results of a UK-based survey of SOA adopters. However, one of the benefits that I think a lot of people miss, or do not attribute enough importance to at least, is Visibility.

Basically, the visibility story goes that with SOA, since you break up operational components into discrete business services, then it becomes easy to monitor entry and exit to these services and hence business operations flow. This gives a clear picture in business terms of execution and performance – not just what is happening, or how many times, but HOW business is being carried out.

Gartner did touch upon visibility,

Improved Efficiency in Business Processes Execution – Isolating the business logic from the functional application work enables a clearer view of what a process is, and the rules to which it adheres. This can be measured by lower process administrative costs, higher visibility on existing/running business processes, and reduced number of manual, paper-based steps; better service-level effectiveness; quicker implementation of process iterative or of variants of the same process for different contexts.

However, the Gartner focus was only on visibility as it relates to execution efficiency. In fact, SOA-based visibility offers another benefit which, in today’s tough times particularly, can be a real big hitter for executive management. It enables management to see how process are being executed – in other words, it provides the ideal tool to monitor compliance against a growing raft of regulatory requirements across just about every industry. In order to demonstrate that your systems comply, it is necessary to be able to see what they are doing and how they are doing it. This is what SOA delivers.

So how does improved compliance management fit into the ROI picture? True, it is very hard to attach a dollar amount to compliance – however it certainly matters. With the amount of public and political scrutiny of corporations today, it is absolutely imperative that executives can show they are faithfully adhering to regulations and guidelines. Failure to do so will not only risk severe penalties, but also probably lose them their jobs! Now THAT’s a compelling business case….

Steve

As one might have expected, Microsoft tried to ignore the Enterprise Service Bus (ESB) movement for a long time, but eventually it had to do something to answer demands of its customer base looking for SOA support.

Its response was Microsoft ESB Guidance, a package of

architectural guidance, patterns, practices, and a set of BizTalk Server and .NET components to simplify the development of an Enterprise Service Bus (ESB) on the Microsoft platform

Let’s be honest. This is a typical Microsoft ‘fudge’. Microsoft ESB Guidance is not a Supported Product, but is instead a set of guidelines and one or two components. It is a Microsoft Patterns and Practices offering – in other words, you are on your own. This may be fine if you are a Microsoft development shop, but far more worrying if you are real business user with extensive Microsoft presence. It has a lot of the disadvantages of Open Source, but you still have to pay for Bizt Talk etc..

So what does the future hold? Will trying to bring the Microsoft server world into the SOA domain always be a matter of risk and going it alone? Will Microsoft productize Microsoft ESB Guidance? Are there any alternatives other than just consigning the Microsoft platform to run in isolation on the fringes of the enterprise?

Fortunately, the Microsoft model may actually be working here. I do not believe Microsoft will ever productize ESB Guidance – after all, they have had two years and are still maintaining there are no plans to do this. However, what this position does do is it encourages opportunists to jump in and develop products based around the Microsoft technology and guidance materials. An example is Neuron-ESB, from Microsoft specialists Neudesic.

So, while Lustratus strongly cautions users about the effort, cost and risk of using Microsoft’s own ESB Guidance package, the idea of utilizing a Microsoft-based supported ESB product from a specialist vendor is much more attractive. Of course, whether these new Microsoft-based ESBs are any good is yet to be seen….

Steve

I was reading a recent post from Joe McKendrick the other day on measuring SOA success…

…and it reminded me of a related issue – that of measuring services reuse. SOA adopters often moan to me that despite having implemented SOA and deployed many services, reuse rates are down at the 1-1.2 level – in other words, virtually no reuse. They seem to want to pick a fight with me because as an advocate of SOA I have often pointed to reuse as one of the more measurable benefits. After all, achieving a high level of reuse is a clear indicator to business executives that efficiency is increasing, since the implication is less development is required to do new things.

I am starting to get pretty short now inthese conversations. I wish, wish, wish that people would heed my previous advice - don’t think of SOA delivering reusable services, think of it as a great tool for SHARED services. Obviously reuse will come through services being shared - so what point am I trying to make? The problem is people are choosing to build ‘reusable services’ with SOA and assuming that others will start reusing them. It is the old ‘Build and they will come’ philosophy. This rarely works – it is worse than a scatter-gun approach. If users instead think about what services would be good candidates for being shared first, and then develop these as SOA services, reuse levels will definitely improve.

So, when getting started with SOA, don’t encourage everyone to start building code into services and hope that reuse will come as if by magic. Start off by deciding on the logical services to build that will be shared – things like get customer history, or create new customer. Then go ahead and build these shared services candidates, and see reuse levels climb….hopefully making it easier to justify your SOA investments to the business.

Steve

Ultramatics has just announced SOA SafeGuard product, which is designed to shut one of the major SOA security holes – the opportunity to inject virus and other malware threats through XML file sharing.

This is good news for SOA implementers, but also introduces an interesting new stress point for IBM. Back in 2007 I was on a podcast where I identified the five SOA security traps, one of which was the XML problem. To summarize, most virus and other threat detection solutions look at the datastreams coming into the system and identify threat signatures that indicate the presence of some noxious code, but unfortunately they cannot see inside the XML wrapper, so to all intents and purposes the contents of any attached XML file are invisible. This offers the opportunity for malicious agencies to pop in some nasty code into the XML content and smuggle it through the security gates to the enterprise. Of course, it is not immediately obvious how this would help, in that getting this code executed might not be so easy, but hackers are smart….therefore it is best to close this exposure.

One way to close the window is simply to forbid any XML file sharing, but since industries such as healthcare now more or less rely on this to conform to industry standards and regulations, this is not really practical. The new Ultramatics product claims to be able to protect from these types of intruders. It runs on the IBM DataPower XI50 Integration Appliance, providing a hardware-based shield that can see into the XML files and weed out anything unpleasant. This solution will be very valuable to many SOA companies worried about security.

But there is something else interesting in the product details. The datasheet for the product says it can be used (in conjunction with IBM’s MQSeries) to:

Create a SOA ESB that can perform routing, transformation and protocol mediation functions

This is intriguing. Of course, the idea of an ESB appliance is not new, but the interesting point is that IBM is supplying this capability through the Ultramatics product…..I wonder if the other IBM ESBs, WebSphere ESB and WebSphere Message Broker, see this is encroachment?

Steve

I was reading Danny Goodall’s post on his strategic marketing blog about standards-based marketing…

….and it brilliantly illustrated a point that I think is often experienced in the software marketplace – some vendors rush to back standards and push them, but only to the point that they fit with their own goals.

The example Danny discusses is Sonic Software, part of software vendor Progress. Sonic is well known as the first software vendor to use the ESB acronym (Enterprise Service Bus), and did indeed peddle the standards message hard asDanny, the marketing guru behind Sonic’s early success, remembers:

All the while I was creating marketing programs that stressed Sonic’s commitment to standards and, by implication, I was de-positioning other vendors’ technologies as being the Devil’s spawn due to their reliance on proprietary features. “How,” we asked “would organisations ensure interoperability between their, and their trading partners’ infrastructures if they didn’t conform to the emerging standards?

Of course the standards message is very attractive to users. Buyers are keen to be able to ensure that not only can components interoperate without a lot of extra work, but also that vendor lock-in is weakened through the ability to substitute components from different suppliers, bringing prices down and reducing risk. Therefore, vendors that preach standards may come across initially as ‘good guys’. However, it pays to look more closely to find out how serious the vendor REALLY is about standards. In the Sonic case, while it talked a great story, the mystery was that its own ESB product was unable to run over any standard JMS-based messaging pipe for years. Instead, it used a proprietary interface that ensured Sonic ESB would only work over SonicMQ, the Sonic messaging pipe. This was a real problem for many prospects, because IBM’s WebSphereMQowns around 80% of the messaging pipe business and therefore prospects interested in an ESB were frequently looking to run it over their existing software. This restriction was arguably one of the key reasons Sonic lost its leadership position in the ESB market.

So why did Sonic take this line? Obviously, only Sonic knows, but a cynic would argue that it consistently refused to support the JMS standard in the early years to ensure that it could force the sale of its own messaging pipe. No matter that this meant the user often had to buy another one on top of the incumbent solution.

I am not picking on Sonic here – this is only one of many examples where vendors claim to be standards-based while not shrinking from proprietary solutions when in their own interests. And of course, it is entirely understandable – after all, software vendors are businesses too. To me, the important thing is that users keep away from the rose-colored spectacles. Standards are valuable, and vendors do provide important support, but there will always be compromises.

Steve