Open Source SOA – Are we there yet?
I am often asked whether open source (OSS) SOA is a reality yet – whether it is ready for prime-time, as they say.
The answer, as is often the case, is ‘It depends’. There are many OSS projects in the marketplace around ESBs, Integration and SOA, but just having a project in place is a long way from having production-ready software. For a start, there are the questions of maintenance, support and even indemnification against possible future legal activities. The most useful projects are those that have an associated commercial company addressing these types of areas.
However, the other aspect to consider is that most opern source projects are started and driven by technically adept programmers, so they tend to be oriented towards programmer usage. In SOA, this may be acceptable, depending on requirements, but it may also be desirable to have a solution more oriented to business analysts. They key is to be clear on what you want, and on what is being offered.
For a longer discussion on this topic, Lustratus has just published a free assessment of one particular OSS integration vendor, MuleSource, and its open source offering Mule. This paper considers a number of these types of key questions over OSS SOA, but of course in the MuleSource context.
Steve
Posted in enterprise service bus, esb, Imported, Mule, MuleSource, Open Source, SOA, SOA developmentWhy does SOA keep forgetting about data?
Every now and then, we all hit that point when we want to stop everything and say enough is enough.
I guess I have just reached that point. I spend my time working with buyers, sellers and implementers of SOA, and just about every conversation is about applications. There is a myriad of tools and platforms that are focused on being able to turn existing code assets into SOA services, building composite service and constructing orchestrated flows…..but everything is discussed from the point of view of the application.
I guess what frustrates me is that when I talk to people about what they want their services to do, particularly when you get to composite services where functions are linked together, the answer is usually two-fold – I need to run the following applications or components, and I need to access the following data. When building an orchestration flow, for example, it is often very useful to be able to interrogate data to help determine the appropriate next step in the flow.
It seems to me that most SOA products don;t really consider this. At most, they allow database calls during flows, but this is hardly in the spirit of SOA. Surely, these calls should be allowed to any data source in the SOA network, whatever the data architecture or format. This would fit with the SOA theme about offering everything under a standard interface.
Come on guys – I know data might seem boring, but it is just as important as the applications themselves.
Steve
Use a hot-house to get better productivity
I was keynoting at the DeutschePost SOA Days technical seminar held in Bonn, Germany last week, and while I was waiting to present I sat through a very informative presentation from a large telecoms company about its efforts with SOA.
However, one point that really caught my attention was the fact that the company has had some success with improving productivity through focusing on 90-day cycles. This extremely challenging timeframe is assisted greatly by the SOA approach used by the company, but a key element is that new projects start off life with a ‘hot-house’ activity.
The idea is to gather the core team of people involved in the project in one room, for however long is required, to spec out the project and select the implementation approach. This looks at the business and IT requirements and implications. What I love is that this ‘hot-house’ is carried out in a special room where there is only local networking available (no internet or email), and it even has mobile phone jammers to block calls. Results appear to have been extremely impressive. The hot-house sets the scene for a rapid development effort. Admittedly not every 90-day cycle completes the project – it can determine that another cycle is needed for additional research for example, but project delivery has definitely speeded up.
As for me, I just want one of those mobile phone jammers – I would love to take one on the train, and turn it on intermittently…..
Steve
Posted in best practices, Imported, Industry trends, organization, productivity, SOA, SOA developmentReuse still seen as top SOA benefit
I was interested to see, in a recent survey carried out by PMP Research, that the top-scoring benefit expected from SOA adoption was still reuse.
The research was documented in the September 2007 issue of Conspectus, and on 1 1-5 scale ‘Business process service re-use’ scored 4.1 on the benefit list, followed closely by ‘provides a more effective integration platform’.
Many SOA advocates talk about SOA being fundamental in terms of building a flexible and adaptable infrastructure while aligning IT and business requirements more closely, resulting in a more agile and effective platform for supporting business needs – so how come reuse still tops the list?
My own view is that reuse is simply the benefit that is the easiest to take in. The others I list above are very powerful, and can promise significantly more overall business value, but they require more than just a technology change. They require changes to working procedures and practices, and much better communications between business and technical communities. So, perhaps, in the eyes of the pragmatic respondents, reuse suggests itself as a reasonably obvious gain which even has a fairly clear and measurable monetary return attached. Reducing redundancy should have a direct correlation on application maintenance costs, for example. And on top of this, it can deliver value from a purely IT point of control (although the value will increase if business departments are involved too).
Steve
Don’t forget productivity when selecting SOA software
When I talk to vendors of SOA software about their offerings, I always focus on the tooling provided with the product – from development tools to deployment tools to life cycle management tools.
Of course vendors will claim that their products are easy to use and highly productive. Unfortunately, these claims are often not properly challenged as products are being selected for SOA projects based primarily on functionality and ‘enterprise’ attributes. This is partly because it can be quite hard to establish just how productive the software will be when used outside the demo or pilot comfort zone. I said unfortunately above because anybody involved with integration projects (SOA or otherwise) will understand the high proportion of effort spent on dealing with the unexpected incompatibilities or consequential issues: Effort which can be significantly reduced with good tooling.
Another part of the problem is defining what we mean by easy to use and productive. In particular, whether a tool is easy to use or productive is closely coupled with who will be using it. A productive tool for an expert java developer is very different from a productive tool for a business analyst. Without unfairly blaming our marketing friends, there is a tendency in the industry to assume anything GUI based must be suitable for business analysts and so long as it works in Eclipse, java developers will be happy. (For instance, I remain to be convinced that the business process execution language, BPEL, is something any business analyst would use – pretty interfaces or not) Needless to say, reality is much more complex and when evaluating the tooling around your preferred product makes sure to match the capabilities against your developer base and not be fooled by the apparently pretty pictures (or lack of them).
Ronan
Is your legacy integration just a veneer?
Summer seems to be a time for reading through that huge pile of interesting articles and magazines that you can only find time to look at on vacation.
On flicking through my own mountain of stuff, I came across the Q2 edition of Financial-i, a magazine targeted at the Financial Services industry. One point to jump out at me was a comment from Paul Joynt of Nordea, a Scandinavian finance house. Paul was pointing out that SOA does not necessarily solve the problems associated with legacy integration.
The article, ‘SOA – is it worth the effort’, is available from the Financial-i site if you register, but Paul comments that covering a legacy system with a wrapper “so it looks like what you want” still leaves problems with the next level of change, because “it’s only a veneer”.
I think Paul has hit on an important point here. Different vendors in the SOA space have different approaches to addressing the problem of integrating legacy systems. Some will simply ‘hand off’ the request for legacy information to a tool from the legacy supplier – in the case of IBM mainframes this might mean using WebSphereMQ as the bridge, for instance. Others might approach the problem in some sort of screen-scraping or other interface simulation approach, where the legacy application is fooled into thinking it is running in its normal mode of operations. Yet more may generate code-based wrappers for each individual need, to be executed whenever a particular service is required.
To me, this all sounds too much like veneer in Paul’s terms. Although this might address immediate needs, future changes will continue to generate substantial additional work and the generation of more and more ‘special-case’ code and wrappers.
Instead, the best of breed legacy integration solution should embrace SOA and integration rather than try to fool it with wrappers designed to seal off the legacy world from the outside. Legacy integration should be about making the legacy system a full and active participant in the service definition and execution. For example, orchestration should be possible both outside and within the legacy environment. Services should be built with full participation from both sides. By taking this approach, the best of breed legacy integration tool will ensure that future changes will become easier, quicker, cheaper and more reliable.
For more information on the whole subject of legacy integration, specifically in the case of mainframe systems, Lustratus offers a free paper on the subject.
Steve
Posted in BAM, enterprise service bus, esb, Financial Services, Imported, legacy systems, SOA, SOA development, Web ServicesSOA for the scientific community – a practical example
I was intrigued to discover a discussion paper from the University of Southampton in the UK about how SOA is helping the scientific community.
The paper, entitled ‘A Collaborative Orthopaedic Research Environment’, describes how SOA has been used to enable a Virtual Research Environment for orthopaedic researchers to collaborate in the design, analysis and dissemination of experiments.
What I found most interesting were the reasons for using SOA as the base architecture for this environment. The key objective, based on input from the specialists who will use the system, was to provide an easy way to share scientific data and results from collaborative research. However, it was deemed essential that the system could also evolve based on the changing requirements of the user community. One example of change that the paper gives is that of knowing which of a wide range of protocols will be followed for a particular clinical trial. Not only do these vary considerably, but it appears they are also susceptible to changes in regulations.
The solution decided to utilize a coarse level of services, essentially using just four – to manage the trial-related data, analyse it, submit and disseminate related research articles and support discussion forums. However, through the reliance on SOA, these services are flexible and extensible, making it much easier to address the changing needs of this particular scientific discipline. In addition, the services are reusable, and some could therefore be usable for other scientific areas.
It’s good to see SOA being used to effectively address clear user needs in this way.
Steve
WS-Madness
OK, I think it is time to stop all the WS-Madness.
Web services standards have become a complete joke. As far as I can see, there are now at least 70 (seventy) web services standards and drafts – more than anyone could humanly want, and enough to create chaos, and completely negate the advantages of standards in the first place. Standards are supposed to increase choice, but with so many the likelihood is it will actually REDUCE choice (do you support standards 27, 39 and 40? no, I support 23, 42 and 63 though, any good?).
So, who is to blame for this debacle? Well, the blame is pretty evenly spread. Perhaps the most obvious target is the vendors, who have unashamedly used WS standards as a battleground to try to create differentiation from competition. So, by creating a standard that fits in with ones own design, it is then possible to use this as a reason for rejecting other players. However, vendors are easy targets – but are they really the villains here? After all, you could argue that they are just doing what they have to do – trying to compete, to win business and pay their employees / shareholders. The next obvious choice is the standards bodies themselves. Sadly, there are many ‘professional’ standards body members who get intellectual kudos from defining standards – whether they are worth anything or not. But even this may be missing the point.
Perhaps, instead, blame should be turned on users. The vendors have stepped in because of two things – the opportunity created by the desire for standards in the SOA space, and the vacuum resulting from the failure of users to take an active role. Similarly, standards bodies have leapt into the void because in a way they have to generate standards to have any worth in the world. But the real accusation is that the standards are rubbish – most are immature, many are useless or pedantic. In other words, they do not add value for SOA users and implementers. And isn’t this a case of users getting what they asked for? If users don’t want to get involved to ensure the RIGHT standards are created, that really mean something to users, then they cannot complain when others jump in to fill the vacuum.
My advice to users on web services standards is to ignore all but the important ones – SOAP, WSDL, WS-Security, and maybe WS-Addressing although this is not as mature as the other three. Then take a more active role to ensure these standards mature into what you actually NEED.
Steve
The sad state of SOA development tools
I think it is becoming generally accepted that SOA presents some pretty big issues outside of the technology domain in terms of skills and organisation.
Even before a new SOA developer in a particular organisation gets to dealing with cool new technology, he or she must understand the additional layer of process and operational knowledge in order to work within the SOA-based world.
One example is ‘service discovery’ which for the developer means finding services which work for their task in hand and understanding how to work with them. I use the term service discovery with caution as I do not mean automated discovery of the insidious WSDL that Steve referred to here. Rather I mean the task by which an individual developer gathers sufficient information to make the decision to use a service in his or her project. At some point WSDL may well be needed but the task really requires search capabilities and lots of human-centric information that explains and motivates the use of each potential service.
Let’s now step back for a moment and think about SOA and infrastructure software in general:
-
Infrastructure projects (whether SOA based or not) require an understanding of many aspects of both technology and business domain unlike business applications which for the most part require understanding of the business domain alone. This raises the technical skills bar for infrastructure projects.
-
SOA is meant to reduce the cost of development and maintenance. As there is a global shortage of highly skilled developers and these are concentrated in a smaller number of large organisations, it can only do that if it allows less expert developers be productive. It will be of some use if it makes the job easier for the experts but this won’t help most business departments and SMEs.
How well are the current SOA enabling products are doing? In particular, have they made the task of developing SOA easier than using EAI products 5 years ago which were held to be too difficult and expensive to use by many organisations? My opinion is that there has been some maturing of development tools but in general too little has changed. This means that SOA risks running into the same skill shortages that EAI ran into.
I think there are probably a few reasons why there hasn’t been a radical improvement in development tools:
- SOA stacks tend to focus on run-time capabilities (such as SOA run-time registries stuffed full of WSDL instead of knowledge management systems designed to guide the developer to the right service). The reasons for this may well be that software architects recognise that run-time problems (such as lack of scalability) can’t be got over as easily as development problems: Development problems will increase the cost and failure rate, but they won’t bring down the business.
- It is much harder to create good tools for less skilled developers than more code generation wizards that plug into eclipse that appeal to experts. Hence if you look at most SOA stacks, they will include forms-based user interfaces which help the already knowledgeable programmer with some routine tasks. They do not allow the newbie to focus on the business logic instead of the integration logic. Unfortunately, this point is somewhat obscured when people say they don’t need yet more new technology for SOA – they really mean they don’t need more new run-time technology. They certainly need more productive development tools.
- Middleware has put the premium price on run-time software and development tools have traditionally been sold for low prices or even given away. This may be because developers in early adopting companies tend to be more technically skilled and put lower value on easy-to-use tools. They also favour tools which augment their own skills – rather than tools which make it easy for somebody less skilful. What this means is that it is hard to fund or build a business selling really good development tools.
All of which leaves a significant challenge for SOA adopters and vendors. In order to be successful with SOA, significantly better development tools are needed. For this to happen, end-users need to push their vendors and be willing to pay them when the tools are delivered.
Ronan
p.s. If any readers want to point me at some good SOA development tools, I would be delighted to take a look.