lustratus LiteBytes

Golden Source, a vendor of enterprise data management solutions for the financial services industry, has just released a report highlighting the major problems associated with the management of data connected with some classes of financial instruments (those known as Over The Counter or OTC derivatives).

In particular, it focuses on the strain placed on the existing data models by the need to roll-out new types of financial products ever more quickly.  The relevance to SOA and enterprise integration is two-fold:

• Data models are core to SOA, whether defined implicitly within services or explicitly as part of a separate data modelling exercise.  As the scope of SOA grows within an organisation, the challenge is to balance the need to create global models (and hence reduce the need to translate between models) and the need for different data models reflecting real differences between business units (and hence provide powerful transformation capabilities in the infrastructure).  In the case of business-to-business integration, as is the case with financial trading, there is a need for an agreed data model which is used to exchange messages between the two parties.  FpML is a leading example of such a B2B format which addresses derivatives in particular.

• As the level of integration increases, there are more messages flowing through the network and more complexity in data models and hence more complexity in the transformation between data models.  An objective of SOA is to deliver agility – to allow business to change unhampered by the traditionally slow moving integration capability.  Therefore as SOA grows, the data modelling and transformation challenges move centre stage.

The market in complex financial derivatives (bundled combinations of multiple types of financial instrument designed to have specific risk and return profiles) shows a potential future for SOA in terms of data complexity and rate of change.  However, it should be noted that it is unusual in a number of ways: as with all financial instruments, they are entirely virtual; with a fast moving and growing market, advantage is firmly with the organisation that can create and roll-out a new type of derivative and they are highly complex in their structure.  Nevertheless, the challenges faced in this problem domain should be indicative of the challenges that will be faced the most advanced SOA adopters.  The key findings of the Golden Source commissioned report were:

• Most of the manual data management effort goes into fixing the 1%-2% problem cases.   In the SOA context, this statistic should focus the mind and justify the business case for prioritising data modelling in any wide-scale SOA programme.
• Flexibility is essential in creating data models capable of changing as business changes and markets become more complex.  Data modelling is not about making a perfect representation of the business frozen in time – it is about making something sufficient for today which is built to change and extend as the business changes.
• While there is a clear need for industry wide collaboration in defining base models, the report notes scepticism that it will actually happen due to lack of incentives for each institution to share information.  While there are huge economic drivers towards standardisation within financial services, the natural suspicion and lack of trust between competitors has meant that the significant progress towards standards has taken a long time and has been painful process.  Therefore, it is reasonable to assume that most other industries will take a lot longer.  In the SOA context, this means that organisations should not wait for standards to emerge but rather focus back onto addressing their own needs and build data models in such a way to ensure easy integration to other organisation’s models.

Ronan