In a recent Aberdeen Group Analyst Insight paper it was identified that 50% of their survey respondents were currently integrating Hierarchical data sources with 13% planning to implement this capability in the next 12 months. But the changing trend is that of those organisations currently integrating XML data where nearly a third are using or are planning to integrate other Hierarchical sources with the need to integrate JSON coming out in the lead with COBOL records and Google Protocol Buffers close behind. Apache AVRO has not been integrated much currently but shows the biggest growth in planned integration and also number of projects.
The expected growth of data integration projects is 64% for internal XML sources and a whopping 133% for external XML sources from partner organisations. Non XML Hierarchical data sources growth are forecast at 68% for JSON, 41% for COBOL, 35% for Google Protocol Buffers and 375% for Apache AVRO in the coming year.
This research shows the increasing need to effectively manage Data Integration and Transformation projects. But what is driving this increase in Hierarchical data usage? Well, there seems to be three major areas that are driving this development.
Firstly the growth of XML-based formats is being pushed by industry and regulatory initiatives to report in XML. Many Government financial reporting requirements worldwide are now mandated to be in XBRL (eXtensible Business Reporting Language) format; also FpML (Financial Product Markup Language) which is used by the CFTC in the US and the ESMA in Europe to report on electronic dealing and processing of financial derivatives instruments.
The inclusion of Hierarchical data, such as web logs, is also growing as a new data source for advanced analytics. Increasingly data such as PDF Forms and Microsoft Office documents, which are now XML based, are also being included in the corporate Business Intelligence platform where once this huge amount of data was siloed in corporate operational systems.
And finally the evolution to Cloud based processing is generating increased demand on Hierarchical data format integration with many cloud API’s based on web services; also an XML format.
More Hierarchical data to manage and convert into usable formats for operational and increasingly analytical and reporting requirements; the problem is that with manual Data Integration and Transformation the process is frequently time-consuming and likely to introduce errors; this also does not enable any standardization of process and produces no learning and reuse of best practices.
So to maximize the benefit from Hierarchical data not just as an operational data format but also as a key source for business intelligence and reporting it is necessary to automate the transformation process as much as possible. Most organisations now want a single solution to manage simple to complex XML and hierarchical data integration projects such as Informatica’s XMap; with the benefits of this approach seen in automation, cost saving, faster, more reliable and more accurate handling of data, improved analysis and in better quality of information and decision-making.
Check our more XML resources at the Informatica XML Resource Center.