Tag Archives: Service Oriented Architecture

Avoiding Big Data, and Big Data Integration Confusion

We discussed Big Data and Big Data integration last month, but the rise of Big Data and the systemic use of data integration approaches and technology continues to be a source of confusion.  As with any evolution of technology, assumptions are being made that could get many enterprises into a great deal of trouble as they move to Big Data.

Case in point: The rise of big data gave many people the impression that data integration is not needed when implementing big data technology.  The notion is, if we consolidate all of the data into a single cluster of servers, than the integration is systemic to the solution.  Not the case.

As you may recall, we made many of the same mistakes around the rise of service oriented architecture (SOA).  Don’t let history repeat itself with the rise of cloud computing.  Data integration, if anything, becomes more important as new technology is layered within the enterprise.

Hadoop’s storage approach leverages a distributed file system that maps data wherever it sits in a cluster.  This means that massive amounts of data reside in these clusters, and you can map and remap the data to any number of structures.  Moreover, you’re able to work with both structured and unstructured data.

As covered in a recent Read Write article, the movement to Big Data does indeed come with built-in business value.  “Hadoop, then, allows companies to store data much more cheaply. How much more cheaply? In 2012, Rainstor estimated that running a 75-node, 300TB Hadoop cluster would cost $1.05 million over three years. In 2008, Oracle sold a database with a little over half the storage (168TB) for $2.33 million – and that’s not including operating costs. Throw in the salary of an Oracle admin at around $95,000 per year, and you’re talking an operational cost of $2.62 million over three years – 2.5 times the cost, for just over half of the storage capacity.”

Thus, if these data points are indeed correct, Hadoop clearly enables companies to hold all of their data on a single cluster of servers.  Moreover, this data really has no fixed structure.  “Fixed assumptions don’t need to be made in advance. All data becomes equal and equally available, so business scenarios can be run with raw data at any time as needed, without limitation or assumption.”

While this process may look like data integration to some, the heavy lifting around supplying these clusters with data is always a data integration solution, leveraging the right enabling technology.  Indeed, consider what’s required around the movement to Big Data systems additional stress and you’ll realize why strain is placed upon the data integration solution.  A Big Data strategy that leverages Big Data technology increases, not decreases, the need for a solid data integration strategy and a sound data integration technology solution.

Big Data is a killer application that most enterprises should at least consider.  The business strategic benefits are crystal clear, and the movement around finally being able to see and analyze all of your business data in real time is underway for most of the Global 2000 and the government.  However, you won’t achieve these objectives without a sound approach to data integration, and a solid plan to leverage the right data integration technology.

Posted in Big Data, Data Integration, SOA, Uncategorized | Tagged , , , , , | Leave a comment

Does your Data Virtualization Solution Adapt To (Big) Data Problems

Late last year at a company all-hands meeting, the CEO of a large consumer electronics company had a serious mandate. He needed big results and fast.

“Competition is really heating up and customer churn is on the rise. I need visibility on-demand – a common view of all enterprise data or else we cannot continue to grow.”  IT teams across the enterprise have been scurrying and working furiously to create a common view of CUSTOMER, PRODUCT, SALES, and INVENTORY, but the results have been incomplete, inaccurate and too slow. This is no easy task. (more…)

Posted in Big Data, Business Impact / Benefits, CIO, Customer Acquisition & Retention, Data Integration, Data Services, Data Warehousing, Informatica 9.1, Real-Time, SOA | Tagged , , , | Leave a comment

To Successfully Service-Orient, Data-Orient First!

Informatica 9 I have firmly believed that a day would come when it would be you, my fellow integrators, telling me that one needs to data-orient first before benefiting from service-orientation. That day has indeed come!

Just recently, I created a quick one question survey and sent it off to a number of application and enterprise architects as well as IT managers at leading enterprises. The question was:

  • What are the top three things on your mind as you architect or re-architect your infrastructures?

There was a common thread across the responses that I received:

  • Service-orientation or an architectural approach to increase the speed and agility of how IT responds to a business’ requests,
  • Doing more with less or something to that effect, and
  • An easy way to leverage all relevant information, when it is needed and how it is needed

When I saw these responses, the pragmatic part of me started to build a list of questions in my head and I decided to call a number of these professionals and get the real scoop on their selections.

Here is what I heard…


Posted in Business Impact / Benefits, Data Integration Platform, Data Services, Real-Time, SOA | Tagged , , , , , | 1 Comment

‘Half-Baked Ideas’ About Mega-Changes Ahead in the Next Decade (Part 2)

The following are the remaining five “HBIs” (half-baked ideas) I am formulating to explore how SOA, cloud computing, Enterprise 2.0 and virtualization are shaping our information and business environments in the coming decade – the 2010s. These are based on a keynote speech I presented at the recent Cloud QCamp online event. (For HBIs 1-5, click here.)

HBI #6: Made to order — application vendors will become assemblers of made-to-order, pre-built software components. (more…)

Posted in Business Impact / Benefits, Cloud Computing, Data Integration, Data Services, Enterprise Data Management | Tagged | Leave a comment

‘Half-Baked Ideas’ for Mega-Changes Ahead in the Next Decade

I just delivered the keynote address for ebizQ’s latest Cloud QCamp, exploring the growing convergence of SOA with cloud computing, Enterprise 2.0 and virtualization, and I thought I would share some salient highlights here at the Perspectives site.

There are several key forces converging that are reshaping the way we will do business in the 2010s (you know, that next decade that is almost upon us). (more…)

Posted in Business Impact / Benefits, Cloud Computing, Data Governance, Data Integration, Data Services, Enterprise Data Management | Tagged , | 1 Comment

How About a ‘Service Oriented Architecture’ for Data?

The principles of service oriented architecture (SOA) show a lot of promise for building greater agility and integration into applications. So why not extend these capabilities to the data integration world as well? With initiatives such as data warehousing and master data management (MDM) coming to the fore, data integration needs to be addressed on an enterprise scale.

Madan Sheina, principal analyst within Ovum’s Software Applications group, proposes that a data architecture be designed along the same lines as SOA for applications – in what he calls process-driven data integration. (more…)

Posted in Data Integration, Data Services, Data Warehousing, Enterprise Data Management | Tagged , | 1 Comment

Service-Orientation is the New Mantra!

If you have been following the blog circles lately, there is a big buzz about SOA being dead. It all started with a recent blog post by Anne Thomas Manes in which she says “although the word ‘SOA’ is dead, the requirement for service-oriented architecture is stronger than ever.”

SOA at its very core is simply an architectural approach and not a technology stack nor a vendor-recommended product or platform. As Anne says, “they missed the important stuff: architecture and services.”

As I have always maintained, an SOA implementation can be as simple as a few business services that wrap business or application logic, and in its most complex form it can be an entire ecosystem of technologies selected based on thoroughly analyzing needs and that most importantly support service-orientation principles. (more…)

Posted in Business Impact / Benefits, Data Integration, Data Quality, Data Services, Operational Efficiency, Real-Time | Tagged , , , , , | Leave a comment

Webinar: Why Data Integration is Critical to SOA and BPM Success

In recent years, there has been plenty of attention on the dual approaches of service oriented architecture (SOA) and business process management (BPM) to help businesses realize greater agility. However, the data equation often gets lost amidst all the buzz and excitement over SOA and BPM. Process-driven data integration and the delivery of data services as part of SOA are critical to the success of these efforts.

On Tuesday, December 9th, Ash Parikh, Informatica’s resident real-time data integration and SOA expert — and a blogger here at the Perspectives community — will be joining Madan Sheina, principal analyst within Ovum’s Software Applications group, in a Webinar to discuss the urgency of data integration within emerging SOA and BPM environments. Beth Gold-Bernstein, my colleague at ebizQ, will moderate.

Click here to register for and access the Webinar, “Guaranteeing Agility in SOA and BPM with Process-Driven Data Integration.”  (December 9, 12:00 Noon Eastern)

As Ash and Madan will explain, most SOA and BPM efforts have centered on dealing with the integration of application silos and automating business processes at the application layer. With data being the backbone of these applications and business processes, many enterprise architects end up struggling with a number of data-centric issues and equipped with the wrong tools for the job. For example, there are issues caused by the inability to access diverse and fragmented data, by integrating enterprise data only to find out after-the-fact that the data is inaccurate and inconsistent, and by dealing with enterprise information that is typically delivered at various latencies.

Ash and Madan will also discuss how to leverage process-driven data integration, and make information-as-a-service a reality. These are approaches that can guarantee agility by enabling SOA and BPM with the seamless delivery of accurate, consistent and timely information.

Guaranteeing Agility in SOA and BPM with Process-Driven Data Integration”  (December 9, 12:00 Noon Eastern)

Posted in Business Impact / Benefits, Data Integration, Data Quality, Data Services, Data Warehousing, Enterprise Data Management, News & Announcements, Real-Time | Tagged , | Leave a comment

Where’s the Beef? Why SOA Needs MDM

Years, ago, I came across this question in an article in Boardroom Reports:  “What do you call a hamburger that’s 99% meat and 1% garbage?”

The answer was a “garbageburger.” In other words, even if a small fraction of the burger is tainted, the whole meal is tainted. The original analogy was being used to illustrate the challenges of time management, but it’s an apt analogy for data environments as well. That is, if a portion of the information is bad or unreliable, trust in all the data eventually breaks down. In essence, many implementations of service-oriented architecture (SOA) taking place across companies may be garbageburgers because they are serving up unreliable information – an element that has been out of the control of SOA designers.

Sorry if I ruined anyone’s lunch, but the point had to be made. (more…)

Posted in Data Integration, Data Quality, Data Services, Data Warehousing, Enterprise Data Management | Tagged , , | Leave a comment

How SOA Enhances Data Warehousing and Business Intelligence

“Despite certain rumors to the contrary, data warehousing is thriving.”

I couldn’t agree more with Judy Ko in her recent post, in which she points out that predictions that data warehousing was going to be abstracted away — by service-oriented architecture (SOA) and other new approaches — didn’t quite pan out. Instead, if anything, the need for data warehousing solutions only continues to grow. Data volumes are growing, and businesses are demanding ever-more sophisticated business intelligence and analytics to run against that data.

If anything, approaches such as SOA promise to greatly enhance – not replace – data warehousing, (more…)

Posted in Business Impact / Benefits, Customers, Data Integration, Data Services, Data Warehousing, Enterprise Data Management | Tagged , | 5 Comments