Category Archives: Cloud Data Integration

Cloud Data Integration

Embracing the Hybrid IT World through Cloud Integration

Embracing the Hybrid IT World through Cloud Integration

Embracing Hybrid IT through Cloud Integration

Being here at Oracle Open World, it’s hard not to think about Oracle’s broad scope in enterprise software and the huge influence it wields over our daily work. But even as all-encompassing as Oracle has become, the emergence of the cloud is making us equally reliant on a whole new class of complementary applications and services. During the early era of on-premise apps, different lines of businesses (LOBs) selected the leading application for CRM, ERP, HCM, and so on. In the cloud, it feels like we have come full circle to the point where best of breed cloud applications have been deployed across the enterprise, with the exception that the data models, services and operations are not under our direct control. As a result, Hybrid IT, and the ability to integrate major on-premises applications such as Oracle E-Business, PeopleSoft, and Siebel, to name a few with cloud applications such as Oracle Cloud Applications, Salesforce, Workday, Marketo, SAP Cloud Applications, and Microsoft Cloud Apps, has become one of businesses’ greatest imperatives and challenges.

With Informatica Cloud, we’ve long tracked the growth of the various cloud apps and its adoption in the enterprise. Common business patterns – such as opportunity-to-order, employee onboarding, data migration and business intelligence – that once took place solely on-premises are now being conducted both in the cloud and on-premises.

The fact is that we are well on our way to a world where our business needs are best met by a mix of on-premises and cloud applications. Regardless of what we do or make, we can no longer get away with just on-premises applications – or at least not for long.  As we become more reliant on cloud services, such as those offered by Oracle, Salesforce, SAP, NetSuite, Workday, we are embracing the reality of a new hybrid world, and the imperative for simpler integration it demands.

So, as the ground shifts beneath us, moving us toward the hybrid world, we, as business and IT users, are left standing with a choice: Continue to seek solutions in our existing on-premises integration stacks, or go beyond, to find them with the newer and simpler cloud solution. Let us briefly look at five business patterns we’ve been tracking.

One of the first things we’ve noticed with the hybrid environment is the incredible frequency with which data is moved back and forth between the on-premises and cloud environments. We call this the data integration pattern, and it is best represented by getting data, such as price list or inventory from Oracle E-Business into a cloud app so that the actual user of the cloud app can view the most updated information. Here the data (usually master data) is copied toserves a certain purpose. Data Integration also involves the typical needs of data to be transformed before it can be inserted or updated. The understanding of metadata and data models of the involved applications is key to do this effectively and repeatedly.

The second is the application integration pattern, or the real time transaction flow between your on-premises and cloud environment, where you have business processes and services that need to communicate with one another. Here, the data needs to be referenced in real time for a knowledge worker to take action.

The third, data warehousing in the cloud, is an emerging pattern that is gaining importance for both mid- and large-size companies. In this pattern, businesses are moving massive amounts of data in bulk from both on-premises and cloud sources into a cloud data warehouse, such as Amazon Redshift, for BI analysis.

The fourth, the Internet of Things (IOT) pattern, is also emerging and is becoming more important, especially as new technologies and products, such as Nest, enable us to push streaming data (sensor data, web logs, etc.) and combine them with other cloud and on-premises data sources into a cloud data store. Often the data is unstructured and hence it is critical for an integration platform to effectively deal with unstructured data.

The fifth and final pattern, API integration, is gaining prominence in the cloud. Here, an on-premise or cloud application exposes the data or service as an external API that can be consumed directly by applications or by a higher-level composite app in an orchestration.

While there are certainly different approaches to the challenges brought by Hybrid IT, cloud integration is often best-suited to solving them.

Here’s why.

First, while the integration problems are more or less similar to the on-premise world, the patterns now overlap between cloud and on-premise. Second, integration responsibility is now picked up at the edge, closer to the users, whom we call “citizen integrators”. Third, time to market and agility demands that any integration platform you work with can live up to your expectations of speed. There are no longer multiyear integration initiatives in the era of the cloud. Finally, the same values that made cloud application adoption attractive (such as time-to-value, manageability, low operational overhead) also apply to cloud integration.

One of the most important forces driving cloud adoption is the need for companies to put more power into hands of the business user.  These users often need to access data in other systems and they are quite comfortable going through the motions of doing so without actually being aware that they are performing integration. We call this class of users ‘Citizen Integrators’. For example, if a user uploads an excel file to Salesforce, it’s not something they would call as “integration”. It is an out-of-the-box action that is integrated with their user experience and is simple to use from a tooling point of view and oftentimes native within the application they are working with.

Cloud Integration Convergence is driving many integration use cases. The most common integration – such as employee onboarding – can span multiple integration patterns. It involves data integration, application integration and often data warehousing for business intelligence. If we agree that doing this in the cloud makes sense, the question is whether you need three different integration stacks in the cloud for each integration pattern. And even if you have three different stacks, what if an integration flow involves the comingling of multiple patterns? What we are noticing is a single Cloud Integration platform to address more and more of these use cases and also providing the tooling for both a Citizen Integrator as well as an experienced Integration Developer.

The bottom line is that in the new hybrid world we are seeing a convergence, where the industry is moving towards streamlined and lighter weight solutions that can handle multiple patterns with one platform.

The concept of Cloud Integration Convergence is an important one and we have built its imperatives into our products. With our cloud integration platform, we combine the ability to handle any integration pattern with an easy-to-use interface that empowers citizen integrators, and frees integration developers for more rigorous projects. And because we’re Informatica, we’ve designed it to work in tandem with PowerCenter, which means anything you’ve developed for PowerCenter can be leveraged for Informatica Cloud and vice versa thereby fulfilling Informatica’s promise of Map Once, Deploy Anywhere.

In closing, I invite you to visit us at the Informatica booth at Oracle Open World in booth #3512 in Moscone West. I’ll be there with some of my colleagues, and we would be happy to meet and talk with you about your experiences and challenges with the new Hybrid IT world.

FacebookTwitterLinkedInEmailPrintShare
Posted in Cloud, Cloud Application Integration, Cloud Computing, Cloud Data Integration | Tagged , , , , , , | Leave a comment

Government Cloud Data Integration: Some Helpful Advice

Government Cloud Data Integration

Government Cloud Data Integration

Recently, a study found that the Government Cloud Data Integration has not been extremely effective. This post will provide some helpful advice.

As covered in Loraine Lawson’s blog, MeriTalk surveyed federal government IT professionals about their use of cloud computing. As it turns out, “89 percent out of 153 surveyed expressed ‘some apprehension about losing control of their IT services,’ according to MeriTalk.”

Loraine and I agree that what the survey says about the government’s data integration, management, and governance, is that they don’t seem to be very good at cloud data management…yet. Some of the other gruesome details include:

  • 61 percent do not have quality, documented metadata.
  • 52 percent do not have well understood data integration processes.
  • 50 percent have not identified data owners.
  • 49 percent do not have known systems of record.

“Overall, respondents did not express confidence about the success of their data governance and management efforts, with 41 percent saying their data integration management efforts were some degree of ‘not successful.’ This lead MeriTalk to conclude, ‘Data integration and remediation need work.’”

The problem with the government is that data integration, data governance, data management, and even data security have not been priorities. The government has a huge amount of data to manage, and they have not taken the necessary steps to adopt the best practices and technology that would allow them to manage it properly.

Now that everyone is moving to the cloud, the government included, questions are popping up about the proper way to manage data within the government, from the traditional government enterprises to the public cloud. Clearly, there is much work to be done to get the government ready for the cloud, or even ready for emerging best practices around data management and data integration.

If the government is to move in the right direction, they must first come to terms with the data. This means understanding where the data is, what it does, who owns it, access mechanisms, security, governance, etc., and apply this understanding holistically to most of the data under management.

The problem within the government is that the data is so complex, distributed, and, in many cases, unique, that it’s difficult for the government to keep good track of the data. Moreover, the way the government does procurement, typically in silos, leads to a much larger data integration problem. I was working with government agencies that had over 5,000 siloed systems, each with their own database or databases, and most do not leverage data integration technology to exchange data.

There are ad-hoc data integration approaches and some technology in place, but nowhere close to what’s need to support the amount and complexity of data. Now that government agencies are looking to move to the cloud, the issues around data management are beginning to be better understood.

So, what’s the government to do? This is a huge issue that can’t be fixed overnight. There should be incremental changes that occur over the next several years. This also means allocating more resources to data management and data integration than has been allocated in the past, and moving it much higher up in the priorities lists.

These are not insurmountable problems. However, they require a great deal of focus before things will get better. The movement to the cloud seems to be providing that focus.

FacebookTwitterLinkedInEmailPrintShare
Posted in Cloud Data Integration | Tagged , | Leave a comment

Elevating Your App by Embedding Advanced, REST-ful Cloud Integration

Cloud Integration

Elevating Your App by Embedding Advanced, REST-ful Cloud Integration

While the great migration to the cloud has created opportunities for practically everyone in tech, no one group stands to benefit from it more than application developers. In partnership with category-leading SaaS companies like Salesforce, NetSuite, Ultimate Software and others, ISVs have not only easy access to open APIs to leverage but also complete marketplaces to establish themselves and exploit for market and mind share. And within those ecosystems, there stands a whole new class of users looking to these applications to solve their specific business problems.

But, as Billy Macinnes, in his July MicroScope article, reminds us, the opportunities come with many challenges, and so far only a few ISVs have risen high enough to truly meet them all. While the article itself is more concerned about where ISVs are headed, Macinnes and the industry experts he references, such as Mike West and Philip Howard, make it clear that no one is going anywhere far without a cloud strategy that meaningfully addresses data integration.

As a business app consumer myself, I too am excited by the possibilities that exist. I am intrigued by the way in which the new applications embrace the user-first ethos and deliver consumer-app-like interfaces and visual experiences. What concerns me is what happens next, once you get beyond the pretty design and actually try to solve the business use case for which the app is intended. This, unfortunately, is where many business apps fail. While most can access data from a single specific application, few can successfully interact with external data coming from multiple sources.

Like many of the challenges (such as licensing and provisioning) faced by today’s ISVs, data integration is something that lies outside of the expertise area of a typical app developer. Let’s say, for example, you’ve just come up with a new way to anticipate customer needs and match it with excess inventory. While the developer expertise and art of the app may be, say, in a new algorithm, the user experience, ultimately, is equally dependent on your ability to surface data – inventory, pricing, SKU numbers, etc. – that may be held in SaaS and on-premises systems and seamlessly marry it – behind the scenes – to cloud-based customer information.

The bottom line is that regardless of the genius behind your idea or user interface, if you can’t feed relevant data into your application and ensure its completeness, quality and security for meaningful consumption, your app will be dead in the water. As a result, app developers are spending an inordinate amount of time – in some cases up to 80% of their development cycle – working through data issues. Even with that, many still get stuck and end up with little more for their effort than a hard lesson in the difficulties of enterprise data integration long understood by every integration developer.

Fortunately, there is a better way: cloud integration.

Cloud integration enables the developer to focus on their app and core business. The ISV can offer cloud integration to its customers as an external resource or as an embedded part of its app. While some may see this as a choice, any ISV looking to provide the best possible user experience has no real option other than to embed the integration services as part of their application.

Look at any successful business app, and chances are you’ll find something that empowers users to work independently, without having to rely on other teams or tools for solutions. Take, for example, the common use case of bringing data into an app via a CSV file. With integration built directly into the app, the user can upload the file and resolve any semantic conflicts herself, with no assistance from IT. Without it, the user is now reliant on others to do his or her job, and ultimately less productive. Clearly, the better experience is the one that provides users with easy access to everything needed – including data from multiple sources – to get the work done themselves. And the most effective way you can do that is by embedding integration into the application.

Now that we’ve settled why cloud integration works best as an embedded capability, let’s take a closer look at how it works within the application context.

With cloud integration embedded into your app, you can essentially work behind the scenes to connect different data sources and incorporate the mapping and workflows between your app and the universe of enterprise data sources. How it accomplishes that is through abstraction. By abstracting connectivity to these data sources, you take the complexities involved with bringing data from an external source – such as SAP or Salesforce – and place it within a well-defined integration template or Vibe Integration Package (VIP). Once these abstractions are defined, you can then, as an application developer, access these templates through REST API and bring the specified data into your application.

While connectivity abstraction and REST APIs are important on their own, like all great pairings, it is only in combination that their true utility is realized. In fact, taken separately, neither is of much value to the application developer. Alone, a REST API can access the raw data type, but without the abstraction, the information is too unintelligible and incomplete to be of any use. And without the REST API, the abstracted data has no way of getting from the source to the application.

The value that REST APIs together with connectivity abstraction bring cannot be overstated, especially when the connectivity can span multiple integration templates. The mechanism for accomplishing integration is, like an automobile transmission, incredibly complex. To give an analogy, just like a car’s shift lever exposes a simple interface to move the gears from Park to Drive, activating a series of complex sensors to make the appropriate motions under the hood, the integration templates allow the user to work with the data in any way they want without ever having to understand or know about the complexities going on underneath.

As the leading cloud integration solution and platform, Informatica Cloud has long recognized the importance of pairing REST APIs and connectivity abstraction.

The first and most important function within our REST API is administration. It enables you to set up your organization and the administration of your users and permissions. The second function allows you to run and monitor integration tasks. And with the third, end users can configure the integration templates themselves, and enforce the business rules to apply for their specific process. You can view the entire set of Informatica Cloud REST API capabilities here.

It is in this last area – integration configurability – where we are truly setting ourselves apart. The Vibe Integration Packages (VIPs) not only abstract backend connectivity but also ensure that the data is complete – with the needed attributes from the underlying apps – and is of high quality and formatted for easy consumption in the end-user application. With the Packages, we’ve put together many of the most common integrations with reusable integration logic that is configurable through a variety of parameters. Our configurable templates enable your app users to customize and fine-tune their integrations – with custom fields, objects, etc. – to meet the specific behavior and functionality of their integrations. For example, the Salesforce to SAP VIP includes all the integration templates you need to solve different business use cases, such as integrating product, order and account information.

With their reusability and groupings encompassing many of the common integration use cases, our Vibe Integration Packages really are revolutionizing work for everyone. Using Informatica Cloud’s Visual Designer, developers can quickly create new, reusable VIPs, with parameterized values, for business users to consume. And SaaS administrators and business analysts can perform complex business integrations in a fraction of the time it took previously, and customize new integrations on the fly, without IT’s help.

More and more, developers are building great-looking apps with even greater aspirations. In many cases, the only thing holding them back is the ability to access back-office data without using external tools and interfaces, or outside assistance. With Informatica Cloud, data integration need no longer take a backseat to design, or anything else. Through our REST API, abstractions and Vibe Integration Packages, we help developers put an end to the compromise on user experience by bringing in the data directly through the application – for the benefit of everyone.

FacebookTwitterLinkedInEmailPrintShare
Posted in Cloud Computing, Cloud Data Integration | Tagged , | Leave a comment

A Guide for Using Cloud Integration to Load Data into Amazon Redshift: Establishing Connections

Amazon Redshift, one of the fast-rising stars in the AWS ecosystem has taken the data warehousing world by storm ever since it was introduced almost two years ago. Amazon Redshift operates completely in the cloud, and allows you to provision nodes on-demand. This model allows you to overcome many of the pains associated with traditional data warehousing techniques, such as provisioning extra server hardware, sizing and preparing databases for loading or extensive SQL scripting.

However, when loading data into Redshift, you may find it challenging to do so in a timely manner. To reduce the time taken to load this data, you may have to spend a tremendous amount of time writing SQL optimization queries which takes away the value proposition of using Redshift in the first place.

Informatica Cloud helps you load this data quickly into Redshift in just a few minutes. To start using Informatica Cloud, you’ll need to establish connections from Redshift and your other data source first. Here are a few easy steps to help you get started with establishing connections from a relational database such as MySQL as well as Redshift into Informatica Cloud:

  1. Login into your Informatica Cloud account, go to Configure -> Connections, click “New”, and select “MySQL” for “Type”
  2. Select your Secure Agent and fill in the rest of the database details:
    Amazon Redshift Step 2
  3. Test your connection and then click ‘OK’ to save and exit
    Amazon Redshift
  4. Now, login to your AWS account and go to Redshift service page
  5. Go to your cluster configuration page  and make a note of the cluster and cluster database properties: Number of Nodes, Endpoint, Port, Database Name, JDBC URL. You also will need:
    • The Redshift database user name and password (which is  different from your AWS account)
    • AWS account Access Key
    • AWS account Secret Key
  6. Exit the AWS console.
  7. Now, back in your Informatica Cloud account, go to Configure -> Connections and click “New”.
  8. Select “AWS Redshift (Informatica)” for “Type” and fill in the rest of the details from the information you have from above
    Redshift step 3
  9. Test the connection and then click ‘OK’ to save and exit
    Redshift Step 4

As you can see, establishing connections was extremely easy and can be done in less than 5 minutes. To learn how customers such as UBM used Informatica Cloud to deliver next-generation customer insights with Amazon Redshift, please join us on September 16 for a webinar where we’ll have product experts from Amazon and UBM explaining how your company can benefit from cloud data warehousing for petabyte-scale analytics using Amazon Redshift.

FacebookTwitterLinkedInEmailPrintShare
Posted in Cloud, Cloud Data Integration, Data Integration, Data Integration Platform | Tagged , , | Leave a comment

Making the Hybrid Cloud Work for Public Sector

Making the Hybrid Cloud Work for Public Sector

Hybrid Cloud and Public Sector

If you’ve been working in the government sector for any amount of time, you had to see the advent of the “hybrid cloud” coming. Like all new technologies, when first introduced, “the cloud” was the answer to all your IT woes. It is cheaper, more reliable, infinitely scalable, instantly adaptable, and so on. But, as time has gone by and many of you have dipped your toes in the water, the reality is beginning to surface, and challenges are beginning to appear. Sure, moving email to the cloud was a great first step, and it certainly gave most agencies the ability to show progress in leveraging the cloud. Yes, archiving data to the cloud is also a good use case and is showing progress. But, what’s next? There are plenty of new SaaS offerings popping up, and purpose-built to solve various public sector challenges, and yes, they are generally decent applications. Yet, would it be fair to suggest new challenges are arising as your agency begins to adopt new cloud solutions? In particular, has the advent of specialized applications for government made your overall IT portfolio simpler or more complex? Government has always struggled with a vast array of siloed systems and isn’t the cloud creating yet more challenges in this regard? Well, maybe. Let’s take a look.

What I love about the cloud is it has something of value to offer practically any government organization, regardless of size, maturity, point of view, approach. Even for the most conservative IT shops, there are use cases that just plain make sense. And with the growing availability of FEDRAMP certified offerings, it’s becoming easier to procure. But, thinking realistically, for reasons of law, budget, time, architecture, we know the cloud will not be the solution for every public sector problem. Some applications, some data will never leave your agency’s premises. And here in lies the new complexity. You have applications and data on-prem. You have applications and data in the cloud. And you have business requirements that require these apps to work together, to share data.

So, now that you have a hybrid environment, what can you do about? Let’s face it, we can talk about technology, architecture and approaches all day long, but, it always comes down to this, what should be done with the data. You need answers to questions such as; Is it safe? Is it accessible? It is reliable? How do I know if the integrity has been compromised? What about the quality? How error-prone is the data? How complete is the data? How do we manage it across this new hybrid landscape? How can I get data from a public cloud application to my on-prem data warehouse? How can I leverage the flexibility of public IaaS to build a new application that will need access to data that is also required for an on-prem legacy application?

I know many government IT professional are wrestling with these questions and seeking solutions. So, here’s an interesting thought. Most of these questions are not exactly new, they are just taking on the added context of the cloud. Prior to the cloud, many agencies discovered answers in the form of a data integration platform. The platform is used to ensure every application, every user has access to the data they need to perform their mission or job. I think of it this way. The platform is a “standardized” abstraction layer that ensures all your data gets to where it needs to be, when it needs to be there, in the form it needs to be in. There are hundreds of government IT shops using such an approach.

Here’s the good news. This approach to integrating data can be extended to include the cloud.  Imagine placing “agents” in all the places where your data needs to live, the agents capable of communicating with each other to integrate, alter or move data. Now add to this the idea of a cloud-based remote control that allows you to control all the functions of the agents. Using such a platform now enables your agency to tie on-prem systems to cloud systems, minimizing the effect of having multiple silos of information. Now government workers and warfighters will have the ability to more quickly get complete, accurate data, regardless of where it originates and citizens will benefit from more effectively delivered services.

How would such an approach change your ideas on how to leverage the cloud for your agency? If you live near the Washington, DC area, you may wish to drop in on the Government Cloud Computing and Data Center Conference & Expo. One of my colleagues, Ronen Schwartz will be discussing this topic. For those not in the vicinity, you can learn more here.

FacebookTwitterLinkedInEmailPrintShare
Posted in Cloud, Cloud Application Integration, Cloud Data Integration, Cloud Data Management, Public Sector | Tagged , , | Leave a comment

Informatica Cloud Summer ’14 Release Breaks Down Barriers with Unified Data Integration and Application Integration for Real Time and Bulk Patterns

This past week, Informatica Cloud marked an important milestone with the Summer 2014 release of the Informatica Cloud platform. This was the 20th Cloud release, and I am extremely proud of what our team has accomplished.

“SDL’s vision is to help our customers use data insights to create meaningful experiences, regardless of where or how the engagement occurs. It’s multilingual, multichannel and on a global scale. Being able to deliver the right information at the right time to the right customer with Informatica Cloud Summer 2014 is critical to our business and will continue to set us apart from our competition.”

– Paul Harris, Global Business Applications Director, SDL Pic

When I joined Informatica Cloud, I knew that it had the broadest cloud integration portfolio in the marketplace: leading data integration and analytic capabilities for bulk integration, comprehensive cloud master data management and test data management, and over a hundred connectors for cloud apps, enterprise systems and legacy data sources.. all delivered in a self-service design with point-and-click wizards for citizen integrators, without the need for complex and costly manual custom coding.

But, I also learned that our broad portfolio belies another structural advantage: because of Informatica Cloud’s unique, unified platform architecture, it has the ability to surface application (or real time) integration capabilities alongside its data integration capabilities with shared metadata across real time and batch workflows.

With the Summer 2014 release, we’ve brought our application integration capabilities to the forefront. We now provide the most-complete cloud app integration capability in the marketplace. With a design environment that’s meant not for just developers but also line of business IT, now app admins can also build real time process workflows that cut across on-premise and cloud and include built-in human workflows. And with the capability to translate these process workflows instantly into mobile apps for iPhone and Android mobile devices, we’re not just setting ourselves apart but also giving customers the unique capabilities they need for their increasingly mobile employees.

InformaticaCloud

Informatica Cloud Summer Release Webinar Replay

“Schneider’s strategic initiative to improve front-office performance relied on recording and measuring sales person engagement in real time on any mobile device or desktop. The enhanced real time cloud application integration features of Informatica Cloud Summer 2014 makes it all possible and was key to the success of a highly visible and transformative initiative.”

– Mark Nardella, Global Sales Process Director, Schneider Electric SE

With this release, we’re also giving customers the ability to create workflows around data sharing that mix and match batch and real time integration patterns. This is really important.  Because unlike the past, where you had to choose between batch and real time, in today’s world of on-premise, cloud-based, transactional and social data, you’re now more than ever having to deal with both real time interactions and the processing of large volumes of data. For example, let’s surmise a typical scenario these days at high-end retail stores. Using a clienteling iPad app, the sales rep looks up bulk purchase history and inventory availability data in SAP, confirms availability and delivery date, and then processes the customer’s order via real time integration with NetSuite. And if you ask any customer, having a single workflow to unify all of that for instant and actionable insights is a huge advantage.

“Our industry demands absolute efficiency, speed and trust when dealing with financial information, and the new cloud application integration feature in the latest release of Informatica Cloud will help us service our customers more effectively by delivering the data they require in a timely fashion. Keeping call-times to a minimum and improving customer satisfaction in real time.”

– Kimberly Jansen, Director CRM, Misys PLC

We’ve also included some exciting new Vibe Integration packages or VIPs. VIPs deliver pre-built business process mappings between front-office and back-office applications. The Summer 2014 release includes new bidirectional VIPs for Siebel to Salesforce and SAP to Salesforce that make it easier for customers to connect their Salesforce with these mission-critical business applications.

And lastly, but not least importantly, the release includes a critical upgrade to our API Framework that provides the Informatica Cloud iPaaS end-to-end support for connectivity to any company’s internal or external APIs. With the newly available API creation, definition and consumption patterns, developers or citizen integrators can now easily expose integrations as APIs and users can consume them via integration workflows or apps, without the need for any additional custom code.

The features and capabilities released this summer are available to all existing Informatica Cloud customers, and everyone else through our free 30-day trial offer.

FacebookTwitterLinkedInEmailPrintShare
Posted in Big Data, Cloud, Cloud Application Integration, Cloud Computing, Cloud Data Integration, Data Integration, Uncategorized | Tagged , , , , , | Leave a comment

Moving to the Cloud: 3 Data Integration Facts That Every Enterprise Should Understand

Cloud Data Integration

Cloud Data Integration

According to a survey conducted by Dimensional Research and commissioned by Host Analytics, “CIOs continue to grow more and more bullish about cloud solutions, with a whopping 92% saying that cloud provides business benefits, according to a recent survey. Nonetheless, IT execs remain concerned over how to avoid SaaS-based data silos.”

Since the survey was published, many enterprises have, indeed, leveraged the cloud to host business data in both IaaS and SaaS incarnations.  Overall, there seems to be two types of enterprises: First are the enterprises that get the value of data integration.  They leverage the value of cloud-based systems, and do not create additional data silos.  Second are the enterprises that build cloud-based data silos without a sound data integration strategy, and thus take a few steps backward, in terms of effectively leveraging enterprise data.

There are facts about data integration that most in enterprise IT don’t yet understand, and the use of cloud-based resources actually makes things worse.  The shame of it all is that, with a bit of work and some investment, the value should come back to the enterprises 10 to 20 times over.  Let’s consider the facts.

Fact 1: Implement new systems, such as those being stood up on public cloud platforms, and any data integration investment comes back 10 to 20 fold.  The focus is typically too much on cost and not enough on the benefit, when building a data integration strategy and investing in data integration technology.

Many in enterprise IT point out that their problem domain is unique, and thus their circumstances need special consideration.  While I always perform domain-specific calculations, the patterns of value typically remain the same.  You should determine the metrics that are right for your enterprise, but the positive values will be fairly consistent, with some varying degrees.

Fact 2: It’s not just about data moving from place-to-place, it’s also about the proper management of data.  This includes a central understanding of data semantics (metadata), and a place to manage a “single version of the truth” when it comes to dealing massive amounts of distributed data that enterprises must typically manage, and now they are also distributed within public clouds.

Most of those who manage enterprise data, cloud or no-cloud, have no common mechanism to deal with the meaning of the data, or even the physical location of the data.  While data integration is about moving data from place to place to support core business processes, it should come with a way to manage the data as well.  This means understanding, protecting, governing, and leveraging the enterprise data, both locally and within public cloud providers.

Fact 3: Some data belongs on clouds, and some data belongs in the enterprise.  Those in enterprise IT have either pushed back on cloud computing, stating that data outside the firewall is a bad idea due to security, performance, legal issues…you name it.  Others try to move all data to the cloud.  The point of value is somewhere in between.

The fact of the matter is that the public cloud is not the right fit for all data.  Enterprise IT must carefully consider the tradeoff between cloud-based and in-house, including performance, security, compliance, etc..  Finding the best location for the data is the same problem we’ve dealt with for years.  Now we have cloud computing as an option.  Work from your requirements to the target platform, and you’ll find what I’ve found: Cloud is a fit some of the time, but not all of the time.

 

FacebookTwitterLinkedInEmailPrintShare
Posted in Cloud, Cloud Application Integration, Cloud Computing, Cloud Data Integration, Data Integration | Tagged , , | Leave a comment

3 Ways to Simplify SAP Connectivity Protocols

Simplify SAP Connectivity Protocols

Simplify SAP Connectivity Protocols

SAP’s Jam social platform has generated a great deal of buzz since its release last May – and for good reason. As detailed by Alan Lepofsky in his coverage for ZDNet, the new Jam reboot included the kind of things, such as out-of-the-box integration, workflow templates and simplified developer tools, that make both IT and business users very happy.

However, the complexity of SAP’s Business Suite (or ECC as it’s called) does not easily lend itself to integration with other applications. The code underlying it is built on a proprietary language called ABAP – a combination of COBOL and Open SQL with some object-oriented features– that requires specialized knowledge and skill not easily found outside of the SAP ecosystem. Up until now, the typical integration project required the involvement of a specialized SAP consultant to develop custom ABAP code or map complex BAPI/IDoc structures as well as a BASIS administrator to transport the ABAP code from development to QA to production. The result was expensive and manually intensive. Integration projects took a few months or longer to complete and were not agile enough to handle ongoing requirements or even field changes.

Today, Informatica Cloud offers business a more innovative approach to SAP data extraction – ultimately, promoting agile development and enabling rapid deployment with the following three important features.

Automatically Generating ABAP Code

At the core of Informatica’s solution is the Cloud Connector for SAP. While the face of the Connector is a simple, wizard-based, drag-and-drop interface, under the hood it uses a Remote Function Call (RFC) to dynamically generate ABAP code (based on user choices) to connect with SAP and access the data through the application layer.

Drag and Drop Design Palette

The wizard guides the user through the steps necessary to extract the data from SAP and send it to any application where it is needed. Using Informatica Cloud’s drag-and-drop design palette, one can simply choose SAP – like any other application endpoint – and select what is needed to connect to the target, without ever having write specialized code.

Because of the dynamically generated ABAP code, SaaS application administrators trying to connect to SAP don’t have to deal with the SAP transports (and the lengthy development cycle) for each extract, reducing the time from months to weeks for an individual project, as well as the load on the BASIS administrators. The increased agility enables end users to respond to business demands by acquiring related data extracts, field and feature changes and/or additions – in near real time. Informatica also reduces the load on both the admin and the server – even further by eliminating the need for transports and sending the data in packets, and by running the extracts in the background. And since no data is staged or buffered on the SAP server, there is never a risk of compromising the system’s or online users’ performance.

Speedy Development Through Vibe Integration Packages

Informatica’s solution also includes a technology bundle to speed up development time and reduce the user’s learning curve. The bundle, or Informatica Vibe integration package, consists of downloadable templates that help the user to understand and use the complex SAP interfaces. Future roadmap releases will contain resources for additional SAS endpoints.

In his review mentioned above in the opening, Lepofsky notes the importance of integration and the partner ecosystem to the Jam platform. The same can be said of the SAP Business Suite and the specialized LOB cloud apps that orbit it. Without ready and real-time access to SAP’s data, even the most feature-rich app is of little use to anyone. With Informatica’s Cloud Connector for SAP, business users, like Informatica customer Addivant, now have a simple and efficient way to solve the most pressing problems presented by SAP to cloud app integration.

FacebookTwitterLinkedInEmailPrintShare
Posted in Cloud, Cloud Application Integration, Cloud Data Integration, Cloud Data Management | Leave a comment

Bulletproof Tips to Optimize Data Transformations for Salesforce

Optimize Data Transformations for Salesforce

Optimize Data Transformations for Salesforce

In the journey from a single-purpose cloud CRM app to the behemoth that it is today, Salesforce has made many smart acquisitions. However, the recent purchase of RelateIQ may have just been its most ingenious. Although a relatively small startup, RelateIQ has gained a big reputation for its innovative use of data science and predictive analytics, which would be highly beneficial to Salesforce customers.

As relevant from the acquisition, there is little doubt that the cloud application world is making a tectonic shift to data science and the appetite for Big Data to be pieced together to fuel the highly desired 360-degree view is only growing stronger.

But while looking ahead is certainly important, those of us who live in the present have much work yet still to accomplish in the here and now. For many, that means figuring out the best way to leverage data integration strategies and the Salesforce platform to gain actionable intelligence for our sales, marketing and CRM projects – today. Up until recently, this has involved manual, IT-heavy processes.

We need look no further than three common use cases where typical data integration strategies and technologies fail today’s business users:

Automated Metadata Discovery
The first, and perhaps most frustrating, has to do with discovering related objects. For example, objects, such as Accounts, don’t exist in a vacuum. They have related objects such as Contacts that can provide the business user – be it a salesperson or customer service manager, with context of the customer.

Now, during the typical data-integration process, these related objects are obscured from view because most integration technologies in the market today cannot automatically recognize the object metadata that could potentially relate all of the customer data together. The result is an incomplete view of the customer in Salesforce, and a lost opportunity to leverage the platform’s capability to strengthen a relationship or close a deal.

The Informatica Cloud platform is engineered ground up to be aware of the application ecosystem API and understand its metadata. As a result, our mapping engine can automatically discover metadata and relate objects to one another. This automated metadata discovery gives business users the ability to discover, choose and bring all of the related objects together into one mapping flow. Now, with a just a few clicks, business users, can quickly piece together relevant information in Salesforce and take the appropriate action.

Bulk Preparation of Data
The second instance where most data integration solutions typically fall short is with respect to the bulk preparation of data for analytic runs prior to the data transformation process. With the majority of Line-of-Business (LOB) applications now being run in the cloud, the typical business user has multiple terabytes of cloud data that either need to be warehoused on-premise or in the cloud, for a BI app to perform its analytics.

As a result, the best practice for bringing in data for analytics requires the ability to select and aggregate multiple data records from multiple data sources into a single batch, to speed up transformation and loading and – ultimately – intelligence gathering. Unfortunately, advanced transformations such as aggregations are something that is simply beyond the capabilities of most other integration technologies. Instead, most bring the data in one record or message at time, inhibiting speedy data loading and delivery of critical intelligence to business users when they need it most.

Alternatively, Informatica has leveraged its intellectual property in the market-leading data integration product, PowerCenter, and developed data aggregation transformation functionality within Informatica Cloud. This enables business users to pick a select group of important data points from multiple data sources – for example, country or dollar size of revenue opportunities – to aggregate and quickly process huge volumes in a single batch.

In-App Business Accelerators
Similar to what we’ve experienced with the mobile iOS and Android platforms, there recently has been an explosion of new, single-purpose business applications being developed and released for the cloud. While the platform-specific mobile model is predicated on and has flourished because of the “build it and they will come” premise, the paradigm does not work in the same way for cloud-based applications that are trying to become a platform.

The reason for this is that, unlike with iOS and Android platforms, in the cloud world, business users have a main LOB application that they are familiar with and rely on packaged integrations to bring in from other LOB cloud applications. However, with Salesforce extending its reach to become a cloud platform, the center of data gravity is shifting towards it being used for more than just CRM and the success of this depends upon these packaged integrations. Up until now, these integrations have just consisted of sample code and have been incredibly complex (and IT-heavy) to build and very cumbersome to use. As a result, business users lack the agility to easily customize fields or their workflows to match their unique business processes. Ultimately, the packages that were intended to accelerate business ended up inhibiting it.

Informatica Cloud’s Vibe Integration Packages (or VIPs) have made the promise of the integration package as a business accelerator into a reality, for all business users. Unlike sample code, VIPs are sophisticated templates that encapsulate the intelligence or logic of how you integrate the data between apps. While VIPs abstract complexity to give users out-of-the box integration, their pre-built mapping also provides great flexibility. Now, with just a few keystrokes, business users can map custom fields or leverage their unique business model into their Salesforce workflows.

A few paragraphs back I began this discussion with the recent acquisition of RelateIQ by Salesforce. While we can make an educated guess as to what that will bring in the future, no one knows for sure. What we do know is that, at present, Salesforce’s completeness as a platform – and source of meaningful analytics – requires something in addition to run your relevant business applications and solutions through it. A complete iPaaS (Integration Platform as a Service) solution such as Informatica Cloud has the power to make the vision into reality. Informatica enables this through the meta-data rich platform for data discovery, Industry leading data and application integration capabilities and business accelerators that put the power back in the hands of citizen application integrators and business users.

Join our webinar August 14 to learn more: Informatica Cloud Summer 2014: The Most Complete iPaaS

FacebookTwitterLinkedInEmailPrintShare
Posted in Cloud, Cloud Application Integration, Cloud Computing, Cloud Data Integration, Uncategorized | Tagged , , , , , | Leave a comment