Tag Archives: data hub
Building an Enterprise Data Hub with proper Data IntegrationData flows into the enterprise from many sources, in many formats, sizes, and levels of complexity. And as enterprise architectures have evolved over the years, traditional data warehouses have become less of a final staging center for data, but rather, one component of the enterprise that interfaces with significant data flows. But since data warehouses should focus on being powerful engines for high value analytics, they should not be the central hub for data movement and data preparation (e.g. ETL/ELT), especially for the newer data types–such as social media, clickstream data, sensor data, internet-of-things-data, etc.–that are in use today.
When you start seeing data warehouse capacity consumed too quickly and performance degradation where end users are complaining about slower response times, and you risk not meeting your service-level agreements, then it might be time to consider an enterprise data hub (EDH). With an EDH, especially one built on Apache™ Hadoop®, you can plan a strategy around data warehouse optimization to get better use out of your entire enterprise architecture.
Of course, whenever you add another new technology to your data center, you care about interoperability. And since many systems in today’s architectures interoperate via data flows, it’s clear that sophisticated data integration technologies will be an important part of your EDH strategy. Today’s big data presents new challenges as relates to a wide variety of data types and formats, and the right technologies are needed to glue all the pieces together, whether those pieces are data warehouses, relational databases, Hadoop, or NoSQL databases.
Choosing a Data Integration Solution
Data integration software, at a high level, has one broad responsibility: to help you process and prepare your data with the right technology. This means it has to get your data to the right place in the right format in a timely manner. So it actually includes many tasks, but the end result is that timely, trusted data can be used for decision-making and risk management throughout the enterprise. You end up with a complete, ready-for-analysis picture of your business, as opposed to segmented snapshots based on a limited data set.
When evaluating a data integration solution for the enterprise, look for:
- Ease of use to boost developer productivity
- A proven track record in the industry
- Widely available technology expertise
- Experience with production deployments with newer technologies like Hadoop
- Ability to reuse data pipelines across different technologies (e.g. data warehouse, RDBMS, Hadoop, and other NoSQL databases)
Data integration is only part of the story. When you’re depending on data to drive business decisions and risk management, you clearly want to ensure the data is reliable. Data governance, data lineage, data quality, and data auditing remain as important topics in an EDH. Oftentimes, data privacy regulatory demands must be met, and the enterprise’s own intellectual property must be protected from accidental exposure.
To help ensure that data is sound and secure, look for a solution that provides:
- Centralized management and control
- Data certification prior to publication, transparent data and integration processes, and the ability to track data lineage
- Granular security, access controls, and data masking to protect data both in transit and at the source to prevent unauthorized access to specific data sets
Informatica is the data integration solution selected by many enterprises. Informatica’s family of enterprise data integration, data quality, and other data management products can manage data — of any format, complexity level, or size –from any business system, and then deliver that data across the enterprise at the desired speed.
Watch the latest Gartner video to see Todd Goldman, Vice President and General Manager for Enterprise Data Integration at Informatica, as well as executives from Cisco and MapR, give their perspective on how businesses today can gain even more value from big data.
In the evolution of Billing ‘thinking’ for Telcos we’ve seen everything from ‘All you can eat’ offers to ‘Another coin in the slot’. But this perennial business process black-hole can prove to be an area that can add to a Telcos armory in retaining and keeping happy its corporate customers. Not only this but following on from lessons learnt by the Financial Services community it can provide early warnings of customer, partner and service exposure, significant benefits to any organisations Revenue Assurance efforts.
The Integrated Customer Service Hub has evolved to allow customers, frequently the high value corporate organisations, on-line access firstly to Billing information then expanding to encompass other operational data such as new service orders and provisioning data, trouble tickets and service usage data. Increasingly customers are requiring being more in control of their services and so the hub has further evolved to allow customer self-servicing allowing them to place orders and receive information in the format that works for them not just their telecommunications service supplier. (more…)
Let’s look at the steps in more detail for building a business case for data quality using the bottom-up approach. Where do you start? You need to find a sponsor—someone who instinctively knows there is a problem and wants help in quantifying it. Marketing knows it has duplicate customer records and wants to get a better handle on them. You should look at these systems or business processes that work with the customer data. You must assess how the data in these systems is used within marketing. For example, what is the data used for, what critical decisions are made based on this data, and how many people use it to make decisions? The more users or the more critical the decision, the more likely this data is a candidate for evaluation. Also look at more than the initial decision support system and data. Look at any systems that get data from the decision support system. Data flow diagrams are always helpful in assessing this but usually difficult to find. (more…)