Category Archives: Data Integration
The articles cites some research from Ovum, that predicts many enterprises will begin moving toward data integration, driven largely by the rise of cloud computing and big data. However, enterprises need to invest in both modernizing the existing data management infrastructure, as well as invest in data integration technology. “All of these new investments will push the middleware software market up 9 percent to a $16.3 billion industry, Information Management reports.” This projection is for 2015.
I suspect that’s a bit conservative. In my travels, I see much more interest in data integration strategies, approaches, and technology, as cloud computing continues to grow, as well as enterprises understand better the strategic use of data. So, I would put the growth at 15 percent for 2015.
There are many factors driving this growth, beyond mere interest in cloud computing and big data.
The first consideration is that data is more strategic than initially understood. While businesses have always considered data a huge asset, it has not been until the last few years that businesses have seen the true value of understanding what’s going on inside, and outside of their business.
Manufacturing companies want to see the current state of production, as well as production history. Management can now use that data to predict trends to address, such as future issues around employee productivity, or even a piece of equipment that is likely to fail and the impact of that failure on revenue. Healthcare companies are learning how to better monitor patient health, such as spotting likely health problems before they are diagnosed, or leveraging large data to understand when patterns emerge around health issues, such as areas of the country that are more prone to asthma, based upon air quality.
Second, there is the need to deal with compliance issues. The new health care regulations, or even the new regulation around managing a publically traded company, require a great deal of data management issues, including data integration.
As these laws emerge, and are altered over time, the reporting requirements are always more complex and far reaching than they were before. Those who want to avoid fines, or even avoid stock drops around mistakes, are paying close attention to this area.
Finally, there is an expectation from customers and employees that you will have a good handle on your data. 10 years ago you could tell a customer on the phone that you needed to check different systems to answer their question. Those days are over. Today’s customers and employees want immediate access to the data they need, and there is no good excuse for not being able to produce that data. If you can’t, your competition will.
The interest in data integration will experience solid growth in 2015, around cloud and big data, for sure. However, other factors will drive this growth, and enterprises will finally understand that data integration is core to an IT strategy, and should never be an afterthought.
Established in Northwestern United States, North 40 Outfitters, a family owned and operated business, has been outfitting the hardworking and hard playing populace of the region. Understanding the diverse needs of its customers, hardworking people, North 40 Outfitters carries everything from fencing for cattle and livestock to tools and trailers. They have gear for camping and hunting—even fly fishing.
Named after the Homestead Act of 1862, an event with strong significance in the region, North 40 Outfitters heritage is built on its community involvement and support of local small businesses. The company’s 700 employees could be regarded as family. At this year’s Thanksgiving, every employee was given a locally raised free range turkey to bring home. Furthermore, true to Black Friday’s shopping experience, North 40 Outfitters opened its door. Eschewing the regular practice of open as early as 3 AM, North 40 Outfitters opened at the reasonable 7 o’clock hour. They offered patrons donuts as well as coffee obtained from a local roaster.
North 40 Outfitters aims to be different. They achieve differentiation by being data driven. While the products they sell cannot be sourced exclusively from local sources, their experience aims to do exactly that.
Prior to operating under the name North 40 Outfitters, the company ran under the banner of “Big R”, which was shared with several other members of the same buying group. The decision to change the name to North 40 Outfitters was the result of a move into the digital realm— they needed a name to distinguish themselves. Now as North 40 Outfitters, they can focus on what matters rather than having to deal with the confusion of a shared name. They would now provide the “local store” experience, while investing in their digital strategy as a means to do business online and bring the unique North 40 Outfitters experience and value nationwide.
With those organizational changes taking place, lay an even greater challenge. With over 150,000 SKUs and no digital database for their product information, North 40 Outfitters had to find a solution and build everything from the ground up. Moreover, with customers demanding a means to buy products online, especially customers living in rural areas, it became clear that North 40 Outfitters would have to address its data concerns.
Along with the fresh rebrand and restructure, North 40 Outfitters needed to tame their product information situation, a critical step conducive to building their digital product database and launching their ecommerce store.
North 40 Outfitters was clear about the outcome of the recent rebranding and they knew that investments needed to be taken if they were to add value to their existing business. Building the capabilities to take their business to new channels, ecommerce in this case, meant finding the best solution to start on the right foot. Consequently, wishing to become master of their own data, for both online and in-store uses, North 40 Outfitters determined that they needed a PIM application that would act as a unique data information repository.
It’s important to note that North 40 Outfitters environment is not typical to that of traditional retailers. The difference can be found in the large variation of product type they sell. Some of their suppliers have local, boutique style production scales, while some are large multi-national distributors. Furthermore, a large portion of North 40 Outfitters customers live in rural regions, in some cases their stores are a day’s drive away. With the ability to leverage both a PIM and an ecommerce solution North 40 Outfitters is now a step closer to outfitting everyone in the Northwestern region.
It is still very early to talk about results, since North 40 Outfitters has only recently entered the implementation phase. What can be said is that they are very excited. Having reclaimed their territory, and equipped with a PIM solution and an ecommerce solution they have all the right tools to till and plow the playing field.
The meaning of North 40 Outfitters
To the uninitiated the name North 40 Outfitters might not mean much. However, there is a lot of local heritage and history standing behind this newly rebranded name. North 40 is derived from the Homestead Act of 1862. The Act refers to the “North forty”, to the Northern most block of the homesteader’s property. To this day, this still holds significance to the local community. The second half of the brand: “Outfitters” is about the company’s focus on the company ability to outfit its customers both for work and play. On the one hand, you can visit North 40 Outfitters to purchase goods aimed at running your ranch, such as fencing material, horse related goods or quality tools. At the same time, you can buy camping and backpacking goods—they even sell ice fishing huts.
North 40 Outfitters ensures their customers have what they need to work the land, get back from it and ultimately go out and play just as hard if not harder.
The other day I ran across an article on CMO.com from a few months ago entitled “Total Customer Value Trumps Simple Loyalty in Digital World”. It’s a great article, so I encourage you to go take a look, but the basic premise is that loyalty does not necessarily equal value in today’s complicated consumer environment.
Customers can be loyal for a variety of reasons as the author Samuel Greengard points out. One of which may be that they are stuck with a certain product or service because they believe there is no better alternative available. I know I can relate to this after a recent series of less-than-pleasant experiences with my bank. I’d like to change banks, but frankly they’re all about the same and it just isn’t worth the hassle. Therefore, I’m loyal to my unnamed bank, but definitely not an advocate.
The proverbial big fish in today’s digital world, according to the author, are customers who truly identify with the brand and who will buy the company’s products eagerly, even when viable alternatives exist. These are the customers who sing the brand’s praises to their friends and family online and in person. These are the customers who write reviews on Amazon and give your product 5 stars. These are the customers who will pay markedly more just because it sports your logo. And these are the customers whose voices hold weight with their peers because they are knowledgeable and passionate about the product. I’m sure we all have a brand or two that we’re truly passionate about.
Total Customer Value in the Pool
My 13 year old son is a competitive swimmer and will only use Speedo goggles – ever – hands down – no matter what. He wears Speedo t-shirts to show his support. He talks about how great his goggles are and encourages his teammates to try on his personal pair to show them how much better they are. He is a leader on his team, so when newbies come in and see him wearing these goggles and singing their praises, and finishing first, his advocacy holds weight. I’m sure we have owned well over 30 pair of Speedo goggles over the past 4 years at $20 a pop – and add in the T-Shirts and of course swimsuits – we probably have a historical value of over $1000 and a potential lifetime value of tens of thousands (ridiculous I know!). But if you add in the influence he’s had over others, his value is tremendously more – at least 5X.
This is why data is king!
I couldn’t agree more that total customer value, or even total partner or total supplier value, is absolutely the right approach, and is a much better indicator of value. But in this digital world of incredible data volumes and disparate data sources & systems, how can you really know what a customer’s value is?
The marketing applications you probably already use are great – there are so many great automation, web analytics, and CRM systems around. But what fuels these applications? Your data.
Most marketers think that data is the stuff that applications generate or consume. As if all data is pretty much the same. In truth, data is a raw ingredient. Data-driven marketers don’t just manage their marketing applications, they actively manage their data as a strategic asset.
How are you using data to analyze and identify your influential customers? Can you tell that a customer bought their fourth product from your website, and then promptly tweeted about the great deal they got on it? Even more interesting, can you tell that that five of their friends followed the link, 1 bought the same item, 1 looked at it but ended up buying a similar item, and 1 put it in their cart but didn’t buy it because it was cheaper on another website? And more importantly, how can you keep this person engaged so they continue their brand preference – so somebody else with a similar brand and product doesn’t swoop in and do it first? And the ultimate question… how can you scale this so that you’re doing this automatically within your marketing processes, with confidence, every time?
All marketers need to understand their data – what exists in your information ecosystem , whether it be internally or externally. Can you even get to the systems that hold the richest data? Do you leverage your internal customer support/call center records? Is your billing /financial system utilized as a key location for customer data? And the elephant in the room… can you incorporate the invaluable social media data that is ripe for marketers to leverage as an automated component of their marketing campaigns?
This is why marketers need to care about data integration…
Even if you do have access to all of the rich customer data that exists within and outside of your firewalls, how can you make sense of it? How can you pull it together to truly understand your customers… what they really buy, who they associate with, and who they influence. If you don’t, then you’re leaving dollars, and more importantly, potential advocacy and true customer value, on the table.
This is why marketers need to care about achieving a total view of their customers and prospects…
And none of this matters if the data you are leveraging is plain incorrect or incomplete. How often have you seen some analysis on an important topic, had that gut feeling that something must be wrong, and questioned the data that was used to pull the report? The obvious data quality errors are really only the tip of the iceberg. Most of the data quality issues that marketers face are either not glaringly obvious enough to catch and correct on the spot, or are baked into an automated process that nobody has the opportunity to catch. Making decisions based upon flawed data inevitably leads to poor decisions.
This is why marketers need to care about data quality.
So, as the article points out, don’t just look at loyalty, look at total customer value. But realize, that this is easier said than done without a focusing in on your data and ensuring you have all of the right data, at the right place, in the right format, right away.
Now… Brand advocates, step up! Share with us your favorite story. What brands do you love? Why? What makes you so loyal?
Building an Enterprise Data Hub with proper Data IntegrationData flows into the enterprise from many sources, in many formats, sizes, and levels of complexity. And as enterprise architectures have evolved over the years, traditional data warehouses have become less of a final staging center for data, but rather, one component of the enterprise that interfaces with significant data flows. But since data warehouses should focus on being powerful engines for high value analytics, they should not be the central hub for data movement and data preparation (e.g. ETL/ELT), especially for the newer data types–such as social media, clickstream data, sensor data, internet-of-things-data, etc.–that are in use today.
When you start seeing data warehouse capacity consumed too quickly and performance degradation where end users are complaining about slower response times, and you risk not meeting your service-level agreements, then it might be time to consider an enterprise data hub (EDH). With an EDH, especially one built on Apache™ Hadoop®, you can plan a strategy around data warehouse optimization to get better use out of your entire enterprise architecture.
Of course, whenever you add another new technology to your data center, you care about interoperability. And since many systems in today’s architectures interoperate via data flows, it’s clear that sophisticated data integration technologies will be an important part of your EDH strategy. Today’s big data presents new challenges as relates to a wide variety of data types and formats, and the right technologies are needed to glue all the pieces together, whether those pieces are data warehouses, relational databases, Hadoop, or NoSQL databases.
Choosing a Data Integration Solution
Data integration software, at a high level, has one broad responsibility: to help you process and prepare your data with the right technology. This means it has to get your data to the right place in the right format in a timely manner. So it actually includes many tasks, but the end result is that timely, trusted data can be used for decision-making and risk management throughout the enterprise. You end up with a complete, ready-for-analysis picture of your business, as opposed to segmented snapshots based on a limited data set.
When evaluating a data integration solution for the enterprise, look for:
- Ease of use to boost developer productivity
- A proven track record in the industry
- Widely available technology expertise
- Experience with production deployments with newer technologies like Hadoop
- Ability to reuse data pipelines across different technologies (e.g. data warehouse, RDBMS, Hadoop, and other NoSQL databases)
Data integration is only part of the story. When you’re depending on data to drive business decisions and risk management, you clearly want to ensure the data is reliable. Data governance, data lineage, data quality, and data auditing remain as important topics in an EDH. Oftentimes, data privacy regulatory demands must be met, and the enterprise’s own intellectual property must be protected from accidental exposure.
To help ensure that data is sound and secure, look for a solution that provides:
- Centralized management and control
- Data certification prior to publication, transparent data and integration processes, and the ability to track data lineage
- Granular security, access controls, and data masking to protect data both in transit and at the source to prevent unauthorized access to specific data sets
Informatica is the data integration solution selected by many enterprises. Informatica’s family of enterprise data integration, data quality, and other data management products can manage data — of any format, complexity level, or size –from any business system, and then deliver that data across the enterprise at the desired speed.
Watch the latest Gartner video to see Todd Goldman, Vice President and General Manager for Enterprise Data Integration at Informatica, as well as executives from Cisco and MapR, give their perspective on how businesses today can gain even more value from big data.
How would you like to wake up to an extra billion dollars, or maybe nine, in the bank? This has happened to a teacher in India. He discovered to his astonishment a balance of $9.8 billion in his bank account!
How would you like to be the bank who gave the client an extra nine Billion dollars? Oh, to be a fly on the wall when the IT department got that call. How do you even begin to explain? Imagine the scrambling to track down the source of the data error.
This was a glaringly obvious error, which is easily caught. But there is potential for many smaller data errors. These errors may go undetected and add up hurting your bottom line. How could this type of data glitch happen? More importantly, how can you protect your organization from these types of errors in your data?
A primary source of data mistakes is insufficient testing during Data Integration. Any change or movement of data harbors risk to its integrity. Unfortunately there are often insufficient IT resources to adequately validate the data. Some organizations validate the data manually. This is a lengthy, unreliable process, fraught with data errors. Furthermore manual testing does not scale well to large data volumes or complex data changes. So the validation is often incomplete. Finally some organizations simply lack the resources to conduct any level of data validation altogether.
Many of our customers have been able to successfully address this issue via automated data validation testing. (Also known as DVO). In a recent TechValidate survey, Informatica customers have told us that they:
- Reduce costs associated with data testing.
- Reduce time associated with data testing.
- Increase IT productivity.
- Increase the business trust in the data.
Customers tell us some of the biggest potential costs relate to damage control which occurs when something goes wrong with their data. The tale above, of our fortunate man and not so fortunate bank, can be one example. Bad data can hurt a company’s reputation and lead to untold losses in market-share and customer goodwill. In today’s highly regulated industries, such as healthcare and financial services, consequences of incorrect data can be severe. This can include heavy fines or worse.
Using automated data validation testing allows customers to save on ongoing testing costs and deliver reliable data. Just as important, it prevents pricey data errors, which require costly and time-consuming damage control. It is no wonder many of our customers tell us they are able to recoup their investment in less than 12 months!
TechValidate survey shows us that customers are using data validation testing in a number of common use cases including:
- Regression (Unit) testing
- Application migration or consolidation
- Software upgrades (Applications, databases, PowerCenter)
- Production reconciliation
One of the most beneficial use cases for data validation testing has been for application migration and consolidation. Many SAP migration projects undertaken by our customers have greatly benefited from automated data validation testing. Application migration or consolidation projects are typically large and risky. A Bloor Research study has shown 38% of data migration projects fail, incurring overages or are aborted altogether. According to a Harvard Business Review article, 1 in 6 large IT projects run 200% over budget. Poor data management is one of the leading pitfalls in these types of projects. However, according to Bloor Research, Informatica’ s data validation testing is a capability they have not seen elsewhere in the industry.
A particularly interesting example of above use case is in the case of M&A situation. The merged company is required to deliver ‘day-1 reporting’. However FTC regulations forbid the separate entities from seeing each other’s data prior to the merger. What a predicament! The automated nature of data validation testing, (Automatically deploying preconfigured rules on large data-sets) enables our customers to prepare for successful day-1 reporting under these harsh conditions.
And what about you? What are the costs to your business for potentially delivering incorrect, incomplete or missing data? To learn more about how you can provide the right data on time, every time, please visit www.datavalidation.me
Data warehousing systems remain the de facto standard for high performance reporting and business intelligence, and there is no sign that will change soon. But Hadoop now offers an opportunity to lower costs by transferring infrequently used data and data preparation workloads off of the data warehouse and process entirely new sources of data coming from the explosion of industrial and personal devices. This is motivating interest in new concepts like the “data lake” as adjunct environments to traditional data warehousing systems.
Now, let’s be real. Between the evolutionary opportunity of preparing data more cost effectively and the revolutionary opportunity of analyzing new sources of data, the latter just sounds cooler. This revolutionary opportunity is what has spurred the growth of new roles like data scientists and new tools for self-service visualization. In the revolutionary world of pervasive analytics, data scientists have the ability to use Hadoop as a low cost and transient sandbox for data. Data scientists can perform exploratory data analysis by quickly dumping data from a variety of sources into a schema-on-read platform and by iterating dumps as new data comes in. SQL-on-Hadoop technologies like Cloudera Impala, Hortonworks Stinger, Apache Drill, and Pivotal HAWQ enable agile and iterative SQL-like queries on datasets, while new analysis tools like Tableau enable self-service visualization. We are merely in the early phases of the revolutionary opportunity of big data.
But while the revolutionary opportunity is exciting, there’s an equally compelling opportunity for enterprises to modernize their existing data environment. Enterprises cannot rely on an iterative dump methodology for managing operational data pipelines. Unmanaged “data swamps” are simply unpractical for business operations. For an operational data pipeline, the Hadoop environment must be a clean, consistent, and compliant system of record for serving analytical systems. Loading enterprise data into Hadoop instead of a relational data warehouse does not eliminate the need to prepare it.
Now I have a secret to share with you: nearly every enterprise adopting Hadoop today to modernize their data environment has processes, standards, tools, and people dedicated to data profiling, data cleansing, data refinement, data enrichment, and data validation. In the world of enterprise big data, schemas and metadata still matter.
I’ll share some examples with you. I attended a customer panel at Strata + Hadoop World in October. One of the participants was the analytics program lead at a large software company whose team was responsible for data preparation. He described how they ingest data from heterogeneous data sources by mandating a standardized schema for everything that lands in the Hadoop data lake. Once the data lands, his team profiles, cleans, refines, enriches, and validates the data so that business analysts have access to high quality information. Another data executive described how inbound data teams are required to convert data into Avro before storing the data in the data lake. (Avro is an emerging data format alongside other new formats like ORC, Parquet, and JSON). One data engineer from one of the largest consumer internet companies in the world described the schema review committee that had been set up to govern changes to their data schemas. The final participant was an enterprise architect from one of the world’s largest telecom providers who described how their data schema was critical for maintaining compliance with privacy requirements since data had to be masked before it could be made available to analysts.
Let me be clear – these companies are not just bringing in CRM and ERP data into Hadoop. These organizations are ingesting patient sensor data, log files, event data, clickstream data, and in every case, data preparation was the first task at hand.
I recently talked to a large financial services customer who proposed a unique architecture for their Hadoop deployment. They wanted to empower line of business users to be creative in discovering revolutionary opportunities while also evolving their existing data environment. They decided to allow line of businesses to set up sandbox data lakes on local Hadoop clusters for use by small teams of data scientists. Then, once a subset of data was profiled, cleansed, refined, enriched, and validated, it would be loaded into a larger Hadoop cluster functioning as an enterprise information lake. Unlike the sandbox data lakes, the enterprise information lake was clean, consistent, and compliant. Data stewards of the enterprise information lake could govern metadata and ensure data lineage tracking from source systems to sandbox to enterprise information lakes to destination systems. Enterprise information lakes balance the quality of a data warehouse with the cost-effective scalability of Hadoop.
Building enterprise information lakes out of data lakes is simple and fast with tools that can port data pipeline mappings from traditional architectures to Hadoop. With visual development interfaces and native execution on Hadoop, enterprises can accelerate their adoption of Hadoop for operational data pipelines.
No one described the opportunity of enterprise information lakes better at Strata + Hadoop World than a data executive from a large healthcare provider who said, “While big data is exciting, equally exciting is complete data…we are data rich and information poor today.” Schemas and metadata still matter more than ever, and with the help of leading data integration and preparation tools like Informatica, enterprises have a path to unleashing information riches. To learn more, check out this Big Data Workbook
Insurance companies serve as a fantastic example of big data technology use since data is such a pervasive asset in the business. From a cost savings and risk mitigation standpoint, insurance companies use data to assess the probable maximum loss of catastrophic events as well as detect the potential for fraudulent claims. From a revenue growth standpoint, insurance companies use data to intelligently price new insurance offerings and deploy cross-sell offers to customers to maximize their lifetime value.
New data sources are enabling insurance companies to mitigate risk and grow revenues even more effectively. Location-based data from mobile devices and sensors are being used inside insured properties to proactively detect exposure to catastrophic events and deploy preventive maintenance. For example, automobile insurance providers are increasingly offering usage-based driving programs, whereby insured individuals install a mobile sensor inside their car to relay the quality of their driving back to their insurance provider in exchange for lower premiums. Even healthcare insurance providers are starting to analyze the data collected by wearable fitness bands and smart watches to monitor insured individuals and inform them of personalized ways to be healthier. Devices can also be deployed in the environment that triggers adverse events, such as sensors to monitor earthquake and weather patterns, to help mitigate the costs of potential events. Claims are increasingly submitted with supporting information in a variety of formats like text files, spreadsheets, and PDFs that can be mined for insights as well. And with the growth on insurance sales online, web log and clickstream data is more important than ever to help drive online revenue.
Beyond the benefits of using new data sources to assess risk and grow revenues, big data technologies are enabling insurance companies to fundamentally rethink the basis of their analytical architecture. In the past, probable maximum loss modeling could only be performed on statistically aggregated datasets. But with big data technologies, insurance companies have the opportunity to analyze data at the level of an insured individual or a unique insurance claim. This increased depth of analysis has the potential to radically improve the quality and accuracy of risk models and market predictions.
Informatica is helping insurance companies accelerate the benefits of big data technologies. With multiple styles of ingestion available, Informatica enables insurance companies to leverage nearly any source of data. Informatica Big Data Edition provides comprehensive data transformations for ETL and data quality, so that insurance companies can profile, parse, integrate, cleanse, and refine data using a simple user-friendly visual development environment. With built-in data lineage tracking and support for data masking, Informatica helps insurance companies ensure regulatory compliance across all data.
To try out the Big Data Edition, download a free trial today in the Informatica Marketplace and get started with big data today!
According to the article, in Hamilton County Ohio, it’s not unusual to see kids from the same neighborhoods coming to the hospital for asthma attacks. Thus, researchers wanted to know if it was fact or mistaken perception that an unusually high number of children in the same neighborhood were experiencing asthma attacks. The next step was to review existing data to determine the extent of the issues, and perhaps how to solve the problem altogether.
“The researchers studied 4,355 children between the ages of 1 and 16 who visited the emergency department or were hospitalized for asthma at Cincinnati Children’s between January 2009 and December 2012. They tracked those kids for 12 months to see if they returned to the ED or were readmitted for asthma.”
Not only were the researchers able to determine a sound correlation between the two data sets, but they were able to advance the research to predict which kids were at high-risk based upon where they live. Thus, some of the cause and the effects have been determined.
This came about when researchers began thinking out of the box, when it comes to dealing with traditional and non-traditional medical data. They integrated housing and census data, in this case, with that of the data from the diagnosis and treatment of the patients. These are data sets unlikely to find their way to each other, but together they have a meaning that is much more valuable than if they just stayed in their respective silos.
“Non-traditional medical data integration has begun to take place in some medical collaborative environments already. The New York-Presbyterian Regional Health Collaborative created a medical village, which ‘goes beyond the established patient-centered medical home mode.’ It not only connects an academic medical center with a large ambulatory network, medical homes, and other providers with each other, but community resources such as school-based clinics and specialty-care centers (the ones that are a part of NYP’s network).”
The fact of the matter is that data is the key to understanding what the heck is going on when cells of sick people begin to emerge. While researchers and doctors can treat the individual patients there is not a good understanding of the larger issues that may be at play. In this case, poor air quality in poor neighborhoods. Thus, they understand what problem needs to be corrected.
The universal sharing of data is really the larger solution here, but one that won’t be approached without a common understanding of the value, and funding. As we pass laws around the administration of health care, as well as how data is to be handled, perhaps it’s time we look at what the data actually means. This requires a massive deployment of data integration technology, and the fundamental push to share data with a central data repository, as well as with health care providers.
On November 13, 2014, Informatica acquired the assets of Proact, whose Enterprise Architecture tools and delivery capability link architecture to business strategy. The BOST framework is now the Informatica Business Transformation Toolkit which received high marks in a recent research paper:
“(BOST) is a framework that provides four architectural views of the enterprise (Business, Operational, Systems, and Technology). This EA methodology plans and organizes capabilities and requirements at each view, based on evolving business and opportunities. It is one of the most finalized of the methodologies, in use by several large enterprises.”  (more…)
Or in other words: Did the agency model kill data quality? When you watch the TV series “Homeland”, you quickly realize the interdependence between field operatives and the command center. This is a classic agency model. One arm gathers, filters and synthesizes information and prepares a plan but the guys on the ground use this intel to guide their sometimes ad hoc next moves.
The last few months I worked a lot – and I mean A LOT – with a variety of mid-sized life insurers (<$1B annual revenue) around fixing their legacy-inherited data quality problems. Their IT departments, functioning like Operations Command Centers (intel acquisition, filter and synthesize), were inundated with requests to fix up and serve a coherent, true, enriched central view of a participant (the target) and all his policies and related entities from and to all relevant business lines (planning) to achieve their respective missions (service, retain, upsell, mitigate risk): employee benefits, broker/dealer, retirement services, etc.
The captive and often independent agents (execution); however, often run with little useful data into an operation (sales cycle) as the Ops Center is short on timely and complete information. Imagine Carrie directing her strike team to just wing it based on their experience and dated intel from a raid a few months ago without real-time drone video feeds. Would she be saying, “Guys, it’s always been your neck, you’ll figure it out.” I think not.
This becomes apparent when talking to the actuaries, claims operations, marketing, sales, agency operations, audit, finance, strategic planning, underwriting and customer service, common denominators appeared quickly:
- Every insurer saw the need to become customer instead of policy centric. That’s the good news.
- Every insurer knew their data was majorly sub-standard in terms of quality and richness.
- Every insurer agreed that they are not using existing data capture tools (web portals for agents and policy holders, CRM applications, policy and claims mgmt systems) to their true potential.
- New books-of-business were generally managed as separate entities from a commercial as well as IT systems perspective, even if they were not net-new products, like trust products. Cross-selling as such, even if desired, becomes a major infrastructure hurdle.
- As in every industry, the knee-jerk reaction was to throw the IT folks at data quality problems and making it a back-office function. Pyrrhic victory.
- Upsell scenarios, if at all strategically targeted, are squarely in the hands of the independent agents. The insurer will, at most, support customer insights around lapse avoidance or 401k roll-off indicators for retiring or terminated plan participants. This may be derived from a plan sponsor (employer) census file, which may have incorrect address information.
- Prospect and participant e-mail addresses are either not captured (process enforcement or system capability) or not validated (domain, e-mail verification), so the vast majority of customer correspondence, like scenarios, statements, privacy notices and policy changes, travels via snail mail (and this typically per policy). Overall, only 15-50% of contacts have an “unverified” e-mail address today and of these less than a third subscribed to exclusive electronic statement delivery.
- Postal address information is still not 99% correct, complete or current, resulting in high returned mail ($120,000-$750,000 every quarter), priority mail upgrades, statement reprints, manual change capture and shredding cost as well as the occasional USPS fine.
- Data quality, as unbelievable as it sounds, took a back-burner to implementing a new customer data warehouse, a new claims system, a new risk data mart, etc. They all just get filled with the same old, bad data as business users were – and I quote –“used to the quality problem already”.
- Premium underpricing runs at 2-4% annually, foregoing millions in additional revenue, due to lack of a full risk profile.
- Customer cost –of-acquisition (CAC) is unknown or incorrect as there is no direct, realistic tracking of agency campaign/education dollars spent against new policies written.
- Agency historic production and projections are unclear as a dynamic enforcement of hierarchies is not available, resulting in orphaned policies generating excess tax burdens. Often this is the case when agents move to states where they are not licensed, they passed or retired.
What does a cutting-edge insurer look like instead? Ask Carrie Mathison and Saul Bernstein. They already have a risk and customer EDW as well as a modern (cloud based?) CRM and claims mgmt system. They have considered, as part of their original implementation or upgrade, capabilities required to fix the initial seed data into their analytics platforms. Now, they are looking into pushing them back into operational systems like CRM and avoiding bad source system entries from the get-go.
They are also beyond just using data to avoid throwing more bodies in every department at “flavor-of-the-month” clean-up projects, e.g. yet another state unclaimed property matching exercise, total annual premium revenue written in state X for tax review purposes by the state tax authority.
So what causes this drastic segmentation of leading versus laggard life insurers? In my humble opinion, it is the lack of a strategic refocusing of what the insurer can do for an agent by touching the prospects and customers directly. Direct interaction (even limited) improves branding, shortens the sales cycle and rates based on improved insights through better data quality.
Agents (and insurers) need to understand that the wealth of data (demographic, interactions, transactions) corporate possesses already via native and inherited (via M&A) can be a powerful competitive differentiator. Imagine if they start tapping into external sources beyond the standard credit bureaus and consumer databases; dare I say social media?
Competing based on immediate instead of long term needs (in insurance: life time earnings potential replacement), price (fees) and commission cannot be the sole answer.