Category Archives: Uncategorized

Large Data Sets Experience is Needed by Computer Science Graduates

Large Data Sets Experience is Needed

Large Data Sets Experience is Needed

During recent graduate interview discussions I asked about technical items mentioned on resumes. What I heard surprised me in a couple of cases. In particular, the very low volume data sets recent graduates have worked with. I was surprised that they had not used a large data set. I got the impression that some professors do not care about the volume of data that students work with when they create their applications.

In the media there is a constant discussion about a mismatch between the skills that education provides and the capabilities graduates bring to the work place. And, whether they are prepared for work. The lack of large data set use means that skills needed by employers may be missing. I will outline the skills that could be gained by working with large data sets.

Some types of data handling are just high volume. Business intelligence and analytics consume more data than 20 years ago. Handling the increasing volume is important. Research programming and data science are truly part of big data. Even if you are not doing data science, you may be preparing and handling the data sets. Some industries and organisations just have higher volumes of data. Retail is one example. Companies that used to have less volume are obtaining more data as they adapt to the big data world. We should expect the same trend to continue with organisations that have had higher data volumes in the past. They are going to have to handle a much bigger big data experience.

There are practical aspects to  handling large data sets. These can lead to experience in storage management and design, data loading, query optimization, parallelization, bandwidth issues and data quality when large data sets are used. And when you take on those issues, architecture skills are needed and can be gained.

Today, the trends known as the Internet of Things, All Things Data, and Data First are forming. As a result there will be demand for graduates who are familiar with handling high volumes of data.

The responsibility for using a large data set falls to the student. Faculty staff need to encourage this though. They often set and guide the students’ goals. A number of large data sets that could be used by students are on the web. An example of one data set would be the Harvard Library Bibliographic Dataset available at http://openmetadata.lib.harvard.edu/bibdata. Another example is the City of Chicago that makes a number of datasets available for download in a wide range of standard formats at https://data.cityofchicago.org/. The advantage of public large data sets is the volume and the opportunity to assess the data quality of the data set. Public data sets can hold many records. They represent many more combinations than we can quickly generate by hand.  Using even a small real world data set is a vast improvement over the likely limited number of variations in self-generated data. It may be even better than using a tool to generate data. Such data when downloaded can be manipulated and used as a base for loading.

Loading large data sets is part of being prepared. It requires the use of tools. These tools can be from loaders to full data integration tool suites. A good option for students who need to load data sets is PowerCenter Express. It was announced  last year. It is free for use with up to 250,000 rows per day. It is an ideal way to experience a full enterprise data integration tool and work with significantly higher volumes.

Big Data is here and it is a growing trend. And so students need to work with larger data sets than before. It is also feasible. The tools and the data sets the students need to work with large data sets are available. Therefore, in view of the current trends, large data set use should become standard practice in computer science and related courses.

FacebookTwitterLinkedInEmailPrintShare
Posted in Data Integration, Data Quality, PowerCenter, Uncategorized | Tagged , , , , | Leave a comment

Cloud Day Tidings and Tidbits at Informatica World 2014

With practically every on-premise application having a counterpart in the SaaS world, enterprise IT departments have truly made the leap to a new way of computing that is transforming their organizations. The last mile of cloud transformation lies in the field of integration, and it is for this purpose that Informatica had a dedicated Cloud Day this year at Informatica World 2014.

The day kicked off with an introduction by Ronen Schwartz, VP and GM of Informatica Cloud, to the themes of intelligent data integration, comprehensive cloud data management, and cloud process automation. The point was made that with SaaS applications being customized frequently, and the need for more data insights from these apps, it is important to have a single platform that can excel at both batch and real-time integration. A whole series of exciting panel discussions followed, ranging from mission critical Salesforce.com integration, to cloud data warehouses, to hybrid integration use cases involving Informatica PowerCenter and Informatica Cloud.

In the mission critical Salesforce.com integration panel, we had speakers from Intuit, InsideTrack, and Cloud Sherpas. Intuit talked about how they went live with Informatica Cloud in under four weeks, with only two developers on hand. InsideTrack had an interesting use case, wherein, they were using the force.com platform to build a native app that tracked performance of students and the impact of coaching on them. InsideTrack connected to several databases outside the Salesforce platform to perform sophisticated analytics and bring them into their app through the power of Informatica Cloud. Cloud Sherpas, a premier System Integrator, and close partner of both Salesforce.com and Informatica outlined three customer case studies of how they used Informatica Cloud to solve complex integration challenges. The first was a medical devices company that was trying to receive up-to-the-minute price quotes be integrating Salesforce and SAP, the second was a global pharmaceuticals company that was using Salesforce to capture data about their research subjects and needed to synchronize that information with their databases, and the third was Salesforce.com itself.

The die-hard data geeks came out in full force for the cloud data warehousing panel. Accomplished speakers from Microstrategy, Amazon, and The Weather Channel discussed data warehousing using Amazon Web Services. A first-time attendee to this panel would have assumed that cloud data warehousing simply dealt with running relational databases on virtual machines spun up from EC2, but instead participants were in enthralled to learn that Amazon Redshift was a relational database that ran 100% in the cloud. The Weather Channel uses Amazon Redshift to perform analytics on almost 750 million rows of data. Using Informatica Cloud, they can load this data into Redshift in a mere half hour. Microstrategy talked about their cloud analytics initiatives and how they looked at it holistically from a hybrid standpoint.

On that note, it was time for the panel of hybrid integration practitioners to take the stage, with Qualcomm and Conde Nast discussing their use of PowerCenter and Cloud. Qualcomm emphasized that the value of Informatica Cloud was the easy access to a variety of connectors, and that they were using connectors for Salesforce, NetSuite, several relational databases, and web services. Conde Nast mentioned that it was extremely easy to port mappings between PowerCenter and Cloud due to the common code base between the two.

FacebookTwitterLinkedInEmailPrintShare
Posted in Cloud Computing, Uncategorized | Tagged , , , , | Leave a comment

MDM Day Advice: Connect MDM to a Tangible Business Outcome or You Will Fail

“Start your master data management (MDM) journey knowing how it will deliver a tangible business outcome. Will it help your business generate revenue or cut costs? Focus on the business value you plan to deliver with MDM and revisit it often,” advises Michael Delgado, Information  Management Director at Citrix during his presentation at MDM Day, the InformaticaWorld 2014 pre-conference program. MDM Day focused on driving value from business-critical information and attracted 500 people.

A record 500 people attended MDM Day in Las Vegas

A record 500 people attended MDM Day in Las Vegas

In Ravi Shankar’s recent MDM Day preview blog, Part 2: All MDM, All Day at Pre-Conference Day at InformaticaWorld, he highlights the amazing line up of master data management (MDM) and product information management (PIM) customers speakers, Informatica experts as well as our talented partner sponsors.

Here are my MDM Day fun facts and key takeaways:

  • Did you know that every 2 seconds an aircraft with GE engine technology is taking off somewhere in the world?

    Ginny Walker, Chief Enterprise Architect at GE Aviation

    Ginny Walker, Chief Enterprise Architect at GE Aviation

    GE Aviation’s Chief Enterprise Architect, Ginny Walker, presented “Operationalizing Critical Business Processes: GE Aviation’s MDM Story.” GE Aviation is a $22 billion company and a leading provider of jet engines, systems and services.  Ginny shared the company’s multi-year journey to improve installed-base asset data management. She explained how the combination of data, analytics, and connectivity results in productivity improvements such as reducing up to 2% of the annual fuel bill and reducing delays. The keys to GE Aviation’s analytical MDM success were: 1) tying MDM to business metrics, 2) starting with a narrow scope, and 3) data stewards. Ginny believes that MDM is an enabler for the Industrial Internet and Big Data because it empowers companies to get insights from multiple sources of data.

  •  Did you know that EMC has made a $17 billion investment in acquisitions and is integrating more than 70 technology companies?
    Barbara Latulippe, EMC

    Barbara Latulippe, Senior Director, Enterprise Information Management at EMC

    EMC’s Barbara Latulippe, aka “The Data Diva,” is the Senior Director of Enterprise Information Management (EIM). EMC is a $21.7 billion company that has grown through acquisition and has 60,000 employees worldwide. In her presentation, “Formula for Success: EMC MDM Best Practices,” Barbara warns that if you don’t have a data governance program in place, you’re going to have a hard time getting an MDM initiative off the ground. She stressed the importance of building a data governance council and involving the business as early as possible to agree on key definitions such as “customer.” Barbara and her team focused on the financial impact of higher quality data to build a business case for operational MDM. She asked her business counterparts, “Imagine if you could onboard a customer in 3 minutes instead of 15 minutes?”

  • Did you know that Citrix is enabling the mobile workforce by uniting apps, data and services on any device over any network and cloud?

    Michael Delgado, Citrix

    Michael Delgado, Information Management Director at Citrix

    Citrix’s Information Management Director, Michael Delgado, presented “Citrix MDM Case Study: From Partner 360 to Customer 360.” Citrix is a $2.9 billion Cloud software company that embarked on a multi-domain MDM and data governance journey for channel partner, hierarchy and customer data. Because 90% of the company’s product bookings are fulfilled by channel partners, Citrix started their MDM journey to better understand their total channel partner relationship to make it easier to do business with Citrix and boost revenue. Once they were successful with partner data, they turned to customer data. They wanted to boost customer experience by understanding the total customer relationship across products lines and regions. Armed with this information, Citrix employees can engage customers in one product renewal process for all products. MDM also helps Citrix’s sales team with white space analysis to identify opportunities to sell more user licenses in existing customer accounts.

  •  Did you know Quintiles helped develop or commercialize all of the top 5 best-selling drugs on the market?

    John Poonnen, Quintiles

    John Poonnen, Director Infosario Data Factory at Quintiles

    Quintiles’ Director of the Infosario Data Factory, John Poonnen, presented “Using Multi-domain MDM to Gain Information Insights:How Quintiles Efficiently Manages Complex Clinical Trials.” Quintiles is the world’s largest provider of biopharmaceutical development and commercial outsourcing services with more than 27,000 employees. John explained how the company leverages a tailored, multi-domain MDM platform to gain a holistic view of business-critical entities such as investigators, research facilities, clinical studies, study sites and subjects to cut costs, improve quality, improve productivity and to meet regulatory and patient needs. “Although information needs to flow throughout the process – it tends to get stuck in different silos and must be manually manipulated to get meaningful insights,” said John. He believes master data is foundational — combining it with other data, capabilities and expertise makes it transformational.

While I couldn’t attend the PIM customer presentations below, I heard they were excellent. I look forward to watching the videos:

  • Crestline/ Geiger: Dale Denham, CIO presented, “How Product Information in eCommerce improved Geiger’s Ability to Promote and Sell Promotional Products.”
  • Murdoch’s Ranch and Home Supply: Director of Marketing, Kitch Walker presented, “Driving Omnichannel Customer Engagement – PIM Best Practices.”

I also had the opportunity MDM Day Sponsorsto speak with some of our knowledgeable and experienced MDM Day partner sponsors. Go to Twitter and search for #MDM and #DataQuality to see their advice on what it takes to successfully kick-off and implement an MDM program.

There are more thought-provoking MDM and PIM customer presentations taking place this week at InformaticaWorld 2014. To join or follow the conversation, use #INFA14 #MDM or #INFA14 #PIM.

FacebookTwitterLinkedInEmailPrintShare
Posted in Business Impact / Benefits, Business/IT Collaboration, CIO, CMO, Customer Acquisition & Retention, Customers, Data Governance, Data Integration, Data Quality, Enterprise Data Management, Informatica World 2014, Master Data Management, Partners, PiM, Product Information Management, Uncategorized | Tagged , , , , , , , , , , , , , , , , , , , , | 1 Comment

Saint-Gobain sees “time to market improvement” using Informatica PIM

Every company wants to see a “time to market improvement.” The wisest companies know this is only possible once you’ve mastered your internal data. One such company is Saint-Gobain, a Netherlands-based distributor of building materials. Saint-Gobain has accelerated and enhanced their customer’s multichannel experience using Informatica Product Information Management (PIM). Using Informatica PIM, Saint-Gobain has unleashed the potential of their information in the following ways:

  1. Ecommerce product introduction: Before using Informatica PIM, it took about one week to update a product to the website – now it is done within a few minutes.
  2. Everywhere commerce: The mobile app helps construction workers, on-site, to learn the details and stock availability of nearly 100,000 products and parts.
  3. Cross-selling: In addition to selling roof tiles, Saint-Gobain is also offering additional materials and tools as an up-sell.
  4. Retail stores: In addition to direct distribution, St. Gobain also sells through retailers. These specialty retailers need to create individual customer quotes which contain potential cross-sell and up-sell items. With Informatica PIM the retailers can create these custom quotes more effectively.

In the video below, Ron Kessels, Saint-Gobain’s Deputy Director of E-Business, talks about how they bring products to market more quickly while creating more opportunities for up-selling building supplies.

If you’d like to learn how your retail business can bring products to market more quickly, consider attending the Informatica World 2014 RETAIL PATH. This collection of sessions will show how to create unique customer experiences with relevant information, analytics, and relationships between data and people. In addition, the pre-conference MDM Day offers a track on “Omnichannel Commerce and Supplier Optimization”.

FacebookTwitterLinkedInEmailPrintShare
Posted in Informatica World 2014, Master Data Management, Product Information Management, Retail, Uncategorized | Tagged , , , , | 2 Comments

Retail Interview: From Product Information to Product Performance

Five questions to Arkady Kleyner, Executive VP & Co-Founder of Intricity LLC on how retailers can manage the transition from Product Information to Product Performance.

Arkady-Kleyner

Arkady-Kleyner

Arkady, you recently came back from the National Retail Federation conference.  What are some of the issues that retailers are struggling with these days?

Arkady Kleyner: There are some interesting trends happening right now in retail.  Amazon’s presence is creating a lot of disruption which is pushing traditional retailers to modernize their customer experience strategies.  For example, most Brick and Mortar retailers have a web presence, but they’re realizing that web presence can’t just be a second arm to their business.  To succeed, they need to integrate their web presence with their stores in a very intimate way.  To make that happen, they really have to peel back the onion down to the fundamentals of how product data is shared and managed.

In the good old days, Brick and Mortar retailers could live with a somewhat disconnected product catalog, because they were always ultimately picking from physical goods.  However in an integrated Web and Brick & Mortar environment, retailers must be far more accurate in their product catalog.  The customers entire product selection process may happen on-line but then picked up at the store.  So you can see where retailers need to be far more disciplined with their product data.  This is really where a Product Information Management tool is critical, with so many SKUs to manage, retailers really need a process that makes sense from end to end for onboarding and communicating a product to the customer.  And that is at the foundation of building an integrated customer experience.

In times of the digital customer, being online and connected always, we announced “commerce relevancy” as the next era of omnichannel and tailoring sales and marketing better to customers. What information are you seeing to be important when creating better customer shopping experience?

Arkady Kleyner:This is another paradigm in the integrated customer experience that retailers are trying to get their heads around. To appreciate how involved this is, just consider what a company like Amazon is doing.  They have millions of customers and millions of products and thousands of partners.  It’s literally a many to many to many relationship.  And this is why Amazon is eating everybody alive.  They know what products their customers like, they know how to reach those customers with those products, and they make it easy to buy it when you do.  This isn’t something that Amazon created over night, but the requirements are no different for the rest of retailers.  They need to ramp up the same type of capacity and reach.  For example if I sell jewelry I may be selling it on my own company store but I may also have 5 other partnering sites including Amazon.  Additionally, I may be using a dozen different advertising methods to drive demand.  Now multiply that times the number of jewelry products I sell and you have a massive hairball of complexity.  This is what we mean when we say that retailers need to be far more disciplined with their product data.  Having a Product Information Management process that spans the onboarding of products all the way through to the digital communication of those products is critical to a retailer staying relevant.

In which businesses do you see the need for more efficient product catalog management and channel convergence?

Arkady Kleyner: There is a huge opportunity out there for the existing Brick & Mortar retailers that embrace an integrated customer experience.  Amazon is not the de facto winner.  We see a future where the store near you actually IS the online store.  But to make that happen, Brick and Mortar retailers need to take a serious step back and treat their product data with the same reverence as they treat the product itself.  This means a well-managed process for onboarding, de-duping, and categorizing their product catalog, because all the customer marketing efforts are ultimately an extension of that catalog.

Which performance indicators are important? How can retailers profit from it?

Arkady Kleyner: There are two layers of performance indicators that are important.  The first is Operational Intelligence.  This is the intelligence that determines what product should be shown to who.  This is all based on customer profiling of purchase history.  The second is Strategic Intelligence.  This type of intelligence is the kind the helps you make overarching decisions on things like
-Maximizing the product margin by analyzing shipping and warehousing options
-Understanding product performance by demographics and regions
-Providing Flash Reports for Sales and Marketing

Which tools are needed to streamline product introduction but also achieve sales numbers?

Arkady Kleyner: Informatica is one of the few vendors that cares about data the same way retailers care about their products.  So if you’re a retailer, you really need to treat your product data with the same reverence as your physical products then you need to consider leveraging Informatica as a partner.  Their platform for managing product data is designed to encapsulate the entire process of onboarding, de-duping, categorizing, and syndicating product data.  Additionally Informatica PIM provides a platform for managing all the digital media assets so Marketing teams are able to focus on the strategy rather than tactics. We’ve also worked with Informatica’s data integration products to bring the performance data from the Point of Sale systems for both Strategic and Tactical uses. On the tactical side we’ve used this to integrate inventories between Web and Brick & Mortar so customers can have an integrated experience. On the strategic side we’ve integrated Warehouse Management Systems with Labor Cost tracking systems to provide a 360 degree view of the product costing including shipping and storage to drive a higher per unit margins.

You can hear more from Arkady in our webinarThe Streamlined SKU: Using Analytics for Quick Product Introductions” on Tuesday, March 4, 2014.

FacebookTwitterLinkedInEmailPrintShare
Posted in Business Impact / Benefits, Customer Acquisition & Retention, PiM, Product Information Management, Real-Time, Retail, Uncategorized | Leave a comment

LovEconomics: Did You Know These Facts on Valentine’s Day Retailing?

Inspired by the fact I was coming home from a business trip on Valentine’s Day.

Money makes the world go round
In the UK Valentine’s Day ranks behind Halloween, Mother’s Day, Easter and Christmas. British men spend 622m GBP, while women spend 354m. But the average purchase is 119 GBP. Germans for example only spend 59 GBP per person. According to a survey 53 per cent of US women will dump their boyfriends who do not give them anything on this day. China invented the singles day, where 3.5b GBP have been spend in 2013. A lot Americans spend money for pet gifts generating 227m of sales on Valentine’s Day.

All you need is love?
No, all you need is the right product to sell. Retailers use a wide range of an eclectic product to sell around this day, ranging from flowers to insurance and ecigaretts. IKEA Australia made furniture relevant for love with offering a free purchase for every child born nine months from Valentine’s Day.

What GfK and Google research say
In February Google searches showed a peak for recipes and poems. According to GfK, 81 per cent are using coupons when doing the purchase for Valentine.

Where and what to shop
Supermarkets have wrapped up to be the one-stop shop for lovers in a rush. Sainsbury reports a 12 per cent growth in sales of condoms. But did you know the top 8 gift ranking?
1. Cards and eCards
2. Flowers
3. Romantic dinners at restaurants
4. Romantic dinners at home (a condom and candles could be the perfect cross-sell to the wine and the recipe – or you plan to get pay-back from IKEA as mentioned above :-) )
5. Chocolate
6. Jewel leery
7. Lingerie
8. Weekends away

Sorry, but I will note tell you what I bring home for my wife. But did you know Informatica World offers a retail path this year? Long tail, ecommerce, from retail to me-tail, supply chain optimization and customer centricity and interesting company speakers are on the agenda.

FacebookTwitterLinkedInEmailPrintShare
Posted in Master Data Management, PiM, Product Information Management, Retail, Uncategorized | Leave a comment

Best Practices: Distributor Kramp Serving All Customer Requests With Long Tail Strategy

Our blog frequently provides best practice stories of our customers using product information management (PIM) for their business model. This case is about the “long tail strategy” at Kramp.

Tines, hand tools, spare parts for agricultural machines and hydraulic motors are the order of the day at Kramp.  Kramp, based in the Netherlands, is Europe’s largest wholesaler of accessories and spare parts for motorized equipment, agricultural and construction machines. The company’s business model and e-commerce strategy is exemplary. Kramp is using product information management (PIM) for their long tail strategy in e-commerce.

Kramp’s Value Proposition: “It’s that easy”

“We want to make it easier for our customers, partners and suppliers. We believe in the future and the power of e-commerce”, said CEO Eddie Perdok. Kramp grew the product assortment from about 200,000 to 1,000,000+ items from about 2,000 suppliers.

Previous stock policies in mail order retail always meant having limited space. In the catalog there were only a certain number of pages available. Even the logistics were limited – warehouse storage limited the possibilities so much that the majority of companies tried to find the “perfect catalog range” with the largest number of bestsellers.

The Digital Assortment Has No Limits

“Compared to other sales channels, the internet gives us significant cost advantages”, says Eddie Perdok. The digital department store consists of servers that can be easily extended at any time.

Adding a new product requires no more than a few additional entries in a database. The challenge is that the product data must be obtained from the suppliers and then distributed before products can be presented in a shop. The range is therefore often limited because the product data cannot be efficiently updated and sale is lost to other vendors are retailers.

Europe’s largest wholesaler of spare parts for agricultural machines and accessories focuses on managing all product data from a central data source for all sales channels and languages.

Longtail_PIM

 

Customer and supplier feedback is an important factor

“We want to bring customer opinion and supplier knowledge together”, explains Eddie Perdock. “Online customer evaluation combined with the knowledge of the manufacturer puts us in the position of being able to optimally control our stock”. In e-commerce, vendors, retailers and customers are coming closer and closer together.

 Benefits Kramp realized with PIM on their long tail strategy

  • Quick ROI due to short implementation phases
  • Better customer satisfaction due to optimal data quality
  • Higher margins and turnover in e-commerce due to niche items in long tail
  • Easy, professional handling of mass data lowers process costs
  • Short product introduction times to new markets

You can learn more on using PIM for long tail business on the entire case study or hear Ronald Renskers, Manager Product Content at Kramp, and others talking on the latest video.

Massively increasing the assortment is one of the top trends retailers and distributors focus on, according to Forrester Principal Analyst, Sucharita Mulpuru. Forrester’s research shows that retailers’ biggest competition are brands that sell directly to consumers. Marketplaces like Amazon result in higher margins, according to Forrester and www.pim-roi.com.

FacebookTwitterLinkedInEmailPrintShare
Posted in Master Data Management, PiM, Product Information Management, Retail, Uncategorized | Tagged , , , , | Leave a comment

Death of the Data Scientist: Silver Screen Fiction?

Maybe the word “death” is a bit strong, so let’s say “demise” instead.  Recently I read an article in the Harvard Business Review around how Big Data and Data Scientists will rule the world of the 21st century corporation and how they have to operate for maximum value.  The thing I found rather disturbing was that it takes a PhD – probably a few of them – in a variety of math areas to give executives the necessary insight to make better decisions ranging from what product to develop next to who to sell it to and where.

Who will walk the next long walk.... (source: Wikipedia)

Who will walk the next long walk…. (source: Wikipedia)

Don’t get me wrong – this is mixed news for any enterprise software firm helping businesses locate, acquire, contextually link, understand and distribute high-quality data.  The existence of such a high-value role validates product development but it also limits adoption.  It is also great news that data has finally gathered the attention it deserves.  But I am starting to ask myself why it always takes individuals with a “one-in-a-million” skill set to add value.  What happened to the democratization  of software?  Why is the design starting point for enterprise software not always similar to B2C applications, like an iPhone app, i.e. simpler is better?  Why is it always such a gradual “Cold War” evolution instead of a near-instant French Revolution?

Why do development environments for Big Data not accommodate limited or existing skills but always accommodate the most complex scenarios?  Well, the answer could be that the first customers will be very large, very complex organizations with super complex problems, which they were unable to solve so far.  If analytical apps have become a self-service proposition for business users, data integration should be as well.  So why does access to a lot of fast moving and diverse data require scarce PIG or Cassandra developers to get the data into an analyzable shape and a PhD to query and interpret patterns?

I realize new technologies start with a foundation and as they spread supply will attempt to catch up to create an equilibrium.  However, this is about a problem, which has existed for decades in many industries, such as the oil & gas, telecommunication, public and retail sector. Whenever I talk to architects and business leaders in these industries, they chuckle at “Big Data” and tell me “yes, we got that – and by the way, we have been dealing with this reality for a long time”.  By now I would have expected that the skill (cost) side of turning data into a meaningful insight would have been driven down more significantly.

Informatica has made a tremendous push in this regard with its “Map Once, Deploy Anywhere” paradigm.  I cannot wait to see what’s next – and I just saw something recently that got me very excited.  Why you ask? Because at some point I would like to have at least a business-super user pummel terabytes of transaction and interaction data into an environment (Hadoop cluster, in memory DB…) and massage it so that his self-created dashboard gets him/her where (s)he needs to go.  This should include concepts like; “where is the data I need for this insight?’, “what is missing and how do I get to that piece in the best way?”, “how do I want it to look to share it?” All that is required should be a semi-experienced knowledge of Excel and PowerPoint to get your hands on advanced Big Data analytics.  Don’t you think?  Do you believe that this role will disappear as quickly as it has surfaced?

FacebookTwitterLinkedInEmailPrintShare
Posted in Big Data, Business Impact / Benefits, CIO, Customer Acquisition & Retention, Customer Services, Data Aggregation, Data Integration, Data Integration Platform, Data Quality, Data Warehousing, Enterprise Data Management, Financial Services, Healthcare, Life Sciences, Manufacturing, Master Data Management, Operational Efficiency, Profiling, Scorecarding, Telecommunications, Transportation, Uncategorized, Utilities & Energy, Vertical | Tagged , , , , | 1 Comment

How Did Humanity Emerge from the Data Dark Ages?

A hundred years from now people will look back at this period of time and refer to it as the Data Dark Ages.  A time when the possibilities were endless but due to siloed data fiefdoms and polluted data sets the data science warlords and their minions experienced an insatiable hunger for data and rampant misinformation driving them to the brink of madness.   The minions spent endless hours in the dark dungeons preparing data from raw and untreated data sources for their data science overseers.  Solutions to the worlds’ most vexing problems were solvable if only the people had abundant access to clean and safe data to drive their analytic engines.

Legend held that a wizard in the land of Informatica possessed the magic of a virtual data machine called Vibe where a legion of data engineers built an intelligent data platform to provide a limitless supply of clean, safe, secure, and reliable data.  While many had tried to build their own data platforms only those who acquired the Informatica Intelligent Data Platform powered by Vibe were able to create true value and meaning from all types of data.

As word spread about Informatica Vibe and the Intelligent Data Platform data scientists and analysts sought its magic so they could have greater predictive power over the future.  The platform could feed any type of data of any volume into a data lake where Vibe, no matter the underlying technology, prepared and managed the data, and provisioned data to the masses hungry for actionable and reliable information.

An analytics renaissance soon emerged as more organizations adopted the Informatica Intelligent Data Platform where data was freely yet securely shared, integrated and cleansed at will, matched and correlated in real-time.  The data prep minions were set free and data scientists were able to spend the majority of their time discovering true value and meaning through big data analytics.  The pace of innovation accelerated and humanity enjoyed a new era of peace and prosperity.

FacebookTwitterLinkedInEmailPrintShare
Posted in Uncategorized | 3 Comments

Why the US Government is Desperately Seeking Data Integration

As covered by Loraine Lawson, “When it comes to data, the U.S. federal government is a bit of a glutton. Federal agencies manage on average 209 million records, or approximately 8.4 billion records for the entire federal government, according to Steve O’Keeffe, founder of the government IT network site, MeriTalk.”

Check out these stats, in a December 2013 MeriTalk survey of 100 federal records and information management professionals. Among the findings:

  • Only 18 percent said their agency had made significant progress toward managing records and email in electronic format, and are ready to report.
  • One in five federal records management professionals say they are “completely prepared” to handle the growing volume of government records.
  • 92 percent say their agency “has a lot of work to do to meet the direction.”
  • 46 percent say they do not believe or are unsure about whether the deadlines are realistic and obtainable.
  • Three out of four say the Presidential Directive on Managing Government Records will enable “modern, high-quality records and information management.”

I’ve been working with the US government for years, and I can tell that these facts are pretty accurate.  Indeed, the paper glut is killing productivity.  Even the way they manage digital data needs a great deal of improvement.

The problem is that the issues are so massive that’s it’s difficult to get your arms around it.  Just the DOD alone has hundreds of thousands of databases on-line, and most of them need to exchange data with other systems.  Typically this is done using old fashion approaches, including “sneaker-net,” Federal Express, FTP, and creaky batching extracts and updates.

The “digital data diet,” as Loraine calls it, really needs to start with a core understanding of most of the data under management.  That task alone will take years, but, at the same time, form an effective data integration strategy that considers the dozens of data integration strategies you likely formed in the past that did not work.

The path to better data management in the government is one where you have to map out a clear path from here to there.  Moreover, you need to make sure you define some successes along the way.  For example, the simple reduction of manual and paper processes by 5 or 10 percent would be a great start.  It’s something that would save the tax payers billions in a short period of time.

Too many times the government gets too ambitious around data integration, and attempts to do too much in too short an amount of time.  Repeat this pattern and you’ll find yourself running in quicksand, and really set yourself up for failure.

Data integration is game-changing technology.  Indeed, the larger you are, the more game-changing it is.  You can’t get much larger than the US government.  Time to get to work.

FacebookTwitterLinkedInEmailPrintShare
Posted in Data Integration, Data Integration Platform, Uncategorized | Tagged | Leave a comment