Tag Archives: data scientist

Fire your Data Scientists – They Don’t Add Value

Data ScientistYears ago, I was on a project to improve production and product quality through data analysis. During the project, I heard one man say: 

“If I had my way, I’d fire the statisticians – all of them – they don’t add value”. 

Surely not? Why would you fire the very people who were employed to make sense of the vast volumes of manufacturing data and guide future production?  But he was right. The problem was at that time data management was so poor that data was simply not available for the statisticians to analyze.

So, perhaps this title should be re-written to be: 

Fire your Data Scientists – They Aren’t Able to Add Value.

Although this statement is a bit extreme, the same situation may still exist. Data scientists frequently share frustrations such as:

  • “I’m told our data is 60% accurate, which means I can’t trust any of it.”
  • “We achieved our goal of an answer within a week by working 24 hours a day.”
  • “Each quarter we manually prepare 300 slides to anticipate all questions the CFO may ask.”
  • “Fred manually audits 10% of the invoices.  When he is on holiday, we just don’t do the audit.”

This is why I think the original quote is so insightful.  Value from data is not automatically delivered by hiring a statistician, analyst or data scientist. Even with the latest data mining technology, one person cannot positively influence a business without the proper data to support them.

Most organizations are unfamiliar with the structure required to deliver value from their data. New storage technologies will be introduced and a variety of analytics tools will be tried and tested. This change is crucial for to success. In order for statisticians to add value to a company, they must have access to high quality data that is easily sourced and integrated. That data must be available through the latest analytics technology. This new ecosystem should provide insights that can play a role in future production. Staff will need to be trained, as this new data will be incorporated into daily decision making.

With a rich 20-year history, Informatica understands data ecosystems. Employees become wasted investments when they do not have access to the trusted data they need in order to deliver their true value.

Who wants to spend their time recreating data sets to find a nugget of value only to discover it can’t be implemented?

Build a analytical ecosystem with a balanced focus on all aspects of data management. This will mean that value delivery is limited only by the imagination of your employees. Rather than questioning the value of an analytics team, you will attract some of the best and the brightest. Then, you will finally be able to deliver on the promised value of your data.

FacebookTwitterLinkedInEmailPrintShare
Posted in Big Data, Business Impact / Benefits, Data Integration, Data Integration Platform, Data Warehousing | Tagged , , , | 4 Comments

Death of the Data Scientist: Silver Screen Fiction?

Maybe the word “death” is a bit strong, so let’s say “demise” instead.  Recently I read an article in the Harvard Business Review around how Big Data and Data Scientists will rule the world of the 21st century corporation and how they have to operate for maximum value.  The thing I found rather disturbing was that it takes a PhD – probably a few of them – in a variety of math areas to give executives the necessary insight to make better decisions ranging from what product to develop next to who to sell it to and where.

Who will walk the next long walk.... (source: Wikipedia)

Who will walk the next long walk…. (source: Wikipedia)

Don’t get me wrong – this is mixed news for any enterprise software firm helping businesses locate, acquire, contextually link, understand and distribute high-quality data.  The existence of such a high-value role validates product development but it also limits adoption.  It is also great news that data has finally gathered the attention it deserves.  But I am starting to ask myself why it always takes individuals with a “one-in-a-million” skill set to add value.  What happened to the democratization  of software?  Why is the design starting point for enterprise software not always similar to B2C applications, like an iPhone app, i.e. simpler is better?  Why is it always such a gradual “Cold War” evolution instead of a near-instant French Revolution?

Why do development environments for Big Data not accommodate limited or existing skills but always accommodate the most complex scenarios?  Well, the answer could be that the first customers will be very large, very complex organizations with super complex problems, which they were unable to solve so far.  If analytical apps have become a self-service proposition for business users, data integration should be as well.  So why does access to a lot of fast moving and diverse data require scarce PIG or Cassandra developers to get the data into an analyzable shape and a PhD to query and interpret patterns?

I realize new technologies start with a foundation and as they spread supply will attempt to catch up to create an equilibrium.  However, this is about a problem, which has existed for decades in many industries, such as the oil & gas, telecommunication, public and retail sector. Whenever I talk to architects and business leaders in these industries, they chuckle at “Big Data” and tell me “yes, we got that – and by the way, we have been dealing with this reality for a long time”.  By now I would have expected that the skill (cost) side of turning data into a meaningful insight would have been driven down more significantly.

Informatica has made a tremendous push in this regard with its “Map Once, Deploy Anywhere” paradigm.  I cannot wait to see what’s next – and I just saw something recently that got me very excited.  Why you ask? Because at some point I would like to have at least a business-super user pummel terabytes of transaction and interaction data into an environment (Hadoop cluster, in memory DB…) and massage it so that his self-created dashboard gets him/her where (s)he needs to go.  This should include concepts like; “where is the data I need for this insight?’, “what is missing and how do I get to that piece in the best way?”, “how do I want it to look to share it?” All that is required should be a semi-experienced knowledge of Excel and PowerPoint to get your hands on advanced Big Data analytics.  Don’t you think?  Do you believe that this role will disappear as quickly as it has surfaced?

FacebookTwitterLinkedInEmailPrintShare
Posted in Big Data, Business Impact / Benefits, CIO, Customer Acquisition & Retention, Customer Services, Data Aggregation, Data Integration, Data Integration Platform, Data Quality, Data Warehousing, Enterprise Data Management, Financial Services, Healthcare, Life Sciences, Manufacturing, Master Data Management, Operational Efficiency, Profiling, Scorecarding, Telecommunications, Transportation, Uncategorized, Utilities & Energy, Vertical | Tagged , , , , | 1 Comment

When it comes to Data Quality Delivery, the Soft Stuff is the Hard Stuff (Part 6 of 6)

In my previous blog I explored the importance of a firm understanding of commercial packaged applications on data quality success. In this final post, I will examine the benefits of having operational experience as a key enabler of effective data quality delivery. (more…)

FacebookTwitterLinkedInEmailPrintShare
Posted in Big Data, Business Impact / Benefits, Data Governance, Data Quality, Master Data Management, Pervasive Data Quality | Tagged , , , , , | 1 Comment

Many Working Data Professionals are Already De Facto ‘Data Scientists,’ Survey Finds

Data scientist may be the hot job of 2013, but many data professionals report they are already doing much of the work that would be defined as the data scientist role. They just aren’t calling themselves data scientists – at least not yet.

In a new survey of 199 data managers I conducted as part of my work with Unisphere Research and Information Today, Inc., we found that the traits of data scientists – individuals whose backgrounds include IT and programming; math and statistics; and a willingness to look at things differently—are already seen within today’s organizations, in the day to day work performed by database administrators, analysts, managers and consultants. The survey was conducted among members of the Independent Oracle Users Group. (more…)

FacebookTwitterLinkedInEmailPrintShare
Posted in Data Integration | Tagged , , , , , , , | 1 Comment

The Data is In: Analytics Does Get You Ahead in Business

Finally, there is now evidence of a clear link between financial performance and the broad use of data by employees. Specifically, organizations that take the lead in data analytics are more than three times more likely to be leaders within their industry groups than companies with standard analytics environments.

That’s the finding of a new survey of 530 senior executives, conducted by the Economist Intelligence Unit. There is little disagreement that the ability to make data available across the entire enterprise means greater productivity and performance. More than 80 percent of respondents believe that employees across their organizations “can and should be using data to do their jobs.” (more…)

FacebookTwitterLinkedInEmailPrintShare
Posted in Data Integration | Tagged , | Leave a comment

My Thoughts on Gartner’s Thoughts About the Challenges of Hadoop for Data Integration

Just read a great Gartner report titled “Hadoop is Not a Data Integration Solution” (January 29, 2013). However, I beg to differ slightly with Gartner on this one. The title should have been “Hadoop Alone is Not a Data Integration Solution.” The report outlines all of the reasons that just deploying Hadoop by itself is often quite challenging.

Issues that we at Informatica have personally seen our customers deal with include: (more…)

FacebookTwitterLinkedInEmailPrintShare
Posted in Data Integration | Tagged , , , , , , , , , | 1 Comment

The Year Ahead: Big Data Reveals its Secrets to Both Data Providers and Consumers

In 2012, we began to recognize the presence of Big Data. In 2013, we will act on Big Data in deep and meaningful ways. This is good news for both data providers and consumers, who are no doubt anxious to begin the process of discovery of the new world opening up to them.

Over the coming year, those organizations who take the lead will be those who have people who understand how Big Data insights can be harvested. Outsmarting the competition means attracting and retaining skilled professionals who know how to manage, mine and draw actionable insights from data streaming into their organizations. That means rising demand for not only technical people who know how to manage Big Data sets, but also for people who can translate Big Data insights into business opportunities. Demand will intensify for professionals who understand the power of Big Data – from either the technical or the business side of data management. (more…)

FacebookTwitterLinkedInEmailPrintShare
Posted in Big Data | Tagged , , , , | Leave a comment

Let Your Data Scientists Be Scientists

With the rise in popularity of the elusive and expensive data scientist it’s very sad that once a data science team is assembled (at a very high recurring cost to the company I may add) that they spend most of their time doing work they weren’t really hired to do in the first place. That’s right! It turns out that data scientists spend only about 20% of their time doing real analysis – that is the work they were trained to do. How is the other 80% of their time spent? (more…)

FacebookTwitterLinkedInEmailPrintShare
Posted in Big Data | Tagged , , , , , , , | Leave a comment

Big Data Analysts’ Secrets, Revealed

Focus on the business impacts of analytics, not methodologies or insights…. keep the business involved in all iterations of an analytics process or project…. and remember, it’s ultimately skilled people that drive successful analytics.

These “secrets” of leading corporate analytical gurus may seem like everyday common sense, but rarely are put into practice. To find out what it takes for organizations to succeed with data analytics, Wayne Eckerson, founder of the BI Leadership Forum, recently spoke with seven exemplary analytics leaders, and distilled their advice in his latest book, Secrets of Analytical Leaders: Insights from Information Insiders. (more…)

FacebookTwitterLinkedInEmailPrintShare
Posted in Big Data | Tagged , , , , , , , , , , | Leave a comment

Data Scientist Named ‘Sexiest Job of the 21st Century’: Who is This Person?

Thomas Davenport, visiting professor at Harvard University and author of the watershed book Competing on Analytics, is once again making waves across the datasphere with his proclamation of data scientist as the “sexiest job of the 21st century.”

To many readers here at the Perspectives site, of course, this is not news, as many data professionals have increasingly been recognizing – and are being recognized – for the increasing power of information in driving new insights and business opportunities. (more…)

FacebookTwitterLinkedInEmailPrintShare
Posted in Big Data, Data Integration | Tagged , , , , , , | 3 Comments