5 Steps to a Successful Big Data POC

big data poc
5 Steps to a Successful Big Data POC

If there’s one thing early big data projects have proven, it’s that you need a carefully planned, phased approach to prove the value of big data to the enterprise. That means starting with a well-planned proof of concept (POC) that gains buy-in and confidence from your key executives.

With this in mind, we’ve created a workbook, “How to Run a Big Data POC in Six Weeks” to share the advice and best practices needed to run a successful big data POC. Based on our lessons learned from talking with many of our customers and partners (and our own marketing big data POC—check out the Naked Marketing blog series by my colleague Franz Aman), the workbook helps you complete your POC in a very short timeframe—an ambitious but very doable six weeks.

The workbook goes into greater detail, but in this blog post, I want to share with you the five principles for a successful big data POC. It’s good to learn from others who have gone through the process so read the list below and let me know in the comments section if you have others to add.

  1. Start Small!

By keeping your big data POC small in scope and limited to a short timeframe, you’ll ensure initial success and set yourself up for additional projects. This means designing a POC that’s of a size that can be completed in six weeks; important enough that it can prove the value of big data to the rest of the organization; and designed intelligently so you can eventually grow the project incrementally.

  1. Get Early Buy-in from Execs.

The success or failure of your POC hinges on your relationship with the executives sponsoring the project. It’s important that you make them active participants in the project.

That means treating them like a crucial part of the POC process, understanding the outcomes they want to champion, and making sure that what you build proves their value. In some cases, you may need to manage expectations, and in others you may need to show them they can actually aim higher.

The bottom line is that without active involvement of key executives, you’ll struggle to prove the value of a POC to the enterprise.

  1. Plan Meticulously.

Even at an early stage, planning, architecture, and design are crucial considerations for big data implementations. After all, you don’t want to spend half of your six weeks reconfiguring your Hadoop cluster because it was set up incorrectly or intended for a different project. While you may not need your project to have a fully designed architecture to prove value, you will need to understand the implications of various technical components. A common theme in all successful POCs is planning well before production.

  1. Avoid Scope Creep.

As you go through the POC build and realize what’s possible, you’ll likely be tempted to broaden the scope of your first use case. Don’t give in to this temptation! Scope creep invariably slows down deployments. And a small project that solves a single business problem is a lot more valuable than a broad project that doesn’t.

Your POC should be tightly focused on precisely what it’s meant to solve in six weeks. That requires focusing everyone involved on a single use case that’s important enough to see through before you start building.

  1. Keep the Focus on the Data.

Focus on the data, and keep things simple—aiming for as few data sources as possible. Go back and forth between what needs to be achieved and what’s possible in six weeks until you have to manage only a few sources. You can always add new ones once you’ve proved value.

In terms of the actual data itself, focus only on the dimensions and measures that actually matter to your specific POC. For example, if your web analytics data has 400 fields but you only need 10 of them, you can rationalize your efforts by focusing on those 10. Similarly, if you’re attempting to run predictive analytics based on customer data, it might be better to validate a model based on a smaller but indicative subset of attributes so as to not over fit your analytic models.

In the workbook, you’ll also find details about why other POCs fail, how to pick the first use case, how to define the right metrics for you and your stakeholders, the three pillars of big data management and how to apply them to your POC. We also provide a practical checklist for getting big data POCs right, and detail our schedule for the marketing big data POC that we successfully implemented.

Download “How to Run a Big Data POC in Six Weeks today.

In the next blog post, I’ll dive into how to turn successful POCs into production.