Ah, the new year is almost upon us, so here are Todd’s 2014 predictions:
1) Real companies, not just internet advertising based businesses and startups, will stop just talking about big data and actually implement real solutions focused on 2 areas:
- Data warehouse offloading where companies take all of that data that is just sitting in their enterprise data warehouses and move the data they aren’t accessing into Hadoop where they will now preprocess the data in Hadoop and put that output into the EDW where they then report on it. The big driver for this? Cost savings.
- Predictive analytics based on collecting, integrating, cleansing and analyzing large amounts of sensor data. The data has always been available, it is just that analyzing it en masse has always been problematic. The big driver for this? Operational efficiency of the systems that are being monitored as well as feedback analysis to build the next generation of efficient systems.
That said, even more companies will still be talking about big data than are actually doing it. There is still a long learning curve.
2) Organizations will starting thinking about how to transition their data management infrastructure from a cost center into a profit center. As more companies identify ways they can take existing data about their customers, products, suppliers, partners etc., they will start identifying ways they can generate new revenue streams by repackaging this data into information products.
3) Data quality will continue to stink. This one is a sure thing. It never ceases to amaze me how people think that either their data isn’t bad, or that they can’t do anything about it or that thanks to big data, they don’t have to worry about data quality because of the law of large numbers. Laugh if you want at that last one on the law of large numbers, but I have heard that story at least three different times this year. Just for clarification for those of you who think that more data means that the dirty data becomes a statistical anomaly, it only means that you have the same percentage of dirty data as before…. You just have more of it.
4) More about big data… There still won’t be enough people who understand Hadoop. There will be lots of vendors (including Informatica) creating cool new tools so average developers and even business users can integrate, cleanse and analyze data on Hadoop without having to know anything about Hadoop. The hype might die down, but this will actually be an even more exciting area.
5) More business self service will come to the data integration space. With the proliferation of data, it is impossible for IT to service all of the integration needs of the business. So the only solution will be the growth of self-service integration capabilities that let business users and shadow IT do integration on their own. While this has existed already, the big change will be that corporate IT will start offering these services to their internal customers so departments can do their own integration but within a framework that is managed and supported by corporate IT. It is the very beginning of this trend, but expect to see IT start to get more control by giving control to their business users.
6) The Pittsburgh Steelers will make it back into the playoffs….. and for a 2015 prediction, my beloved Steelers will however not make it to the Super Bowl, but my adopted home team, the Santa Clara J 49ers, will make it to the Super Bowl and will win.
Happy New Year everyone.