Tag "Apache Spark"

Hadoop Mapreduce to Apache Spark : Data Storage and Processing Strategy Transformation

“In pioneer days they used oxen for heavy pulling, and when one ox couldn’t budge a log, they didn’t try to grow a larger ox. We shouldn’t be trying for bigger computers, but for more systems of computers” — Grace Hopper Data storage and processing technologies has gone through dramatic transformation from pre-stage flat-file system... more

Part 1: Add Spark to a Big Data Application with Text Search Capability

Text search is an essential operation in many applications dealing with semi-structured big data. One such application, which many of us know about, deals with program logs, which not only contains data for troubleshooting, but also often, other information helpful for understanding how the application operates under normal conditions, for diagnosing performance characteristics, snapshot of internal... more