Perspectives

ELT stands for Extract Load and Transform and is a technique for moving and transforming data from one location and format to another instance and format. In this style of integration, the target DBMS becomes the transformation engine. This is in contrast to traditional ETL, or Extract Transform and Load which is the traditional technique for moving and transforming data in which an ETL engine that is separate from either the source or target DBMS performs the data transformations. Many of the discussions I’ve read on the topic of ETL vs. ELT focus on discussions like how does one work vs. the other and when I should one use a particular approach. I’d like to talk about a few of those questions, but first I’d like to tackle a slightly different question: why should you care about ELT? (more…)

Within the first two weeks of any undergraduate algorithms class, students learn about the Traveling Salesman Problem. I’ll let the web fill you in on the gory details of the problem, kindly refer you to Dykstra for one tidy solution and cut straight to the lesson: often the solution that offers the initial lowest cost turns out to be the most expensive solution in the end. The problem is designed to teach neophyte computer scientists the lesson of planning ahead, typically all the way to the final end state, before deciding on a solution. The solution is applicable to everything from database query optimization, to optimal shoe-lacing (if you’re into that) and yes, even to the optimum data integration architectures.

Enterprises are asking a lot of their data and their IT organizations these days. Smart organizations understand that harnessing data is more than just generating stale reports that never get read. Timely, trustable data, available when and where it’s needed, in the right form for the job at hand can drive top-line revenue growth, improve operating margins, facilitate M&A integration and satisfy auditors performing compliance checks. However, few companies possess the foresight to architect their systems to deliver all of these benefits. (more…)