Roger Nolan

Roger Nolan
Roger Nolan is the Director of Solutions at Informatica. He focuses on the Architect community and next-generation architectures that will accelerate business value delivery. Before joining Informatica, Roger held a variety of senior roles in Product Marketing, Product Management, Strategic Alliances, and Corporate Development at Avaya, Sun Microsystems and Metricom. He has deep experience in enterprise software, communications & collaboration software, and internet telephony products. Roger has an MBA from Boston College and a BS from Northeastern University.

Are You Ready to Compete on Analytics?

Building a Data Competence for a Decision Ready Organization

There has been a lot of talk about “competing on analytics.”  And this year, for the third year in a row BI/Analytics is the top spending priority for CIOs according to Gartner.   Yet, the fact is that about half of all analytics projects do not deliver the expected results on time and on budget.  That doesn’t mean that the projects don’t show value eventually, but it’s harder and takes longer than most people think.

To compete on analytics is to establish a company goal to deliver actionable business insights faster and better than anybody in your industry – and possibly competitors who may be looking to jump industry boundaries as Google and Apple have already done several times.

This requires a competence in analytics and a competence in data management, which is the focus of this blog.  As an analytics manager at a healthcare company told me this week, “We suffer from beautiful reports built on crap data.”  Most companies do not yet established standard people, processes and technology for data management.  This is one of the last functional areas in most organizations where this is still true.  Sales, Marketing, and Finance standardized years ago.  It is only in the area of data management, which is shared by business and IT, that there is no real standardization. The result is unconnected silos of data,  long IT backlogs for data-related requests, and a process that is literally getting slower by the day as it gets overwhelmed by data volume and data complexity.

Analytics Use Cases and Data Requirements

It is worthwhile to think of the different broad use cases for analytics within an organization and what that means for data requirements.

Analytics & Innovation

  • Strategic Insights are the high level decisions a company must make. Better performing organizations are moving from “gut feel” to data-driven decision making.  The data for these large decisions needs to be as perfect as possible since the business costs of getting it wrong can be enormous.
  • Operational Insights require quick decisions to react to on-the-ground conditions. Here, the organization might be willing to sacrifice some data quality in order to deliver quick results. There is a speed versus expected benefit tradeoff to consider.
  • Analytics Innovation is the process of asking questions that were often never possible or economic to even ask before. Often, the first step is to see if there is any value in the question or hypothesis.  Here the data does not have to be perfect.  Often approximated data is “good enough” to test whether a question is worth pursuing further.  Some data scientists refer to this as “fail fast and move on quickly.”

The point here is that there is a tradeoff between speed of data delivery and the quality of the data that it is based on.  Managers do not want to be making decisions based on bad data, and analysts do not want to spend a high percentage of their time just defending the data.

The Need for Speed in Business Insight Delivery

We are moving from historical to predictive and proscriptive analytics.  Practically everybody has historical analysis, so while useful, it is not a market differentiator.  The biggest competitive payoff will come from the more advanced forms of analytics.  The need for speed as a market differentiator is built on the need provide service to customers in realtime and to make decisions faster than competitors.  The “half-life” of an analytics insight drops rapidly once competitors gain the same insight.

Analytics Speed Progression

Here are a couple of quick examples or predictive and proactive analytics:

  • Many retailers are looking to identify a customer coming in the door and have a dashboard in front of the customer service representative that will give them a full profile of the customer’s history, products owned, and positive/negative ratings about this product on social media.
  • In Sales, predictive analytics is being used today to recommend the “next best step” with a customer or what to upsell to that customer next and how to position it.
  • Beyond that, we are seeing and emerging class of applications and smart devices that will proactively recommend an action to users, without being asked, based on realtime conditions.

The data problems

The big problem is that the data internal to an organization was never designed to be discovered, access and shared across the organization. It is typically locked into a specific application and that application’s format requirements.  The new opportunity is the explosion of data external to the organization that can potentially enable questions that have never been possible to ask before.   The best insights and most differentiating insights will come from data sources across multiple disparate sources.  Often these sources are a mix of internal and external data.

Common data challenges for analytics:

  • The 2015 Analytics and BI survey by InformationWeek found that the #1 barrier to analytics is data quality. And this does not just mean that that data is in the right format. It must be complete, it must have business meaning and context, it must be fit for purpose, and if joined with another data set, it must be joined correctly.
  • The explosion of data volume and complexity.
  • More than 50% organizations use is coming from external sources (Gartner). This data is often less-structured, of unknown structure, and may have limited business context as to what the data means exactly.
  • The time-value of money. As mentioned earlier, the value of data and insights is eroding at increasing pace.
  • Data Discovery: Gartner estimates that the BI tool market is growing at 8% but says that the market could be growing much faster if issues around data discovery and data management were addressed.

Recommendations for the Decision Ready Organization

If you truly want to compete on analytics, you need to first create a competency center around data management.  Analytics is a great place to start.  First:

  • Break down the data & technology silos
  • Standardize on data management tools, processes, skills to the extent possible
  • Design so that all of your data is immediately discoverable, understandable, and shareable with any application or analytics project that might need it

Requiements

Pick industry-leading data management tools, or even better, tools that are integrated into a comprehensive data management platform.  Make sure that the platform:

  • Works with any data
  • Works with any BI tool
  • Works with any analytics storage technology
  • Supports all the analytics use cases: Strategic Decisions, Operational Decisions, and Innovation
  • Supports multiple delivery modes: business analyst self-service as well as the more traditional IT delivery of data managed by a formal data governance body.

The past focus on applications has resulted in hard-to-access data silos.  New technologies for analytics are causing some organizations to create new data silos in the search for speed for that particular project.  If your organization is serious about being a leader in analytics, it is time to put the focus required into leading-edge data management tools and practices to fuel insight delivery.

We are working with organizations such as EMC, and Fidelity that have done this.  You don’t have to do it all at once.  Start with your next important analytics projects.  Build it out the right way.  Then expand your competence to the next project.

For more information see:

“How to organize the data-ready enterprise”

“What it takes to deliver advanced analytics”

Share
Posted in Vibe | Tagged , , , , , | Leave a comment

Come to Informatica World 2015 and Advance Your Career

INFA15- Data Integration

Come to Informatica World 2015 and Advance Your Career

5 Reasons Data Integration Professionals Absolutely Must Not Miss This Informatica World.

If you are a Data Integration or Data Management professional, you really cannot afford to miss this event.  This year’s theme in the Data Integration track at Informatica World is all about customers.  Over 50 customers will be sharing their experiences and best practices for succeeding with for data integration projects such as analytics, big data, application consolidation and migration, and much more.

If you still need convincing, here are the five reasons:

  1. Big Data:A special Big Data Summit is part of the track.
  2. Immediate Value:over 50 customers will be sharing their experience and best practices. Things you can start doing now to improve your organization.
  3. Architecture for Business Transformation. An architecture track focused on practical approaches for using architecture to enable business transformation, with specific examples and real customer experiences.
  4. Hands on Labs:Everybody loves them. This year we have even more. Sign up early to make sure that you get your choice. They go fast!
  5. New “Meet the Experts” Sessions:These are small group meetings for business-level discussions around subjects like big data, analytics, application consolidation, and more.

This truly will be a one-stop shop for all things data integration at Informatica World.  The pace of both competition and technology change is accelerating.  Attend this event to stay on top of what is happening in the word of data integration and how leading companies and experts are using data for competitive advantage within their organizations.

To help start your planning, here is a listing of the Data Integration, Architecture, and Big Data Sessions this year.  I hope to see you there.

QUICK GUIDE

DATA INTEGRATION AND BIG DATA at INFORMATICA WORLD 2015

Breakout Sessions, Tuesday, May 13

Session Time Location
Accelerating Business Value Delivery with Informatica Platform  (Architect Track Keynote) 10:45am – 11:15am Gracia 6
How to Support Real Time Data Integration Projects with PowerCenter (Grant Thornton) 1:30pm – 2:30pm Gracia 2
Knowledgent 11:30am – 12:15pm Gracia 8
Putting Big Data to Work to Make Cancer History at MD Anderson Cancer Center (MD Anderson) 11:30am – 12:15pm Gracia 4
Modernize your Data Architecture for Speed, Efficiency, and Scalability 11:30am – 12:15pm Castellana 1
An Architectural Approach to Data as an Asset (Cisco) 11:30am – 12:15pm Gracia 6
Accenture 11:30am – 12:15pm Gracia 2
Architectures for Next-Generation Analytics 1:30pm – 2:30pm Gracia 6
Informatica Marketplace (Tamara Strifler) 1:30pm – 2:30pm Gracia 4
Informatica Big Data Ready Summit: Keynote Address (Anil Chakravarthy, EVP and Chief Product Officer) 1:40 – 2:25 Castellana 1
Big Data Keynote: Tom Davenport, Distinguished Professor in Management and Information Technology, Babson College 2:30 – 3:15 Castellana 1
How to Test and Monitor Your Critical Business Processes with PowerCenter (Discount Tire, AT&T) 2:40pm – 3:25pm Gracia 2
Enhancing Consumer Experiences with Informatica Data Integration Hub (Humana) 2:40pm – 3:25pm Gracia 4
Business Transformation:  The Case for Information Architecture (Cisco) 2:40pm – 3:25pm Gracia 6
Succeeding with Big Data and Avoiding Pitfalls (CapGemini, Cloudera, Cognizant, Hortonworks) 3:15 – 3:30
What’s New in B2B Data Exchange: Self-Service Integration of 3rd Party Partner Data (BMC Software) 3:35pm – 4:20pm Gracia 2
PowerCenter Developer:  Mapping Development Tips & Tricks 3:35pm – 4:20pm Gracia 4
Modernize Your Application Architecture and Boost Your Business Agility (Mototak Consulting) 3:35pm – 4:20pm Gracia 6
The Big Data Journey: Traditional BI to Next Gen Analytics (Johnson&Johnson, Transamerica, Devon Energy, KPN) 4:15 – 4:30 Castellana 1
L&T Infotech 4:30 – 5:30 Gracia 2
What’s New in PowerCenter, PowerCenter Express and PowerExchange? 4:30 – 5:30 Gracia 4
Next-Generation Analytics Architecture for the Year 2020 4:30 – 5:30 Gracia 6
Accelerate Big Data Projects with Informatica (Jeff Rydz) 4:35 – 5:20 Castellana 1
Big DataMichael J. Franklin, Professor of Computer Science, UC Berkeley 5:20 -5:30 Castellana 1
  • Informatica World Pavilion5:15 PM – 8:00 PM

Breakout Sessions, Wednesday, May 14

Session Time Location
How Mastercard is using a Data Hub to Broker Analytics Data Distribution (Mastercard) 2:00pm – 2:45pm Gracia 2
Cause: Business and IT Collaboration Effect: Cleveland Clinic Executive Dashboard (Cleveland Clinic) 2:00pm – 2:45pm Castellana 1
Application Consolidation & Migration Best Practices: Customer Panel (Discount Tire, Cisco, Verizon) 2:55pm – 3:55pm Gracia 2
Big Data Integration Pipelines at Cox Automotive (Cox Automotive) 2:55pm – 3:55pm Gracia 4
Performance Tuning for PowerCenter and Informatica Data Services 2:55pm – 3:55pm Gracia 6
US Bank and Cognizant 2:55pm – 3:55pm Castellana 1
Analytics architecture (Teradata, Hortonworks) 4:05pm – 4:50pm Gracia 4
A Case Study in Application Consolidation and Modernization—Migrating from Ab Initio to Informatica (Kaiser Permanente) 4:05pm – 4:50pm Castellana 1
Monetize Your Data With Hadoop and Agile Data Integration (AT&T) 4:05pm – 4:50pm Gracia 2
How to Enable Advanced Scaling and Metadata Management with PowerCenter (PayPal) 5:00pm – 5:45pm Castellana 1
How Verizon is consolidating 50+ legacy systems into a modern application architecture, optimizing Verizon’s enterprise sales and delivery process (Verizon) 5:00pm – 5:45pm Gracia 6
A guided tour to one of the most complex Informatica Installations worldwide (HP) 5:00pm – 5:45pm Gracia 2
Integration with Hadoop:  Best Practices for mapping development using Big Data Edition 5:00pm – 5:45pm Gracia 4

Meet The Experts Sessions, Wednesday, May 14

Session Time Location
Meet the Expert: App Consolidation – Driving Greater Business Agility and Reducing Costs Through Application Consolidation and Migration (Roger Nolan) 12:00pm – 12:50pm, 1:00pm – 1:50pm and 2:55pm – 3:55pm Castelena 2
Meet the Expert: Big Data – Delivering on the Promise of Big Data Analytics (John Haddad) 12:00pm – 12:50pm, 1:00pm – 1:50pm and 2:55pm – 3:55pm Castelena 2
Meet the Expert: Architect – Laying the Architectural Foundation for the Data-Driven Enterprise (David Lyle) 12:00pm – 12:50pm, 1:00pm – 1:50pm and 2:55pm – 3:55pm Castelena 2
  • Informatica World Pavilion11:45 PM – 2:00 PM

Breakout Sessions, Thursday, May 15

Session Time Location
Enterprise Architecture and Business Transformation Panel  (Cisco) 9:00am – 10:00am Gracia 6
The Data Lifecycle: From infancy through retirement, how Informatica can help (Mototak Consulting) 9:00am – 10:00am Gracia 4
How Allied Solutions Streamlined Customer Data Integration using B2B Data Exchange (Allied Solutions) 9:00am – 10:00am Gracia 2
How the State of Washington and Michigan State University are Delivering Integration as a Service (Michigan State University, Washington State Department of Enterprise Services) 9:00am – 10:00am Gracia 1
Real Time Big Data Streaming Analytics (PRA Group) 10:10am – 11:10am Gracia 1
Extending and Modernizing Enterprise Data Architectures (Philip Russom, TDWI) 10:10am – 11:10am Gracia 4
Best Practices for Saving Millions by Offloading ETL/ELT to Hadoop with Big Data Edition and Vibe Data Stream (Cisco) 10:10am – 11:10am Gracia 2
Retire Legacy Applications – Improve Your Bottom-Line While Managing Compliance (Cisco) 11:20am – 12:20pm Gracia 4
How a Data Hub Reduces Complexity, Cost and Risk for Data Integration Projects 11:20am – 12:20pm Gracia 1
Title? (Cap Gemini) 11:20am – 12:20pm Gracia 2
What’s New in PowerCenter, PowerCenter Express and PowerExchange? 2:30pm – 3:30pm Gracia 4
Title?  Keyur Desai 2:30pm – 3:30pm Gracia 2
How to run PowerCenter & Big Data Edition on AWS & connect Data as a Service (Customer) 2:30pm – 3:30pm Gracia 1
Accelerating Business with Near-Realtime Architectures 2:30pm – 3:30pm Gracia 6
  • Informatica World Pavillion12:30 PM – 3:30 PM

Hands-On Labs

Session Time Location
General Interest
PowerCenter 9.6.1 Upgrade 1 Table 01
PowerCenter 9.6.1 Upgrade 2 (repeat) Table 02
PowerCenter Advanced Edition – High Availability & Grid Mon 1:00, 3:00
Tue 7:30, 11:45, 2:40, 4:25
Wed 10:45, 12:45, 2:55, 5:00, 7:00
Thu 9:00, 11:20, 1:15
Fri 7:30, 9:30, 11:30
Table 03a
PowerCenter Advanced Edition – Metadata Manager & Business Glossary Mon 2:00, 4:00
Tue 10:45, 1:45, 3:35
Wed 7:30, 11:45, 2:00, 4:05, 6:00
Thu 7:30, 10:10, 12:15, 2:15
Fri 8:30, 10:30
Table 03b
Data Archive Mon 1:00, 3:00
Tue 7:30, 11:45, 2:40, 4:25
Wed 10:45, 12:45, 2:55, 5:00, 7:00
Thu 9:00, 11:20, 1:15
Fri 7:30, 9:30, 11:30
Table 06a
Test Data Management Mon 2:00, 4:00
Tue 10:45, 1:45, 3:35
Wed 7:30, 11:45, 2:00, 4:05, 6:00
Thu 7:30, 10:10, 12:15, 2:15
Fri 8:30, 10:30
Table 06b
Analytics- Related
PowerCenter Big Data Edition – Delivering on the Promise of Big Data Analytics All other times not taken by 11b. Table 11a
Elastic Analytics:  Big Data Edition in the Cloud Mon 4:00
Tue 11:45, 3:35
Wed 12:45, 5:00, 7:00
Thu  9:00;1:15;2:15
Fri 10:30
Table 11b
Greater Agility and Business-IT Collaboration using Data Virtualization Mon 1:00, 3:00
Tue 7:30, 11:45, 2:40, 4:25
Wed 10:45, 12:45, 2:55, 5:00, 7:00
Thu 9:00, 11:20, 1:15
Fri 7:30, 9:30, 11:30
Table 12a
Boosting your performance and productivity with Informatica Developer Mon 2:00, 4:00
Tue 10:45, 1:45, 3:35
Wed 7:30, 11:45, 2:00, 4:05, 6:00
Thu 7:30, 10:10, 12:15, 2:15
Fri 8:30, 10:30
Table 12b
Democratizing your Data through the Informatica Data Lake Table 13
Enabling Self-Service Analytics with Informatica Rev Table 14
Real-time Data Integration: PowerCenter Architecture & Implementation Considerations Monday 1pm
Tuesday 7:30am, 1:45pm
Wed 7:30, 2:00, 4:05
Thu 9am, 11:20am
Fri 8:30am
Table 15a
Real-time Data Integration: PowerExchange CDC on z/OS Monday 2pm
Tue 10:45, 2:40
Wed 10:45, 5pm
Thu 12:15pm
Fri 9:30am
Table 15b
Real-time Data Integration: PowerExchange CDC on i5/OS Monday 3pm
Tuesday 3:35pm
Wed 11:45am, 6pm
Thu 1:15pm
Fri 10:30am
Table 15c
Real-time Data Integration: PowerExchange CDC for Relational (Oracle, DB2, MS-SQL) Mon 4pm
Tue 11:45am, 4:25pm
Wed 12:45pm, 2:55pm, 7pm
Thu 7:30am, 10:10am, 2:15pm
Fri 7:30am, 11:30am
Table 15d
Healthcare Data Management and Modernization for Healthcare Providers Table 16
Data Management of Machine Data & Internet of Things Mon 1:00, 3:00
Tue 7:30, 11:45, 2:40, 4:25
Wed 10:45, 12:45, 2:55, 5:00, 7:00
Thu 9:00, 11:20, 1:15
Fri 7:30, 9:30, 11:30
Table 17a
Handling Complex Data Types with B2B Data Transformation Mon 2:00, 4:00
Tue 10:45, 1:45, 3:35
Wed 7:30, 11:45, 2:00, 4:05, 6:00
Thu 7:30, 10:10, 12:15, 2:15
Fri 8:30, 10:30
Table 17b
Application Consolidation & Migration Related
Simplifying Complex Data Integrations with Data Integration Hub Table 18
Implementing Trading Partner Integration with B2B Data Exchange Table 19
Operationalizing and Scaling your PowerCenter Environment Mon 1pm, 2pm
Tue 7:30, 10:45, 2:40, 3:35
Wed 10:45, 12:45, 5pm, 6pm, 7pm
Thu 7:30, 9am, 11:20, 1:15
Fri 7:30, 9:30, 11:30
Table 20a
Effective Operations management and Administration – What’s New M: 3:00 – 3:45pm
4:00 – 4:45pm
Tu: 11:45 – 12:30pm
1:45 – 2:30pm
4:25 – 5:15pm
W: 7:30 – 8:15am
11:45 – 12:30pm
2:55 – 3:40pm
4:05 – 4:50pm
Th: 10:10 – 10:55am
12:15 – 1:00pm
2:15 – 3:00pm
F:  8:30 – 9:15am
10:30 – 11:15am
Table 20b
Getting the Most out of your Data Integration & Data Quality Platform – Performance and Scalability Tips & Tricks Mon 1:00, 3:00
Tue 7:30, 11:45, 2:40, 4:25
Wed 10:45, 12:45, 2:55, 5:00, 7:00
Thu 9:00, 11:20, 1:15
Fri 7:30, 9:30, 11:30
Table 21a
Getting the Most out of your BigData Edition – Performance Best Practices Mon 2:00, 4:00
Tue 10:45, 1:45, 3:35
Wed 7:30, 11:45, 2:00, 4:05, 6:00
Thu 7:30, 10:10, 12:15, 2:15
Fri 8:30, 10:30
Table 21b
Modernizing and Consolidating Legacy and Application Data: Leveraging Data Services, Data Quality and Data Explorer Mon 1:00
Tue 10:45,  2:40, 4:25
Wed 10:45, 2:00, 2:55, 4:05, 7:00
Thu 11:20, 2:15 PM
Fri  9:30AM, 10:30AM
Table 22a
Connect to *: Connectivity to Long Tail of Next Generation Data Sources Mon 2:00, 3:00pm
Tue  7:30AM, 11:45, 1:45
Wed 7:30AM,  12:45pm,, 5:00pm
Thu 9:00am, 10:10am, 1:15pm
Fri 7:30am,8:30AM,
Table 22b
Modernizing and Consolidating Legacy and Application Data with PowerExchange Mainframe and CDC Mon 4:00PM
Tue 3:35
Wed 11:45, 6:00
Thu 7:30AM, 12:15, 2:15
Fri 11:30
Table 22c
Retire Legacy Applications and Optimize Application Performance with Informatica Data Archive Table 23
Protect Salesforce Sandboxes with Cloud Data Masking Tue 3:35, 4:25
Wed 6:00, 7:00
Thu 1:15, 2:15
Fri 7:30
Table 24a
Optimally Provision Test Data Sets with Test Data Management Mon: all times Monday
Tues: 7:30,10:45, 11:45, 1:45, 2:40
Wed: 7:30, 10:45, 11:45, 12:45, 2:00, 2:55, 4:05, 5:00
Thurs: 7:30, 9:00, 10:10, 11:20, 12:15
Fri: 8:30, 9:30, 10:30, 11:30
Table 24b
Share
Posted in Data Integration | Tagged , , , , , , | Leave a comment

Speed: The #1 IT Challenge

Agile Data Integration

Speed: The #1 IT Challenge

Speed is the top challenge facing IT today, and it’s reaching crisis proportions at many organizations.  Specifically, IT needs to deliver business value at the speed that the business requires.

The challenge does not end there; This has to be accomplished without compromising cost or quality. Many people have argued that you only get two out of three on the Speed/Cost/Quality triangle, but I believe that achieving this is the central challenge facing Enterprise Architects today.  Many people I talk to are looking at agile technologies, and in particular Agile Data Integration.

There have been a lot of articles written about the challenges, but it’s not all doom and gloom.  Here is something you can do right now to dramatically increase the speed of your project delivery while improving cost and quality at the same time: Take a fresh look you Agile Data Integration environment and specifically at Data Virtualization.  Data Virtualization offers the opportunity to simplify and speed up the data part of enterprise projects.  And this is the place where more and more projects are spending 40% and more of their time.  For more information and an industry perspective you can download the latest Forrester Wave report for Data Virtualization Q1 2015.

Here is a quick example of how you can use Data Virtualization technology for rapid prototyping to speed up business value delivery:

  • Use data virtualization technology to present a common view of your data to your business-IT project teams.
  • IT and business can collaborate in realtime to access and manage data from a wide variety of very large data sources – eliminating the long, slow cycles of passing specifications back and forth between business and IT.
  • Your teams can discover, profile, and manage data using a single virtual interface that hides the complexity of the underlying data.
  • By working with a virtualization layer, you are assured that your teams are using the right data and data that can by verified by linking it to a Business Glossary with clear terms, definitions, owners, and business context to reduce the chance of misunderstandings and errors.
  • Leading offerings in this space include data quality and data masking tools in the interface, ensuring that you improve data quality in the process.
  • Data virtualization means that your teams can be delivering in days rather than months and faster delivery means lower cost.

There has been a lot of interest in agile development, especially as it relates to data projects.  Data Virtualization is a key tool to accelerate your team in this direction.

Informatica has a leading position in the Forrester report due to the productivity of the Agile Data Integration environment but also because of the integration with the rest of the Informatica platform.  From an architect’s point of view it is critical to start standardizing on an enterprise data management platform.  Continuing data and data tool fragmentation will only slow down future project delivery.  The best way to deal with the growing complexity of both data and tools is to drive standardization within your organizations.

Share
Posted in 5 Sales Plays, Architects, CIO, Data Integration | Tagged , , , , , | Leave a comment

Stop Trying to Manage Data Growth!(?)

Data Downpour

Data Downpour

Talking to architects about analytics at a recent event, I kept hearing the familiar theme; data scientists are spending 80% of their time on “data wrangling” leaving only 20% for delivering the business insights that will drive the company’s innovation.  It was clear to everybody that I spoke to that the situation will only worsen.  The coming growth everybody sees in data volume and complexity, will only lengthen the time to value.

Gartner recently predicted that:

“by 2015, 50% of organizations will give up on managing growth and will redirect funds to improve classification and analytics.”

50 percent

“by 2015, 50% of organizations will give up on managing growth and will redirect funds to improve classification and analytics.”

Some of the details of this study are interesting.  In the end, many organizations are coming to two conclusions:

  • It’s risky to delete data, so they keep it around as insurance.
  • All data has potential business value, so more organizations are keeping it around for potential analytical purposes.

The other mega-trend here is that more and more organizations are looking to compete on analytics – and they need data to do it, both internal data and external data.

From an architect’s perspective, here are several observations:

  • The floodgates are open and analytics is a top priority. Given that, the emphasis should be on architecting to manage the dramatic increases in both data quantity and data complexity rather than on trying to stop it.
  • The immediate architectural priority has to be on simplifying and streamlining your current enterprise data architecture. Break down those data silos and standardize your enterprise data management tools and processes as much as possible.  As discussed in other blogs, data integration is becoming the biggest bottleneck to business value delivery in your environment. Gartner has projected that “by 2018, more than half the cost of implementing new large systems will be spent on integration.”  The more standardized your enterprise data management architecture is, the more efficient it will be.
  • With each new data type, new data tool (Hive, Pig, etc.), and new data storage technology (Hadoop, NoSQL, etc.) ask first if your existing enterprise data management tools can handle the task before people go out and create a new “data silo” based on the cool, new technologies. Sometimes it will be necessary, but not always.
  • The focus needs to be on speeding value delivery for the business. And the key bottleneck is highly likely to be your enterprise data architecture.

Rather than focusing on managing data growth, the priority should be on managing it in the most standardized and efficient way possible.  It is time to think about enterprise data management as a function with standard processes, skills and tools (just like Finance, Marketing or Procurement.)

Several of our leading customers have built or are building a central “Data as a Service” platform within their organizations.  This is a single, central place where all developers and analysts can go to get trustworthy data that is managed by IT through a standard architecture and served up for use by all.

For more information, see “The Big Big Data Workbook

*Gartner Predicts 2015: Managing ‘Data Lakes’ of Unprecedented Enormity, December 2014  http://www.gartner.com/document/2934417#

Share
Posted in Architects, CIO, Data Integration Platform | Tagged , , , , , , | Leave a comment

Three Things Every Architect Should Do for 2015

architect

Moving at the Speed of the Business

The start of the year is a great time to refresh and take a new look at your capabilities, goals, and plans for your future-state architecture.  That being said, you have to take into consideration that the most scarce resource in your architecture is probably your own personal time.

Looking forward, here are three things that I would recommend that every architect do.  I realize that all three of these relate to data, but as I have said in the eBook, Think “Data First” to Drive Business Value, we believe that data is the key bottleneck in your enterprise architecture in terms of slowing the delivery of business initiatives in support of your organization’s business strategy.

So, here are the recommendations.  None of these will cost you anything if you are a current Informatica PowerCenter customer.  And #2 and #3 are free regardless.  It is only a matter of your time:

1. Take a look at the current Informatica Cloud offering and in particular the templating capabilities.

Informatica Cloud is probably much more capable than you think.  The standard templating functionality supports very complex use cases and does it all from a very easy to use, no-coding, user interface.  It comes with a strong library of integration stubs that can be dragged & dropped into Microsoft Viseo to create complex integrations.  Once the flow is designed in Viseo, it can be easily imported into Informatica Cloud and from there users have a Wizard-driven UI to do the final customization for sources, targets, mappings, transformations, filters, etc.  It is all very powerful and easy to use.

Resources

Why This Matters to Architects

  • You will see how easy it is for new groups to get going with fairly complex integrations.
  • This is a great tool for departmental or new user use, and it will be completely compatible with the rest of your Informatica architecture – not another technology silo for you to manage.
  • Any mapping created for Informatica on-premise can also run on the cloud version.

2. Download Informatica Rev and understand what it can do for your analysts and “data wranglers.”

Your data analysts are spending 80% of their time managing their data and only 20% on the actual analysis they are trying to provide.  Informatica Rev is a great way to prepare your data before use in analytics tools such as Qlik, Tableau, and others.

With Informatica Rev, people who are not data experts can access, mashup, prototype and cleanse their data all in a User Interface that looks like a spreadsheet and requires no previous experience in data tools.

Resources

Why  This Matters for Architects

  • Your data analysts are going to use analytics tools with or without the help of IT. This enables you to help them while ensuring that they are managing their data well and optimizing their productivity.
  • This tool will also enable them to share their “data recipes” and for IT to be involved in how they access and use the organization’s data.

 

3. Look at the new features in PowerCenter 9.6.  First, upgrade to 9.6 if you haven’t already, and particularly take a good look at these new capabilities that are bundled in every version. Many people we talk to have 9.6 but don’t realize the power of what they already own.

  1. Profiling: Discover and analyze your data quickly.  Find relationships and data issues.
  2. Data Services: This presents any JDBC or ODBC repository as a logical data object. From there you can rapidly prototype new applications using these logical objects without worrying about the complexities of the underlying repositories. It can also do data cleansing on the fly.

Resources

 

Why This Matters for Architects

  • The key challenge for IT and for Architects is to be able to deliver at the “speed of business.” These tools can dramatically improve the productivity of your team and speed the delivery of projects for your business “customers.”

Taking the time to understand what these tools can do in terms of increasing the productivity of your IT team and enabling your end users to self-service will make you a better business partner overall and increase your influence across the organization.  Have a great year!

Share
Posted in Architects, CIO | Tagged , , , , | Leave a comment

Rising DW Architecture Complexity

Rising DW Architecture Complexity

Rising DW Architecture Complexity

I was talking to an architect-customer last week at a company event and he was describing how his enterprise data warehouse architecture was getting much more complex after many years of relative calm and stability.  In the old days of yore, you had some data sources, a data warehouse (with single database), and some related edge systems.

The current trend is that new types of data and new types of physical storage are changing all of that.

When I got back from my trip I found a TDWI white paper by Philip Russom that describes the situation very well in a white paper detailing his research on this subject;  Evolving Data Warehouse Architectures in the Age of Big Data.

From an enterprise data architecture and management point of view, this is a very interesting paper.

  • First the DW architectures are getting complex because of all the new physical storage options available
    • Hadoop – very large scale and inexpensive
    • NoSQL DBMS – beyond tabular data
    • Columnar DBMS – very fast seek time
    • DW Appliances – very fast / very expensive
  • What is driving these changes is the rapidly-increasing complexity of data. Data volume has captured the imagination of the press, but it is really the rising complexity of the data types that is going to challenge architects.
  • But, here is what really jumped out at me. When they asked the people in their survey what are the important components of their data warehouse architecture, the answer came back; Standards and rules.  Specifically, they meant how data is modeled, how data quality metrics are created, metadata requirements, interfaces for data integration, etc.

The conclusion for me, from this part of the survey, was that business strategy is requiring more complex data for better analyses (example: realtime response or proactive recommendations) and business processes (example: advanced customer service).  This, in turn, is driving IT to look into more advanced technology to deal with different data types and different use cases for the data.  And finally, the way they are dealing with the exploding complexity was through standards, particularly data standards.  If you are dealing with increasing complexity and have to do it better, faster and cheaper, they only way you are going to survive is by standardizing as much as reasonably makes sense.  But, not a bit more.

If you think about it, it is good advice.  Get your data standards in place first.  It is the best way to manage the data and technology complexity.  …And a chance to be the driver rather than the driven.

I highly recommend reading this white paper.  There is far more in it than I can cover here. There is also a Philip Russom webinar on DW Architecture that I recommend.

Share
Posted in Architects, CIO | Tagged , , , , , , , , , , | Leave a comment

CES, Digital Strategy and Architecture: Are You Ready?

CES, Digital Strategy and Architecture

CES, Digital Strategy and Architecture

CES, the International Consumer Electronics show is wrapping up this week and the array of new connected products and technologies was truly impressive. “The Internet of Things” is moving from buzzword to reality.  Some of the major trends seen this week included:

  • Home Hubs from Google, Samsung, and Apple (who did not attend the show but still had a significant impact).
  • Home Hub Ecosystems providing interoperability with cars, door locks, and household appliances.
  • Autonomous cars, and intelligent cars
  • Wearable devices such as smart watches and jewelry.
  • Drones that take pictures and intelligently avoid obstacles.  …Including people trying to block them.  There is a bit of a creepy factor here!
  • The next generation of 3D printers.
  • And the intelligent baby pacifier.  The idea is that it takes the baby’s temperature, but I think the sleeper hit feature on this product is the ability to locate it using GPS and a smart phone. How much money would you pay to get your kid to go to sleep when it is time to do so?

Digital Strategies Are Gaining Momentum

There is no escaping the fact that the vast majority of companies out there have active digital strategies, and not just in the consumer space. The question is: Are you going to be the disruptor or the disruptee?  Gartner offered an interesting prediction here:

“By 2017, 60% of global enterprise organizations will execute on at least one revolutionary and currently unimaginable business transformation effort.”

It is clear from looking at CES, that a lot of these products are “experiments” that will ultimately fail.  But focusing too much on that fact is to risk overlooking the profound changes taking place that will shake out industries and allow competitors to jump previously impassible barriers to entry.

IDC predicted that the Internet of Things market would be over $7 Trillion by the year 2020.  We can all argue about the exact number, but something major is clearly happening here.  …And it’s big.

Is Your Organization Ready?

A study by Gartner found that 52% of CEOs and executives say they have a digital strategy.  The problem is that 80% of them say that they will “need adaptation and learning to be effective in the new world.”  Supporting a new “Internet of Things” or connected device product may require new business models, new business processes, new business partners, new software applications, and require the collection and management of entirely new types of data.  Simply standing up a new ERP system or moving to a cloud application will not help your organization to deal with the new business models and data complexity.

Architect’s Call to Action

Now is the time (good New Year’s resolution!) to get proactive on your digital strategy.  Your CIO is most likely deeply engaged with her business counterparts to define a digital strategy for the organization. Now is the time to be proactive in terms of recommending the IT architecture that will enable them to deliver on that strategy – and a roadmap to get to the future state architecture.

Key Requirements for a Digital-ready Architecture

Digital strategy and products are all about data, so I am going to be very data-focused here.  Here are some of the key requirements:

  • First, it must be designed for speed.  How fast? Your architecture has to enable IT to move at the speed of business, whatever that requires.  Consider the speed at which companies like Google, Amazon and Facebook are making IT changes.
  • It has to explicitly directly link the business strategy to the underlying business models, processes, systems and technology.
  • Data from any new source, inside or outside your organization, has to be on-boarded quickly and in a way that it is immediately discoverable and available to all IT and business users.
  • Ongoing data quality management and Data Governance must be built into the architecture.  Point product solutions cannot solve these problems.  It has to be pervasive.
  • Data security also has to be pervasive for the same reasons.
  • It must include business self-service.  That is the only way that IT is going to be able to meet the needs of business users and scale to the demands of the changes required by digital strategy.

Resources:

For a webinar on connecting business strategy to the architecture of business transformation see; Next-Gen Architecture: A “Business First” Approach for Agile Architecture.   With John Schmidt of Informatica and Art Caston, founder of Proact.

For next-generation thinking on enterprise data architectures see; Think “Data First” to Drive Business Value

For more on business self-service for data preparation and a free software download.

Share
Posted in Architects, CIO, Data Integration Platform, Enterprise Data Management | Tagged , , , , , , , , | Leave a comment

Adding Big Data to Your EDW Architecture

Adding Big Data to Your EDW Architecture

Adding Big Data to Your EDW Architecture

As you think forward towards how you will use Big Data to compliment your current enterprise data warehouse (EDW) environment, check out the excellent webinar by Ralph Kimball and Matt Brandwein of Cloudera:

Webinar: Building a Hadoop Data Warehouse: Hadoop 101 for EDW Professionals

A couple comments on the importance of integration platforms like Informatica in an EDW/Hadoop environment.

  • Hadoop does mean you can do some quick and inexpensive exploratory analysis with little or no ETL.  The issue is that it will not perform at the level you need to take it to production.  As the webinar points out, applying some structure to the data with columnar files (not RDBMS) will dramatically speed up query performance.
  • The other thing that makes an integration platform more important than ever is the explosion of data complexity.    As Dr. Kimball put it: 

“Integration is even more important these days because you are looking at all sorts of data sources coming in from all sorts of directions.” 

To perform interesting analyses, you are going to have to be able to join data with different formats and different semantic meaning.  And that is going to require integration tools.

  • Thirdly, if you are going to put this data into production, you will want to incorporate data cleansing, metadata management, and possibly formal data governance to ensure that your data is trustworthy, auditable, and has business context.  There is no point in serving up bad data quickly and inexpensively.  The result will be poor business decisions and flawed analyses.

For Data Warehouse Architects

The challenge is to deliver actionable content from the exploding amount of data available.  You will need to be constantly scanning for new sources of data and looking for ways to quickly and efficiently deliver that to the point of analysis.

For Enterprise Architects

The challenge with adding Big Data to Your EDW Architecture is to define and drive a coherent enterprise data architecture across your organization that standardizes people, processes, and tools to deliver clean and secure data in the most efficient way possible.  It will also be important to automate as much as possible to offload routine tasks from the IT staff.  The key to that automation will be the effective use of metadata across the entire environment to not only understand the data itself, but how it is used, by whom, and for what business purpose.  Once you have done that, then it will become possible to build intelligence into the environment.

For more on Informatica’s vision for an Intelligent Data Platform and how this fits into your enterprise data architecture see Think “Data First” to Drive Business Value

Share
Posted in Architects, Big Data, CIO, Data Warehousing | Tagged , , , , | Leave a comment

Getting to Your Future State Enterprise Data Architecture

Getting to Your Future State Enterprise Data Architecture

Future State Enterprise Data Architecture

Just exactly how do your move from a “Just a Bunch of Data” (JBOD) architecture to a coherent enterprise data architecture?

The white paper, “The Great Rethink: Building a Highly Responsive and Evolving Data Integration Architecture” by Claudia Imhoff and Joe McKendrick provides an interesting view of what such an architecture might look like.  The paper describes how to move from ad hoc Data Integration to an Enterprise Data Architecture.  The paper also describes an approach towards building architectural maturity and a next-generation enterprise data architecture that helps organizations to be more competitive.

Organizations that look to compete based on their data are searching for ways to design an architecture that:

  • On-boards new data quickly
  • Delivers clean and trustworthy data
  • Delivers data at the speed required of the business
  • Ensures that data is handled in secure way
  • Is flexible enough to incorporate new data types and new technology
  • Enables end user self-service
  • Speeds up the speed of business value delivery for an organization

In my previous blog, Digital Strategy and Architecture, we discussed the demands that digital strategies are putting on enterprise data architecture in particular.  Add to that the additional stress from business initiatives such as:

  • Supporting new mobile applications
  • Moving IT applications to the cloud – which significantly increases data management complexity
  • Dealing with external data.  One recent study estimates that a full 25% of the data being managed by the average organization is external data.
  • Next-generation analytics and predictive analytics with Hadoop and No SQL
  • Integrating analytics with applications
  • Event-driven architectures and projects
  • The list goes on…

The point here is that most people are unlikely to be funded to build an enterprise data architecture from scratch that can meet all these needs.  A pragmatic approach would be to build out your future state architecture in each new strategic business initiative that is implemented.  The real challenge of being an enterprise architect is ensuring that all of the new work does indeed add up to a coherent architecture as it gets implemented.

The “Great Rethink” white paper describes a practical approach to achieving an agile and responsive future state enterprise data architecture that will support your strategic business initiatives.  It also describes a high level data integration architecture and the building blocks to achieving that architecture.  This is highly recommended reading.

Also, you might recall that Informatica sponsored the Informatica Architect’s Challenge this year to design an enterprise-wide data architecture of the future.  The contest has closed and we have a winner.  See the site for details, Informatica Architect Challenge .

Share
Posted in Architects, CIO, Data First, Data Integration Platform | Tagged , , , , , | 2 Comments