Tag Archives: Best Practices
Last week I described how Informatica Identity Resolution (IIR) can be used to match data from different lists or databases even when the data includes typos, translation mistakes, transcription errors, invalid abbreviations, and other errors. IIR has a wide range of use cases. Here are a few. (more…)
Whether you are establishing a new outsourced delivery model for your integration services or getting ready for the next round of contract negotiations with your existing supplier, you need a way to hold the supplier accountable – especially when it is an exclusive arrangement. Here are four key metrics that should be included in the multi-year agreement. (more…)
If you have been following publications in the Potential at Work Community or any number of Linkedin discussions such this one on the DrJJ group (a think-tank for information management best practices), you will have noticed the Agile methodology topic come up time and time again. For instance, check out the article Architect Your Way From Sluggish to Speed or the video Focus on Agility Adaptability. It hasn’t always been this way. For many years the architectural focus was on RASP.
In previous posts, we introduced the concept of the Informatica ILM Nearline and discussed how Informatica ILM Nearline could help your business. To recapitulate: the major advantage of Informatica ILM Nearline is its superior data access performance, which enables a more aggressive approach to migrating huge volumes of data out of the online repository to an accessible, highly compressed archive (on inexpensive 2nd and 3rd tier storage infrastructure).
Today, I will be considering the question of when an enterprise should consider implementing Informatica ILM Nearline. Broadly speaking, such implementations fall into two categories: they either offer a “cure” for an existing data management problem or represent a proactive implementation of data best practices within the organization.
Cure or Prevention?
The “cure” type of implementation is typically associated with a data warehouse or business application “rescue” project. This is undertaken when the production system grows to a point where database size causes major performance problems and affects the ability to meet Service Level Agreements (SLAs) and manage business processes in a timely manner. In these kinds of situation, it is mainly the operations division of the organization that is affected, and who demand an immediate fix that can take the form of an Informatica ILM Nearline implementation. The question here is: How quickly can the “cure” implementation stabilize performance and ensure satisfaction of SLAs?
On the other hand, the best practice approach, much like current practices related to healthy living, focuses on prevention rather than on curing. In this respect, best practices dictate that the Informatica ILM Nearline implementation should start as soon as some of the data in the production system becomes “infrequently accessed”, or “cold”. In data warehouses and data marts where the current month or two is being analyzed most often, this means data older than 90 days. For transactional systems the archiving cutoff may be a year or two, depending on typical length of your business processes. The main idea is to keep the size production databases from inflating for no good business reason and ‘nearlining’ the data as soon as possible without interrupting business operations or hurting the value of your data. Ultimately this should work to protect the enterprise from an operational crisis arising from deteriorating performance and unmet SLAs.
In order to better judge the impact of using either of these two approaches, it is important to understand the various steps involved in the “Nearlining” process. What do we find when we “dissect” the process of leveraging the Informatica ILM Nearline?
Dissecting the “Informatica ILM Nearline” Process
Informatica Informatica ILM Nearline involves multiple processes, whose performance characteristics can significantly influence the speed at which data is migrated out of the online database. The various processes are managed by the overall integrated nearline solution of Informatica coupled with a SAP Business Warehouse system:
- The first step is to lock the data that is targeted by the archiving process, in order to ensure that the data is not modified while the process is going on. SAP Business Warehouse does it automatically and you execute Data Archive Processes (DAP) for the cold data.
- Next comes the extraction of the data to be migrated. This is usually achieved via an SQL statement based on business rules for data migration. Often, the extraction can be performed using multiple extraction/consumer processes working in parallel.
- The next step is to secure the newly extracted data, so that it is recoverable.
- Then, the integrity of the extracted data must be validated (normally by comparing it to its online counterpart).
- Next, delete the online data that has been moved to nearline.
- Then, reorganize the tablespace of the deleted data.
- Finally, rebuild/reorganize the index associated with the online table from which data has been nearlined.
The Database Housekeeping process is often the slowest part of a Data Nearlining process, and thus can dictate the pace and scheduling of the implementation. In a production environment, the database housekeeping process is frequently decoupled from ongoing operations and performed over a weekend. It may be surprising to learn that deleting data can be a more expensive process than inserting it, but just ask an enterprise DBA about what is involved in deleting 1 TB from an Enterprise Data Warehouse and see what answer you get: for many, the task of fitting such a process into standard Batch Windows would be a nightmare.
So, it is easy to see that starting earlier in implementing Informatica ILM Nearline as a best practice can help to massively reduce not only the cost of the implementation, but also the time required to perform it. Therefore, the main recommendation to take away from this discussion is: Don’t wait too long to consider embarking on your Informatica ILM Nearline strategy!
That’s it for today. In my next post, I will take up the topic of which data should be initially considered as a candidate for migration.
If you haven’t already, check out the Potential at Work for Information Leaders site. We’ve just posted three great new articles designed to help you be more successful:
- “Driving value without locking down your data” Securing your data doesn’t mean inhibiting its use – far from it. Did you know that effective data masking practices allow information leaders to optimize the value data delivers to the organization while ensuring its security? Some forward-thinking information leaders are doing this and getting great results.
- “How fresh is your data?” Simply delivering data is not good enough anymore. You must get it to the right people at the right time while it is still fresh enough to be useful. Find out how to do it right.
- “Turn an application data migration initiative into a data governance pilot” A data migration effort can accomplish so much more than simply transferring data. Think about using it as an opportunity to improve the quality of existing data and apply new, higher standards to the information powering your organization.
Don’t miss out on topics that are key to your success. Please join the Potential at Work for Information Leaders community today. Available in nine languages, this site will continue to feature fresh, new ideas to promote the value of information management from a variety of top technology leaders.Sign up now!
We are excited to announce the new Potential at Work Community for Application Leaders.
As an application leader, you have a very demanding job. You have to successfully manage issues such as:
- Driving the maximum business value from your company’s enterprise application investments
- Keeping all of your enterprise applications current and meeting user requirements
- Delivering on your service agreements and managing all of the “ilities.”
- Defining an enterprise application strategy that includes on-premise and cloud
- Delivering timely, authoritative and trustworthy data for your enterprise applications
This community is here to help you to do exactly that and to help you to excel in both your current job and your career ahead. Our goal is to provide tips, insights, best practices and information from experts to help you become more successful.
Our first edition is focused on the theme of managing an enterprise cloud application strategy. For those who are in the process of selecting cloud application vendors, I’ve included a very handy Vendor Selection Checklist that is used by Informatica’s Vice President of Applications.
Are we interested in your input to the community? Absolutely! If you have an idea or content to share with the community, please contact us and we will get you published.
Join the community and start unleashing your potential by clicking on this link:
Roger Nolan email@example.com
Julie Lockner firstname.lastname@example.org
Click here for more information on the Potential at Work communities.
During Informatica World in early June, we were excited to announce our new Potential at Work Community. You can read Jakki Geiger’s blog introducing the Community to learn more about the goals for this great resource. (more…)
So might read the subject line in a memo to business users from IT staff responsible for implementing a new application system. Changing requirements in a project is one of the most frustrating (for both business and IT staff) and time-consuming activities in a large project; so much so that sometimes it is the cause of massive project delays or even cancellation. But there is something wrong with the subject line; it presumes that the business users are to blame. They are not. Let’s explore the real root causes and the solutions to them. (more…)
I’ve been in the data management industry for over 20 years, and you’ve always been very supportive of my career – even though you admit you have no clue what it is I do for a living. So here’s my best shot at explaining what I do so you can more accurately brag about me to your friends! J
If you are asked “what is the biggest application in your organization”, what would you say? If you’re in banking you might say it’s the Hogan deposit system. If you’re in Telecom maybe it’s the Amdocs Customer Care and Billing system. If you’re in retail, you might say the Retek Merchandizing system. If you are a manufacturer, it might be your SAP ERP system. The list goes on, but you get the point. The prevailing perception is that the core business application of whatever industry you are in is the biggest application. But this is a case where perception is not reality. (more…)