Tag Archives: Information Lifecycle Management
Why Do You Need Database Archiving When You Already Have Database Compression?
In one of my earlier blogs, I wrote about why you still need database archiving, when you already partition your database. On a similar vein, many people also ask me why you still need to archive when you already have database compression to reduce your storage capacity and cost. The benefits of archiving, which you can’t achieve with just compression and/or partitioning are still the same:
- Archiving allows you to completely move data volumes out of the production system to improve response time and reduce infrastructure costs. Why keep unused data, even if compressed, on high cost server infrastructure when you don’t need to? Why add overhead to query processing when you can remove the data from being processed at all?
- Avoid server and software license upgrades. By removing inactive data from the database, you no longer require as much processing power and you can keep your existing server without having to add CPU cores and additional licenses for your database and application. This further eliminates costs.
- Reduce overall administration and maintenance costs. If you still keep unused data around in your production system, you still need to back it up, replicate it for high availability, clone it for non-production copies, recover it in the event of a disaster, upgrade it, organize and partition it, and consider it as part of your performance tuning strategy. Yes, it will take less time to backup, copy, restore, etc., since the data is compressed and is smaller, but why even include that data as part of production maintenance activities at all, if it’s infrequently used?
- Remove the multiplier effect. The cost of additional data volume in production systems is multiplied when you consider how many copies you have of that production data in mirrors, backups, clones, non-production systems, and reporting warehouses. The size multiplier is less since the data is compressed, but it’s still more wasted capacity in multiple locations. Not to mention the additional server, software license, and maintenance costs associated with the additional volumes in those multiple copies. So it’s best to just remove that data size at the source.
- Ensure compliance by enforcing retention and disposition policies. As I discussed in my previous blog on the difference between archiving and backup, archiving is the solution for long term data retention. Archiving solutions, such as Informatica Data Archive, have integration points with records management software or provide built-in retention management to enforce the retention of data for a specified period based on policies. During that period, the immutability and authenticity of the archived data is ensured, and when the retention period expires, records are automatically purged after the appropriate review and approval process. Regulated data needs to be retained long enough to comply with regulations, but keeping data for too long can also become a legal liability. So it’s important that expired records are purged in a timely manner. Just keeping data in production databases indefinitely doesn’t help you to reduce your compliance and legal risks.
Implementing enterprise application and database archiving is just plain best practices. The best way to improve performance and reduce infrastructure and maintenance costs is to reduce the data volume in your production systems. Why increase overhead when you don’t have to? Today’s archiving solutions allow you to maintain easy access to the data after archival, so there is no reason to keep data around just for the sake of accessibility. By moving inactive but regulated data to a central archival store, you can uniformly enforce retention policies. At the same time, you can reduce the time and cost of eDiscovery by making all types of data centrally and easily searchable.
Architecting A Database Archiving Solution
Many of my clients struggle with how to design a database archiving solution. Database archiving is not as clean as email or file archiving. Project owners who have done their research understand why they need an archiving solution: either to address performance degradation or increased costs (or both) due to uncontrolled data volume growth in their production databases. Where help is appreciated is during the planning phase of a project and defining what requirements are critical and how those requirements translate into an archive architecture.
Creating Subsets Of Test Data For Database Applications Addresses Key Issues
Test data sets need to be created to validate or confirm specific use cases during testing and development phases for packaged or custom database applications. Most companies use full copies of production data to seed test data sets. Using live, up to date data is preferable by Quality Assurance teams to increase confidence in the testing results. Two key issues with using full live data sets are increasing costs as well as introducing security risks.
Full Copies of Production Data Sets Increase Cost
As the data volumes grow, so does each copy of the data used in each test environment, increasing the cost of infrastructure required to store and maintain performance with larger data volumes and increasing the time it takes to complete testing cycles. According to the Enterprise Strategy Group, the number of secondary copies of production data sets required for development, testing and training is four (at a minimum). Multiply the size of the production data sets for each copy to get the total cost of ownership. With larger data sets, queries and reports take longer to complete. Many times, functional tests only require a small segment of data to validate a test. Subsets of test data would be adequate for most testing scenarios. (more…)
Streamline Your Move To Oracle Fusion Applications With Data Archiving
Oracle has been relatively quiet of late about Oracle Fusion Applications availability. But as the first applications are scheduled to rollout this year (2010), it might be a good time to revisit your upgrade to Fusion Applications plans. Is data archiving part a of it? If it isn’t, it should be.
(more…)
How Many Metadata Repositories Do You Have?
One of the key deliverables for an ILM project that my team is wrapping up is a metadata repository of all the database tables we have applied an ILM solution to. In this repository, we list the database, schema and table name; what Record Series the data belong to; the corresponding retention period and criteria; business owner; and source information. Not only will this repository be used to archive and purge data on a regular, operational basis, but it will also be used by Records Management to track Records Retention compliance.
(more…)
Adding A Dash Of ILM And MDM To A Bowl Of ECM And SAN Soup Makes Data Governance Taste Good
What do you get when you combine Information Lifecycle Management (ILM), Master Data Management (MDM), Enterprise Content Management (ECM), and Storage Area Networks (SAN)? A compelling Data Management solution on a single purchase order that finally closes the gap on databases. EMC announced this last week at EMC World 2010 that they will start reselling Informatica’s Data Archive and Master Data Management products as part of the EMC Select program. Why does this taste good to data management practitioners? Because EMC customers have been in need for a single solution to manage, retain and archive ALL data, not just email and files.
What Are Your Legacy Applications Costing You?
Has your application portfolio changed over the past 10 years? Have you migrated to newer systems and infrastructure to make your company more competitive? Has your organization gone through mergers or acquisitions where you ‘inherited’ additional applications? Over time, even though these applications may not be used to support current business processes, they are kept on life support simply for occasional access requirements or maybe to meet retention compliance.
(more…)
Information Stores That Persist Too Long
In a recent blog from Informatica’s Data Archive Product Manager, Claudia Chandra, she discusses the need for and the business value of retiring legacy applications. This is another excellent topic on how to keep what you need and delete everything else. Purge what is not needed and reduce cost. From a legal discussion perspective, The Sedona Conference has invested in a Working Group Series Publication on the very same topic. The working group on Electronic Document Retention and Production (WG1) shares their insight in the paper “The Sedona Conference® Commentary on Inactive Information Sources: Guidance Principles for Identifying, Classifying, Retaining and Destroying Orphaned, Legacy and Dormant ESI”.
(more…)
Does Database Archiving Fit In With Your Enterprise Content Management Strategy?
Enterprise Content Management (ECM) solutions have typically focused on managing electronic documents and unstructured content. What about structured data that is ‘locked up’ and traditionally retained indefinitely in your application databases? Can your existing ECM system help manage this content as well? Many people consider database content to be a completely different animal and not a good fit for storing in ECM repositories. However, with the increasing costs of electronic discovery (eDiscovery) during litigation, consider leveraging your ECM investment for managing the long-term retention of structured data.
Structured data archiving has proven to be a great way to trim production databases and get tremendous cost savings by aligning the lifecycle of the data with the underlying infrastructure. ECM adds value to this process by adding functionality in the area of content indexing, search and retention management. (more…)
Archiving And Purging
The main goals of any Application ILM project are to reduce cost, improve application performance while maintaining compliance. To meet these goals, data has to be moved from a production database to either an online, accessible archive or purged completely from the system. In either case, data is deleted from production. Deleting data can have a significant impact on the production system’s performance if not executed carefully. However, once the data is gone – the benefits have a ripple effect. Production tables are smaller and more manageable. Recovery times and maintenance windows can be reduced. The cost of managing a smaller production database is reduced proportionally with the amount of data removed.

