Gartner hosted a webinar on January 10, 2012: Gartner Worldwide IT Spending Forecast. One of the topics covered was industry IT spend for 2012.
In covering that topic they made a point of saying that due to severe flooding in Thailand, they expect storage to become in short supply (as much as a 29% global shortfall) through the end of 2012. It is expected that the price of storage/GB will increase as a result and supplies will fall short of demand. They recommended finding alternatives to purchasing storage to keep costs down.
Purchasing more storage to manage database data growth is really a “band aid” solution to structured data growth management as one has to consider the fully burdened cost of storage, including electricity. Structured data growth management aka structured data archiving is an ideal solution as it solves the root cause of having to continue to increase your storage footprint and costs, in addition to several other benefits that it provides. Storage used for production environments typically have the best performance, availability, and disaster recovery capabilities and (as a result) are usually the most expensive storage devices.
Archiving moves inactive or dormant data off production storage onto less expensive storage. There are a few options with robust data archive solutions. One is to have “seamless access” where you archive directly to a database sitting on a less expensive tier of storage, with lower level SLAs (such as less frequent backups) applied to the archive database environment. Your users can log in to their native application using standard navigation (thus “seamless access”) and have access to both production and archived data. One of the more common use cases for this configuration is that the production database has an extremely high data growth rate and you want to be as aggressive as possible with your retention policies, leaving as little data in production as your users can live with. Your users will be more agreeable in that aggressive approach if you provide them “seamless access”.
You can then implement a true multi-tiered storage ILM solution by applying retention policies to the archive database and archiving it to a highly compressed (up to 98%) optimized archive file. As a file, the archive can sit on a relatively inexpensive of storage devices as it doesn’t need storage that supports a database. You can also archive directly to the highly compressed optimized archive file from production without implementing the “seamless access” configuration, eliminating the need to support the archive database (and the “middle” tier), if that scenario works best for you and your users. Access to the optimized archive file is done via ODBC/JDBC compliant reporting tools (most reporting or Business Intelligence tools support this protocol). A robust data archive solution has unique compliance features as well, such as automated enforcement of data retention/disposal policies. With storage prices expected to increase, you can archive to inexpensive storage AND reduce the data footprint by as much as 98%. With an industry best data compression of up to 98%, feature rich data archive solutions offer the best alternative to the “band aid” approach of throwing more storage at explosive data growth.
Once you have an archiving program in place and your production database has been optimized in size by archiving, there is a ripple effect out to any production clones you use for testing, development, etc. As an example, if you have a 1TB production database plus 4 copies, the total footprint is 5TB. After archiving off the dormant or inactive data, production is now 500 GB. Once you replicate the optimized production database out to the other instances, you now have a total footprint of 2.5 TB. This “ripple” effect provides additional operational and cost benefits in the non-production copies. There are also Test Data Management solutions that take this reduction in database footprint one step further. There are subset products that filter data according to the requirements of the user community (testers, developers, etc.) and can create minimally sized, optimal environments for non-production use that can significantly further reduce the database footprint of non-production instances. These solutions are relatively small maintenance and typically provide rapid return on investment (especially with storage cost on the rise – usually 6-12 months ROI).
By incorporating data archiving and subsetting (test data management), you will be optimized for structured data growth management minimizing the impact of storage costs and/or supply on your organization.