Tag Archives: data subset
Everyone is worried about data security and privacy as they should be; for data to be trusted, users and management need confidence in not just knowing that data is correct, but also in knowing that it is secure and that access is permitted only in controlled situations. There is no shortage of security disaster stories, but I’m not worried about production data since it is at the heart of application management disciplines which, while still not perfect, have had 50 years to mature. This perspective is stated succinctly by Ronald Reagan when he spoke about the economy and said “I am not worried about the deficit. It is big enough to take care of itself.” (more…)
Test data sets need to be created to validate or confirm specific use cases during testing and development phases for packaged or custom database applications. Most companies use full copies of production data to seed test data sets. Using live, up to date data is preferable by Quality Assurance teams to increase confidence in the testing results. Two key issues with using full live data sets are increasing costs as well as introducing security risks.
Full Copies of Production Data Sets Increase Cost
As the data volumes grow, so does each copy of the data used in each test environment, increasing the cost of infrastructure required to store and maintain performance with larger data volumes and increasing the time it takes to complete testing cycles. According to the Enterprise Strategy Group, the number of secondary copies of production data sets required for development, testing and training is four (at a minimum). Multiply the size of the production data sets for each copy to get the total cost of ownership. With larger data sets, queries and reports take longer to complete. Many times, functional tests only require a small segment of data to validate a test. Subsets of test data would be adequate for most testing scenarios. (more…)
As the market continues to consolidate and companies sell off assets, not only are the physical assets separated and sold, so are the digital assets – or liabilities. John Schmidt covered it in one of his recent blogs.
When the information to be separated and sold resides in a database, you need to understand the data model and determine what master data, or reference data, and transactions belong to the new company. In the case of separating master data this may involve understanding the relationships between multiple systems to make sure that when the data is moved into the new company, it maintains context, is accurate and complete. With transactional data, you need to know the tables and rows that comprise a complete transaction so that when data is moved, no orphaned rows are left behind.