Ensuring Data Quality in Loading and Viewing

Once Joshua Boger, CEO and founder of Vertex Pharmaceuticals said that, "I've never made a bad decision. I've just had bad data". This speaks for the importance of providing quality data to users for internal business decision making and external regulatory compliance. Data quality is essential for business intelligence success. Better data quality has a positive influence on sales, profit making, and value added. System quality has positive influence on data quality and information quality. During the load process, from operational source to the temporal data warehouse, performing transformation and loading from staging area to analytical area of data warehouse, utmost care must be taken to avoid data corruption. This data quality might be compromised for many reasons. While joining multiple source tables if join operators do not consider inequality predicates the integrity of timestamp-based data might be compromised. Data quality might be compromised due to lack of maintaining referential integrity as well. Data quality also might be compromised while retrieving data from analytical tables viewing information via reporting, business intelligence, and data mining tools. This could happen due to missing join criteria or bad transformation logic. All care must be taken to ensure the most accurate data retrieval. This is very true in the case of temporal data, as it is stored with different versions of time-referenced data.