A Systematic Literature Review for Implementing Data Ops in the Data Warehouse Lifecycle during the ETL Phase

Abstract

Nowadays, no one can deny the importance of Data Ware House (DWH) in all organizations. The most important components in Data Ware House (DWH) are the Extraction, Transformation, Loading (ETL) phase. Data cleaning is a basic piece of the transformation stage in Data Warehousing. This may affect critical activities such as data collection and decision-making in various organizations Data Ops is an evaluation technique of Dev Ops in the data domain. This study conducts a Systematic Literature Review (SLR) to assess the previous studies of data warehouses related to Data Ops efforts. This study collects 55 primary studies related to the detection of Data Scrubbing, Data Consistency, Data warehouse, Dev Ops and Data Ops and we have conducted a Systematic Literature Review (SLR). Based on these findings, we discuss many concerns related to the study of current approaches in terms of abstraction level, metrics used, implementation and validation. That is why the analysis covers the published efforts between 2016 and 2021 since Data Ops is a significantly new technique. The survey should cover only research that took plan in recent years. The result of the study observed that 29% of the studies focused on solving the importance of data quality in the data warehouse, 62% of them focused on related Dev Ops, only 9% focused on Data Ops techniques and no 0% survey on enhancing ETL phase with Data Ops. This SLR brings to the attention of the research community several opportunities for using Data Ops in future research and the nearly proposed model DW Ops.