A Comparative Review of Data Warehousing ETL Tools with New Trends and Industry Insight

摘要

Data Warehouse is a repository of strategic data from many sources gathered over a long period of time. Traditional DW operations mainly comprise of extracting data from multiple sources, transforming these data into a compatible form and finally loading them to DW schema for further analysis. The extract-transform-load (ETL) functions need to be incorporated into appropriate tools so that organisations can utilise these tools efficiently as required. There is a wide variety of such tools available in market. In this paper, we have compared different aspects of some popular ETL tools (Informatica, Datastage, Ab Initio, Oracle Data Integrator, SSIS) and have analysed their advantages and disadvantages. We have also highlighted some salient features (performance optimisation, data lineage, real time data analysis, cost, language binding etc.) of these tools and represented them with a comparative study. Apart from the review of the ETL tools, the paper also provides feedback from data science industry which narrates the market value and relevance of the tools in practical scenario. However, the traditional DW concept is expanding rapidly with the advent of big data, cloud computing, real time data analysis and the growing need of parsing information from structured and unstructured data sources. In this paper, we have also identified these factors which are transforming the definition of data warehousing.