一种基于SCA的ETL架构的设计和实现

摘要

针对传统ETL（Extraction，Transformation，Loading）流程的开发周期长的问题，提出一种基于SCA（Service Component Ar-chitecture）的ETL架构。该架构将传统的ETL具体流程实现分为两大类组件：第一大类组件解决数据质量问题，而第二大类组件解决维度建模问题。根据ETL各流程中不同问题的特点，选择适合的ETL工具及实现技术制作其相应组件，并通过SCA容器及其装配机制把组件发布成Web服务，使得不同的ETL工具及实现之间可以相互调用，同时提高复用性和扩展性。给出实例验证了该架构的有效性。%To shorten the long development cycle of traditional ETL process,we propose an SCA-based ETL architecture.This architecture divides the specific process of traditional ETL into two types of components:the first type mainly solves the problem of data quality,and the second one solves the problem of dimensional modelling.Appropriate ETL tools and implementation technique are selected according to the characteristics of the different issues in ETL processes to develop the corresponding components,and these components are then published as Web service through SCA container and its assembly mechanism,so that various ETL tools and implementations can call each other while im-proving the reusability and scalability.An application example given in end of the paper verifies the effectiveness of the architecture.