Automated Data Mining Platform for Large Volume Data Processing

摘要

A huge volume of user and media data has been generated during the daily operation in online media sites. In this circumstance, traditional database technology cannot fulfill the demand for the increasing data process capacity. In this paper a novel data processing model, i.e. Data Warehouse Schema Generator (DWSG), has been suggested to improve the data processing efficiency through utilizing snow model based on the dependency structure of dimension tables and dimensional hierarchy. The test-bed work has been conducted in Company A, an online Electronic Commerce service company.