Extraction, modeling, and predicting: a web driven approach for Taiwan stock prediction

摘要

In this research, the goal is to develop a framework for Web-driven data analysis. The Web contains a huge amount of data and allows easy access. However, performing analysis against data fetched from the Web is not an easy task due to its document-centric nature. The proposed framework covers three aspects of data analysis: extraction, modeling, and prediction. To deal with data extraction, we developed a Web browser simulator based on HtmlUnit. For modeling and prediction, we developed a lean algorithm in R, which is a programming language specifically designed for data science. Besides, we implemented a bridge library to integrate different tools utilized in this research. For proof of concept, we used the Taiwan stock market as our case study. In order to keep the framework as simple and straightforward as possible, we did not expect dependencies on any specific Web service APIs. Based on the process of implementing such a tool, it appears that with the proposed tool, the overhead of collecting Web data for analysis will be greatly eased.