Machine learning for stock selection.

摘要

The Efficient Market Hypothesis (EMH) states that the prices of assets, e.g., stocks, already reflect all known information in the market and therefore are unpredictable. It has several forms. A commonly believed weak form of the EMH hypothesizes that the future stock price is completely unpredictable given the past trading history information of the stock. The weak-form EMH is challenged by some recent research. For example, Lehmann and Cooper suggest that there exists a potential predictable component in the past trading information. However, data snooping is used, or the profit after the trading costs is not evident, or both, indicating only a very weak regularity has been found. We believe that a strong regularity could be found by machine learning techniques. In our framework, we first learn the regularities or patterns in a tremendous amount of raw stock data points and then apply those patterns to the future stock data. If the learned patterns consistently earn excess profits in a long period, the stock price must be at least partly predictable, which contradicts the weak-form EMH. Clearly, the effectiveness of the machine learning method plays a key role in such a process. The general machine learning methods usually fail in stock prediction because the stock data are noisy and imbalanced. In this thesis, we describe a specially designed adaptive stock selection method called Prototype Ranking (PR). The primary target of PR is to select the top performing stocks among many ordinary stocks. PR is designed to perform learning and testing in a noisy stocks sample set where the top performing stocks are usually the minority. The experimental results show that PR produces a clear profit improvement compared to Lehmann's and Cooper's approaches. After taking into account reasonable trading costs, our model can still make a sizable profit over the wide range period of 1978 to 2004. The results seem to seriously challenge the weak form of EMH.;Another important contribution of this thesis is that we develop a stock selection method for risk-adjusted performance. The original PR method is return-optimized and can result in large volatility in the return. Such a method is usually considered risky. We develop an expanded version of PR called EPR. The EPR is designed to lower the risk of return by using a less aggressive pruning criterion and an ensemble prediction technique. The experimental result shows that EPR significantly increases the risk-adjusted performance of PR.;In addition, directly motivated from real-world trading, we also develop a new scheme for testing the learned patterns in the future data. The new scheme simulates the real stock trading process. It applies some intelligent techniques to decide whether a stock is worth being held. Compared with the original scheme, the new scheme eliminates many unprofitable transactions and significantly increases the total profits, especially when the transaction cost level is high.