Systemic Risk Document Classification on Indonesian News Articles using Deep Learning and Active Learning

摘要

Indonesian online news articles are growing fastly in this decade. One of the information is about economic news, including the information on financial systemic risk. In order to get information on financial systemic risk in real time, the task on systemic risk document classification should be done automatically. Here, we employ deep learning and active learning to classify systemic risk document automatically. We use 15 classes of financial systemic risk, such as defined before by Bank of Indonesia. The task is a multi-label classification, where a text document may contain more than 1 information of systemic risk. For the deep learning strategy, we've conducted several experiments of CNN, Bi-LSTM and Bi-GRU. We've also compared it with two steps of classification. In the experimental result, using 1752 documents as the training data and 228 documents as the testing data, the highest F1 score was achieved by using Bi-LSTM topology with one classification step and large common corpus as the resource for the word embedding. The highest F1 score was 45.37% for 15 classes with probability threshold defined as 0.15. In the two steps of classification, the first classification for 2 classes (contain risk information or not), the accuracy was 82.46%. To handle the limited data, we've conducted active learning to select the next candidate to be labeled as training data. In the experiment, for 420 new data with each iteration of 20 new data, the results showed that using active learning couldn't improve the performance.