Prioritized Sampling Method for Autoencoder to Reduce Loss Rate for Skewed Data

摘要

With the development of machine learning and deep learning in recent years, many topics in production and industry have used these technologies. In particular, problem solving using deep learning is often been used. However, in real life, considering the actual reality problem, there are a lot of data that is skewed, which means that the data is not so uniform, which results in the inability to obtain correct features when using machine learning to train, or to cause the error value is extremely high and not evenly distributed in this part of the data, for this reason we will lose the meaning of machine learning. Especially when using Autoencoder to do low dimensional compression or feature extraction, skewed data will lead to Autoencoder can not well compress the characteristics of this part of the skewed data. In this research, we improved the general Denoising Autoencoder, used a prioritized sampling method to solve the problem bring from skewed data.