Dimension Reduction on Open Data Using Variational Autoencoder

摘要

Open Data movement has led to large number of databases published in the web. However, effectively accessing these databases remain a challenge due to its large volume and heterogeneity. To retrieve similar queries, min-wise independent permutation locality sensitive hashing (MinHash LSH) became a popular technique to estimate the similarity between two domains. With the recent advancements in deep learning and its ability to revolutionize multiple fields of science, we explored how deep learning could improve similarity search on an internet scale. To do so, we first formulate the similarity search problem as a Euclidean nearest neighbour problem by transforming the set representation into a latent representation of learned features.We then apply Variational Autoencoders (VAEs) to embed domains into a significantly smaller, continuous latent dimension. VAEs learn an embedding that minimizes reconstruction error and the Kullback-Leibler divergence between the encoder and prior. Optimizing both terms allow the model to learn a dense representation with local similarities preserved from the original input space. We evaluate our algorithm using a subset of joint Open Data (Canada, US and UK) that contain more than 1.4 million documents with a domain size greater than 128 thousand. We demonstrate that the latent space correlates significantly with the Jaccard similarity coefficient. Then, we show domains that embed spatially closer in latent space are indeed similar. Lastly, we show that our algorithm outperforms MinHash LSH in accuracy and precision for all dimensions tested. These improvements show that deep learning techniques can be a promising approach for internet scale domain search.