In-memory distributed software solution to improve the performance of recommender systems

摘要

Many recommender systems are currently available for proposing content (movies, TV series, music, etc.) to users according to different profiling metrics, such as ratings of previously consumed items and ratings of people with similar tastes. Recommendation algorithms are typically executed by powerful servers, as they are computationally expensive. In this paper, we propose a new software solution to improve the performance of recommender systems. Its implementation relies heavily on Apache Spark technology to speed up the computation of recommendation algorithms. It also includes a webserver, an API REST, and a content cache. To prove that our solution is valid and adequate, we have developed a movie recommender system based on two methods, both tested on the freely available Movielens and Netflix datasets. Performance was assessed by calculating root-mean-square error values and the times needed to produce a recommendation. We also provide quantitative measures of the speed improvement of the recommendation algorithms when the implementation is supported by a computing cluster. The contribution of this paper lies in the fact that our solution, which improves the performance of competitor recommender systems, is the first proposal combining a webserver, an API REST, a content cache and Apache Spark technology. Copyright (C) 2016 John Wiley & Sons, Ltd.