Research on Data Query Optimization Based on SparkSQL and MongoDB

摘要

With the arrival of the era of big data, the analysis and processing of massive data has become a very critical computing problem. This paper proposes a query optimization method based on SparkSQL and MongoDB. It analyzes the principle and compares it with other literature in order to draw the conclusion. The conclusion shows that when dealing with problems such as interactive SQL queries, the Apache Spark engine can reasonably decompose the tasks based on the dependencies between the massive data, thereby reducing the data query processing time and improving the operating efficiency. Also it is very suitable for storing some simple data with large amount due to flexible query and index of MongoDB. Obviously, the combination of the two can significantly improve the query speed of massive data.