RESEARCH OF THE EFFECTIVENESS OF SQL ENGINES WORKING IN HDFS

摘要

The rapid data growth at the beginning of the XXI century gave impetus to the development of big data technologies. A distributed platform Hadoop became a key element of big data technologies. Initially, it was difficult using Hadoop for tabular data processing, on which many modern industrial information systems are built. Therefore, a variety of SQL tools for Hadoop began to appear, which gave rise to the problem of choosing a specific solution. The aim of this work is to identify the most efficient SQL tools for tabular data processing in a distributed Hadoop system. For this purpose a comparative analysis of six most popular tools: Apache Hive, Cloudera Impala, Spark SQL, Presto, Apache Drill, Apache HAWQ has been done. The result of the study was the choice of the most effective means from the standpoint of completeness of the list of functions performed, tool performance and the level of SQL standards support. After summarizing of the results of a study, which has been done on all selected space coordinates comparison, Presto was the most effective tool.