Query grouping–based multi-query optimization frameworkrnfor interactive SQL query engines on Hadoop

摘要

In the past few years, executing high-concurrency queries with interactive SQL query engines onrnHadoop has become an important activity formany organizations.However, these systems do notrnadoptMulti-QueryOptimization (MQO) to accelerate the process. There are twomajor concerns.rnFirstly, traditional MQO researches assume that multiple queries have high similarity. However,rnthese systems usually serve a variety of applications.Although queries from the same applicationrnhave high similarity, queries from different applications may have low similarity, so using traditionalrnMQO will be inefficient and time consuming. Secondly, integrating MQO may lead to lotsrnof system modifications. To integrate MQO into interactive SQL query engines on Hadoop efficiently,rna query grouping–basedMQOframeworkis proposed.Alightweight mechanism is used tornrepresent SQLqueries, on which a grouping method is exploited to speed up the optimization process.rnA cost model is integrated to estimate the execution cost of interactive SQL query enginesrnon Hadoop.By using the proposed framework,wemodify Impala system to supportMQO, and thernexperimental results on TPC-DS show significant performance improvements.