Modern database clusters entail two levels of networks: connecting CPUs andNUMA regions inside a single server in the small and multiple servers in thelarge. The huge performance gap between these two types of networks used toslow down distributed query processing to such an extent that a cluster ofmachines actually performed worse than a single many-core server. The increasedmain-memory capacity of the cluster remained the sole benefit of such ascale-out. The economic viability of high-speed interconnects such as InfiniBand hasnarrowed this performance gap considerably. However, InfiniBand's highernetwork bandwidth alone does not improve query performance as expected when thedistributed query engine is left unchanged. The scalability of distributedquery processing is impaired by TCP overheads, switch contention due touncoordinated communication, and load imbalances resulting from theinflexibility of the classic exchange operator model. This paper presents theblueprint for a distributed query engine that addresses these problems byconsidering both levels of networks holistically. It consists of two parts:First, hybrid parallelism that distinguishes local and distributed parallelismfor better scalability in both the number of cores as well as servers. Second,a novel communication multiplexer tailored for analytical database workloadsusing remote direct memory access (RDMA) and low-latency network scheduling forhigh-speed communication with almost no CPU overhead. An extensive evaluationwithin the HyPer database system using the TPC-H benchmark shows that ourholistic approach indeed enables high-speed query processing over high-speednetworks.
展开▼
