Active storage networks: Using embedded computation in the network switch for cluster data processing

摘要

High performance data processing clusters use parallelism to accelerate computation. Often, these computation processes require the transmission of data across networks. In this paper, we propose the use of computation within the network switch to perform computation on data on the fly and further accelerate computation. We call networks built with these compute switches Active Storage Networks (ASN) and they provide an opportunity to optimize storage system and computational performance by offloading some computation to the network switch. We present an approach to perform transformation and reduction data operations in a network switch comprised of FPGAs. In this paper, we demonstrate an ASN using representative data processing applications, namely data search, data sort, k-min/max, and k-means clustering.