Modular architecture for extreme-scale distributed processing applications

摘要

Embodiments of the present invention relate to a new data center architecture that provides for efficient processing in distributed analytics applications. In one embodiment, a subnode of a distributed processing node is provided. The subnode includes at least one processor core operatively connected to a memory. A first interconnect operatively connects to the subnode. A second interconnect operatively connects the subnode to a storage. The storage includes a first storage unit and a second storage unit. The second storage unit has lower access time and latency than the first storage unit. A storage manager is provided that is operative to allocate data between the first and second storage units based on access patterns. The storage manager preferentially relocates non-sequentially accessed data to the second storage unit from the first storage unit.