A Split Cache Hierarchy for Enabling Data-Oriented Optimizations

摘要

Today's caches tightly couple data with metadata (Address Tags) at the cache line granularity. The co-location of data and its identifying metadata means that they require multiple approaches to locate data (associative way searches and level-by-level searches), evict data (coherent writebacks buffers and associative level-by-level searches) and keep data coherent (directory indirections and associative level-by-level searches). This results in complex implementations with many corner cases, increased latency and energy, and limited flexibility for data optimizations. We propose splitting the metadata and data into two separate structures: a metadata hierarchy and a data hierarchy. Themetadata hierarchy tracks the location of the data in the data hierarchy. This allows us to easily apply many differentoptimizations to the data hierarchy, including smart data placement, dynamic coherence, and direct accesses. The new split cache hierarchy, Direct-to-Master (D2M), provides a unified mechanism for cache searching, eviction, and coherence, that eliminates level-by-level data movement and searches, associative cache address tags comparisons andabout 90% of the indirections through a central directory. Optimizations such as moving LLC slices to the near-side ofthe network and private/shared data classification can easily be built on top off D2M to further improve its efficiency. Thisapproach delivers a 54% improvement in cache hierarchy EDP vs. a mobile processor and 40% vs. a server processor, reducesnetwork traffic by an average of 70%, reduces the L1 miss latency by 30% and is especially effective for workloads with high cache pressure.