Scalable Shared Memory

The goal of this research is to develop techniques for building highly scalable shared memory multiprocessors. Factors that have traditionally have limited the scalability of shared memory systems include:

We began this effort with a paper design of Willow, an architecture that attempted to address each of these issues directly. Willow is a shared memory multiprocessor whose design provides system capacity and performance that is capable of supporting hundreds of microprocessors. We developed a detailed design of a sixty-four processor prototype that tested most of our ideas about scalability. Our results studying the Willow design, as well as the research of others investigating these issues, have led us to a cluster-based approach.

Parallel computing using networks of workstations has increasingly become a viable alternative to large, tightly-coupled parallel machines. Recently, it has become feasible for a single workstation to contain two to four processors. These multiprocessor workstations, combined with a high-speed network, comprise a cluster-based multiprocessor. Our recent research has focused on the means by which these individual clusters are connected to the network. In particular, we are investigating shared network caches and multi-channel networks.

A network cache augments the caching already provided by private, per-processor caches, and is particularly appropriate when large, second-level processor caches are not present. The network cache stores local copies of remote data and provides three potential benefits: increased intra-cluster sharing, reduced network traffic, and prefetching. Intra-cluster sharing occurs when processors on the same cluster actively share data without a corresponding network data transfer. There are two ways in which this can occur: direct processor-to-processor transfers, and sharing of network cache lines. Only direct transfer is provided in systems without a network cache. By providing a network cache, we increase the amount of intra-cluster sharing, since the network cache holds copies of remote data that processors can access locally. Many requests for remote data that would have otherwise incurred long network latencies are satisfied in the network cache, thereby reducing the demand placed on the network. Finally, network cache lines that are longer than processor cache lines may result in improved performance by prefetching needed data. We are currently examining the structure, feasibility, and performance of cluster multiprocessors employing network caches.

We have observed that the adverse impact of cache inclusion-related evictions must be mitigated for the performance benefits of network caches to be fully realized. We have evaluated three network cache architectural alternatives designed to address this issue: increasing network cache associativity, adding a network victim cache, and adding a tag cache to relax inclusion requirements for clean network cache lines. We found that a four-way set associative network cache, or a four-entry victim cache, dramatically reduced execution time for all applications examined. Employing a tag cache for replaced clean lines improved performance for some applications, but this benefit is highly dependent upon processor cache associativity.

Publications Related to Scalable Shared Memory