An architecture for highly concurrent, well-conditioned Internet services.

摘要

This dissertation presents an architecture for handling the massive concurrency and load conditioning demands of busy Internet services. Our thesis is that existing programming models and operating system structures do not adequately meet the needs of complex, dynamic Internet servers, which must support extreme concurrency (on the order of tens of thousands of client connections) and experience load spikes that are orders of magnitude greater than the average. We propose a new software framework, called the staged event-driven architecture (or SEDA), in which applications are constructed as a network of event-driven stages connected with explicit queues. In this model, each stage embodies a robust, reusable software component that performs a subset of request processing. By performing admission control on each event queue, the service can be well-conditioned to load, preventing resources from being overcommitted when demand exceeds service capacity. SEDA employs dynamic control to tune runtime parameters (such as the scheduling parameters of each stage) automatically, as well as to manage load, for example, by performing adaptive load shedding.; In this dissertation, we show that the SEDA design yields higher performance than traditional service designs, while exhibiting robustness to huge variations in load. We begin by evaluating existing approaches to service design, including thread-based and event-driven concurrency mechanisms, and demonstrate that these approaches fail to meet the concurrency and load conditioning demands of busy network services. Next, we present the SEDA design, and motivate its use through a set of design patterns that describe how to map an Internet service onto the SEDA structure. We also derive a performance model for SEDA-based Internet services based on queueing networks; this model is used to highlight the performance and load aspects of the architecture, as well as to drive design decisions affecting the decomposition of services into stages.; We present an implementation of an Internet services platform, called Sandstorm, based on the SEDA architecture. Sandstorm is constructed entirely in Java and makes use of nonblocking I/O primitives for sustaining high concurrency. We evaluate the use of SEDA through several applications, including a high-performance HTTP server, a packet router for the Gnutella peer-to-peer file sharing network, and a Web-based e-mail service using dynamic scripting and database access. Finally, we describe several control-based mechanisms for automatic tuning and load conditioning of busy Internet services in the SEDA design, including thread pool sizing, event hatching, and queue-based admission control for overload management. Our results show that SEDA is an effective design approach for complex Internet services, yielding high performance and robust behavior under heavy load.