Architecture design for distributed content-based publish-subscribe systems.

摘要

Content-based publish-subscribe (pub-sub) is a powerful information-driven communication model. In such a model, information producers publish information as events; information consumers subscribe to the events of their interest by specifying conditions on event content, and are notified when events matching their subscriptions are published.; Architecture design for distributed content-based pub-sub systems is a challenging problem, because the dynamic and diversified pub-sub communication paradigm cannot be readily supported by state-of-the-art Internet protocol primitives.; In this thesis, we propose a pub-sub network architecture design called MEDYM, for Match-Early with DYnamic Multicast. MEDYM follows the End-to-End distributed system design principle, by decoupling the content-based pub-sub service into a computationally intensive, application-specific event matching at network edge, and a simple, generic multicast routing in the network. We design a multicast scheme called dynamic multicast, which routes events to interested subscribers along dynamically computed routes with high network efficiency. Because the diversity of such dynamic routing is not limited by server states storage, MEDYM is able to accurately deliver events to and only to the servers that subscribe to them. This is a highly desirable property that has not been achieved in existing solutions. Without maintenance for static overlay networks, MEDYM is also easy to deploy and manage.; The basic form of MEDYM architecture is scalable to pub-sub networks with thousands of servers. For further scalability, we design a novel hierarchical approach called H-MEDYM, for Hierarchical-MEDYM. H-MEDYM partitions a pub-sub network along two dimensions: the server network and the content space. As a result, it effectively reduces both the matching and routing states maintained at pub-sub servers without introducing severe event delivery inefficiency or skewed load distribution.; We evaluate MEDYM and H-MEDYM using detailed simulations and real-world experiments, and compare them with major existing design approaches. Results show that MEDYM and H-MEDYM achieve high event delivery efficiency and system scalability. Their advantages are most prominent when user subscriptions are highly selective and diversified, which is exactly the scenario when content-based pub-sub service is most valuable.