Resource discovery in large resource-sharing environments.

摘要

Opportunistic sharing of Internet-connected resources is a low cost method for obtaining access to unprecedented-scale collections of resources. An essential service in any resource-sharing environment is resource discovery: given a description of the resources desired, a resource discovery mechanism returns locations of resources that match the description.; Two resource-sharing environments are particularly well defined by applications, user communities, and deployments: Grid and peer-to-peer systems. Grids are sharing environments that rely on persistent, standards-based service infrastructures that allow well-established, mainly professional communities to share computers, storage space, sensors, software applications, and data across organizational boundaries. Peer-to-peer systems are Internet applications that harness resources from millions of autonomous participants. Thus, Grids provide infrastructure to support a variety of applications on resources shared by relatively small communities; at the scale of the peer-to-peer communities, remarkable sharing patterns are exhibited, such as free riding and intermittent resource participation.; The focus of this dissertation is on solution design for resource discovery in Grids of the scale and lack of reliability of today's peer-to-peer networks. This hybrid target environment requires fully decentralized solutions that scale with the number of users and resources and tolerate intermittent resource participation.; To explore the solution space, we propose a taxonomy for resource discovery solutions. This taxonomy proves to be a useful tool for discussing and comparing existing solutions.; Using this taxonomy, we delimit and explore a portion of the solution space. We build a scalable Grid emulator to evaluate mechanism performance in this subspace. Large-scale experiments reveal that the performance of mechanisms in this subspace is strongly dependent on sharing characteristics.; For inspiration, we turned to studying user behavior in various communities. We uncovered a significant usage pattern in file-sharing communities: users naturally form interest-based groups. This pattern can be exploited for system design in a variety of problems: we designed a file-location mechanism, FLASK, that exploits and benefits from this naturally emerging pattern. Trace-driven evaluations show FLASK leads to lower response latency, good scalability, support for intermittent participation; and satisfies requirements typical of scientific usage of data.