Using Social Network and Semantic Overlay Network approaches to share knowledge in distributed data mining scenarios

摘要

Developing distributed data mining applications in decentralized environments requires services both to find the needed resources (data, algorithms, computing nodes) and to share the inferred knowledge among users after the data mining process has been performed. The need for an efficient implementation of such functionalities is of great importance in large-scale scenarios, like the Grid, where centralized approaches are not scalable. To address this issue, we investigated the use of decentralized P2P approaches, like Social Networks (SNs) and Semantic Overlay Networks (SONs), to define a set of services and mechanisms to share both resource information and inferred knowledge in the Knowledge Grid for distributed data mining applications. The Knowledge Grid provides services to publish and retrieve metadata about resources and mining models, as well as a basic mechanism to search for such metadata in a group of nodes to support distributed data mining application design. While this search mechanism has proven effective in small-scale scenarios, the use of SN and SON approaches can help to make those services more efficient in large-scale contexts. In particular, this paper presents a two-layered model, in which a SON is built over a SN, to efficiently share knowledge and search resources. Moreover, the paper describes how the resource management services of the Knowledge Grid are extended according to this model.