Platform for General-Purpose Distributed Data-Mining on Large Dynamic Graphs

摘要

We present an approach to data mining on arbitrary graph data that uses a cloud-based distributed computing model for dynamic provisioning of computing resources as the graph model grows or shrinks. Further, we introduce the concept of logging graph changes as a basis for calculating properties of dynamic graphs. We briefly describe queries that leverage the dynamic graph model, for instance, by using a snapshot of the original graph while an algorithm executes or adapting query results as the graph changes. To demonstrate the feasibility of our approach, we conducted an initial evaluation, which shows that our parallel computing model can dramatically improve load times. Raw data imported into our system is processed faster on larger compute clusters.