A Novel Approach to Predictive Graphs Using Big Data

摘要

In enterprise models, relationships between data entities can be expressed by graphically connecting edges and vertices based on the application domain. Such graphs are optimal for understanding and processing simpler relationship-based models. When relationships get huge and complex in terms of large real-world problems such as friend or follower networks in social media or numerous client-corporate relationships for large multinational corporations, then converting and processing these graphs in near real-time is not trivial. This is because the number of relationships (edges) can grow super-exponentially in the number of vertices. We propose a novel method to create and update scalable relationship-data graphs for visualization and for prediction. Given a Big Table (BT) of related data expressed as structured (or semi-structured) data-tuples with associated business keys, the table can be quite easily transformed into a relationship data graph using current Big Data (BD) technologies. Predictive business models (e.g., from machine learning) can then be applied to the graph by our methods given here, utilizing a combination of data-parallel and graph-parallel computations. Applying a new data update to an existing data point in the graph such as a vertex (or edge), it is possible to predict the corresponding change in the model output variable on any vertex (or edge) in the graph. By combining predictive analytics models with data updates at either graph vertices or edges, we can utilize our method to propagate data updates in near real-time to the predictive graph.