Large-scale graph processing using Apache Giraph

摘要

Analysis of graphical data (composed of nodes with edges between them) is a particularly challenging facet of the big data problem. Relational data (tables defining the features of each of a set of entities) enjoy computationally efficient algorithms for streaming processing, such as fast construction of decision trees. But most interesting problems in graph analysis are either NP-hard or require data structures that can quickly grow to exceed the capacity of conventional machines. Such problems benefit greatly from distributing the computation over multiple processors, but the iterative computation characteristic of graph algorithms is not well suited to the MapReduce algorithm [1], which has become ubiquitous in big data implementations.