System and method for grid-based distribution of Java project compilation

摘要

A system for automatically segmenting the compilation of a large Java project under a controller node to a grid of slave nodes so as to reduce build time. A build tree scanner at the controller node sizes the number of files in the project and the length of each source file. Each slave node has a complete and synchronized set of all compilable units in the project. The controller node automatically segments the source files into a number of filesets commensurate with the number and resource capacities of slave nodes available for the build, and allocates the filesets to the slave nodes. Each slave node scans each source file in its allocated fileset or filesets, builds a dependency tree comprising dependency graphs for each source file, and sends that tree to the controller node, from which the controller node then constructs a master dependency tree. The controller node uses the master dependency tree to automatically generate build filesets, which are then equitably allocated to the slave nodes, which track the time required to build each file in each allocated build fileset. Then the targets are reaped and placed on the controller node, which performs post-compilation tasks.