Asynchronous Group-Based ADMM Algorithm under Efficient Communication Structure

摘要

Alternating direction method of multipliers (ADMM) has been recognized as an efficient approach for solving many large-scale machine learning problems. However, the ADMM under master-slave mode suffers from several limitations, e.g., can't make full use of multi-core cluster environment and single master load is too heavy, resulting in huge time overhead. In this paper, we propose a hierarchical communication structure. Since intra-node communications mostly use shared memory, we divide the processes of the same node into one group, the processes within the group synchronize communication, and each group communicate asynchronously with the master. Combining this structure with the ADMM algorithm, a hierarchical asynchronous group-based ADMM algorithm (HAG-ADMM) is proposed. Theoretical analysis and experiments show that the hierarchical communication structure can improve the communication efficiency of the algorithm and has no effect on the convergence.