Improvements to online distributed monitoring systems

摘要

Online monitoring, providing the real-time status information of servers, is indispensable for the management of distributed systems, e.g. failure detection and resource scheduling. The main design challenges for distributed monitoring systems include scalability, fine granularity, reliability and low overheads. And the challenges are growing with the increase of the scales of the distributed systems. To address the above problems, this paper studies improvements to online distributed monitoring systems (ODMSs) from three aspects: online compression algorithm, online compression reliability, and data representation for information interchanges. We summarize and classify the existing online compression algorithms to identify some research gaps that may represent opportunities for future research. A simple solution is proposed to address the problem that the inaccuracy of compression algorithms may be caused by some failures of distributed systems. A bitmap-like data format is presented to reduce the per-node overheads and the overheads of the management node in ODMSs, and compared with other existing formats used in the monitoring system both in mathematical analysis and practical experiment. The results show that the bitmap-like data format achieves best performance overall.