一种大规模分布式应用性能分析系统

摘要

Monitoring and analyzing large-scale distributed applications in grid or cloud environment is very difficult, due to the complexity of platform and network environment. This paper describes a system to monitoring and analysis such applications. This system is based on the concept of data stream management. It uses message queues to collect, cache and distribute trace messages, uses distributed computing framework to analyze the trace messages in real time. The prototype is deployed in a real Petabyte-scale distributed data management system. The usefulness of the collected trace messages is demonstrated by examples. Application result shows that this system is easy to deploy and has little affection on the applications, can well suit the requirement of big data analysis and real-time compute, provides a platform to analyze the performance of large-scale distributed system, predict user behavior.%在网格和云计算环境下,由于平台和网络环境的复杂性,使得对大规模分布式应用的有效监控和性能分析变得非常困难.为此,提出一种基于数据流管理的大规模分布式应用性能分析系统,利用消息队列收集、缓冲和分发追踪消息,使用分布式实时处理框架分析和追踪消息.将该系统部署到一个Petabyte级别的分布式数据管理系统中,通过事例演示追踪消息的重要性.应用结果表明,该系统能够满足大规模分布式应用环境下大数据量处理能力和实时性的要求,为监控并分析系统性能、预测用户行为等提供了较好的平台支持.