Log4Perf: suggesting and updating logging locations for web-based systems' performance monitoring

摘要

Performance assurance activities are an essential step in the release cycle of software systems. Logs have become one of the most important sources of information that is used to monitor, understand and improve software performance. However, developers often face the challenge of making logging decisions, i.e., neither logging too little and logging too much is desirable. Although prior research has proposed techniques to assist in logging decisions, those automated logging guidance techniques are rather general, without considering a particular goal, such as monitoring software performance. In this paper, we present Log4Perf, an automated approach that provides suggestions of where to insert logging statement with the goal of monitoring web-based systems' CPU usage. In the first part of our approach, we leverage the performance model's prediction errors to suggest the need for updating logging locations when software evolves. In the second part of our approach, we build and manipulate a statistical performance model to identify the locations in the source code that statistically significantly influence CPU usage. To evaluate Log4Perf, we conduct case studies on two open source systems, i.e., CloudStore and OpenMRS, and one large-scale commercial system. Our evaluation results show that our approach can suggest the need for updating logging locations and identify the logging locations that can be kept unchanged. We manually examine the logging locations that are newly suggested or deprecated. We find that our approach can build well-fit statistical performance models, indicating that such models can be leveraged to investigate the influence of locations in the source code on performance. The suggested logging locations are often in small and simple methods that do not have logging statements, and are not performance hotspots. Our approach can be used to complement traditional approaches that are based on software metrics or performance hotspots. In addition, we identify seven root-causes of these suggested or deprecated logging locations. Log4Perf is integrated into the release engineering process of the commercial software to provide logging suggestions on a regular basis.