Botched Releases: Do We Need to Roll Back? Empirical Study on a Commercial Web App

摘要

Few minutes after a web-based software release, the release team might encounter log traces showing the new system crashing, hanging, or having poor performance. This is the start of the most nerve-wrecking moments of a product's release cycle, i.e., should one run the risk of not doing anything and users losing precious data, or of prematurely engaging the tedious (and costly) roll-back procedure towards the previous release? Thus far, only little attention has been paid by researchers to these so-called "botched releases", partly because of lack of release log data. This paper studies 345 releases of a large e-commerce web app over a period of 1.5 years, in which we identified 17 recurrent root causes of botched releases, classified into four major categories. We then build explanatory models to understand which root causes are the most important, and to explore the factors leading to botched releases.