Automated Quality Assessment of Unstructured Resolution Text in IT Service Systems

摘要

In customer-care service centers, upon remediation of customer issues, the human agents are expected to record their resolution summary in a clear, concise and understandable manner. These resolution summaries create a rich untapped source of unstructured information. In this work, we have addressed the problem of how to enable human agents to write better quality resolution text. This helps curate data artifacts which can reduce problem diagnosis time and create repeatable resolution recipes by a cognitive system. The problem is addressed through a two pronged approach: (i) On the fly automated scoring of the agent's resolution summary and (ii) identifying concrete areas of improvement in the summary and offering appropriate recommendations. The model for automatic scoring is derived from a feature set that encodes all significant and relevant aspects of the domain and text. The model is trained using annotated data and achieves an accuracy of 88.2 % which is a significant improvement over naive method of text based classification (68.5%).