Using Tools to Assist Identification of Non-requirements in Requirements Specifications - A Controlled Experiment

摘要

[Context and motivation] In many companies, textual fragments in specification documents are categorized into requirements and non-requirements. This categorization is important for determining liability, deriving test cases, and many more decisions. In practice, this categorization is usually performed manually, which makes it labor-intensive and error-prone. [Question/problem] We have developed a tool to assist users in this task by providing warnings based on classification using neural networks. However, we currently do not know whether using the tool actually helps increasing the classification quality compared to not using the tool. [Principal idea/results] Therefore, we performed a controlled experiment with two groups of students. One group used the tool for a given task, whereas the other did not. By comparing the performance of both groups, we can assess in which scenarios the application of our tool is beneficial. [Contribution] The results show that the application of an automated classification approach may provide benefits, given that the accuracy is high enough.