The Relevance of Testing in Engineering Product Development - Investigations on a Robot Programming Task

摘要

Prototyping iterations have been linked to design performance in complex problems solving. Does prototyping also enhance design performance in engineering product development where the information needed to solve the problem is already available? In this paper we experimentally compare if early testing of prototypes leads to different development outcomes compared to an approach that entirely relies on planning in the earlier phases of the project. In the experiment we ask participants to program a mobile robot to autonomously perform a task with a quantifiable performance outcome. The task is open-ended and allows for several solutions to the problem. To enhance the statistical validity of the results we develop and use a digital simulation. Both in the manually recorded and in the simulated results we observe a tendency that the participants that are allowed and encouraged to test their prototypes frequently perform better than participants that are not allowed to test prototypes. The performance of the initially non-testing participants equalizes after they had time to test and improve their designs and may even become better than the performance of the testing participants. However, both these performance differences are not statistically significant and a study with a larger number of participants is required. From qualitative observations, we conclude that the testing behavior of the individual developer may have a stronger influence on the outcome than if or how often a developers test their designs and further studies are needed.