Mining Hidden Profiles in the Collaborative Evaluation of Raw Ideas

摘要

We consider the task of evaluating raw ideas by a team of experts where typically a simple GO/NO-GO vote is taken. However, since both the ideas and the evaluation criterion can be ambiguous, the experts will in general form different mental models of them, which then become the basis for their individual evaluation judgements. This effect casts doubts on the meaning and reliability of the evaluation result. We propose a model for raw ideas and a facilitation algorithm for their evaluation in a group. The algorithm is designed to uncover hidden profiles in the raw idea and in the evaluation criteria and to treat these profiles separately. Our goal is to generate better ideas and a more precise interpretation of the evaluation criterion. An additional feature is increased transparency of the evaluation, which improves the group's acceptance of the result. Two small examples illustrate the behaviour of the algorithm.