Thinking-aloud about web navigation: the relationship between think-aloud instructions, task difficulty and performance

摘要

The concurrent think-aloud method is a highly valued technique that is used to understand peoples’ taskbasedcognitive processes, within both usability testing and the broader study of human-computer interaction.We investigated whether an explicit explanation-based think-aloud instruction leads to differences innavigation performance over the classic think-aloud method. The results showed an interaction betweenthink-aloud instructions and task difficulty. For the low difficulty tasks, there was no difference in tasksuccess or in the number of link traversals between the two think-aloud conditions. However, for the moredifficult tasks there were differences: participants in the classic condition completed fewer tasks successfullyand engaged in more link traversals than participants in the explicit think-aloud condition. A qualitativeanalysis of participants’ verbal data showed that both think-aloud conditions were dominated by proceduraldescriptions. The explicit think-aloud did yield proportionally more explanatory utterances, however theperformance differences in respect of task success and link traversal suggest that an explicit think-aloudmay threaten test validity and therefore evaluators must exercise caution in using this technique.