Increasing Reproducibility in IR: Findings from the Dagstuhl Seminar on 'Reproducibility of Data-Oriented Experiments in e-Science'

摘要

The Dagstuhl Seminar on "Reproducibility of Data-Oriented Experiments in e-Science", held on 24-29 January 2016, focused on the core issues and approaches to reproducibility of experiments from a multidisciplinary point of view, sharing the experience coming from several fields of computer science. In this paper, we discuss, summarize, and adapt the main findings of the seminar to the context of IR evaluation - both system-oriented and user-oriented - in order to raise awareness in our community and stimulate the fields towards and increased reproducibility of our experiments.