Measuring Student Ability to Work on Multi-Disciplinary Teams: A Method for Determining Validity and Reliability of a Rubric

摘要

Engineering educators struggle to provide effective educational experiences for professional skills such as communication, cultural awareness, and ability to work on multi-disciplinary teams. As difficult as these skills are to teach, they are even more difficult to evaluate. Over the past year, we have introduced a new course at the junior-level, "Introduction to Engineering Design." The course focuses on the skills necessary to complete a project in a multi-disciplinary team, and it will eventually be required for all engineering students as a precursor to their department-specific capstone design courses. In a previous paper, we described our approach of using the engineering design process to determine the best solution to the problem of providing students with a multidisciplinary educational experience in engineering at Montana State University. In order to determine if our new course improves student performance in this area, we developed a rubric for evaluating an individual's performance on a multi-disciplinary team. In previous work, we presented our rubric development process and preliminary results. In this paper, we discuss an initial assessment of the validity and reliability of the rubric. Broadly speaking, the validity of a measurement instrument refers to how well it measures what it is intended to measure, whereas reliability is the consistency of the results of using the instrument.