Assessing the Effectiveness of Shah's Innovation Metrics for Measuring Innovative Design within a Virtual Design Space

摘要

Epistemic games, such as the virtual engineering internship Nephrotex, allow students to explore creative ways to approaching engineering problems while providing a novel alternative to the direct transmission method of instruction. Within Nephrotex, students choose a polymer, manufacturing process, surfactant, and percentage of carbon nanotube to create a functioning kidney dialysis membrane prototype. The performance of the membrane is measured using cost, flux, blood cell reactivity, marketability, and reliability thresholds given by stakeholders within the fictitious company. Although Nephrotex has been shown to be a valuable educational tool for modeling the product design process, only limited work has been done to investigate whether it is capable of providing an environment that allows students to generate innovative designs. The innovation assessment framework of Shah and colleagues employs four metrics of innovation - novelty, variety, quality, and quantity; novelty is further divided into a priori and a posteriori metrics. This work found that a priori and a posteriori novelty, variety, and quality were applicable metrics of innovation in the epistemic game environment of Nephrotex. Literature ranges for a priori and a posteriori novelty scores aligned with those found in this study. Comparing prior work on Nephrotex that identified innovative student designs based on a proposed literature definition, it was found that the Shah metrics between the innovative and non-innovative groups showed little variation and no statistically significant differences. A t test and a Mann-Whitney U test showed no significant difference between innovative and non-innovative groups with regard to variety or novelty scores; however, these tests did show a significant difference between groups with regards to the quality score. The same results were found when calculating Cohen's Effect Size - a priori novelty, a posteriori novelty, and variety had a small effect when comparisons were made between the innovative and non-innovative groups while quality had a large effect. The significant difference and large effect in regards to quality however, may be the result of the previous literature definition which employed quality as a measure to define innovation. Results from this study demonstrate that novelty is perhaps the most aligned innovation metric for an epistemic game environment and that both variety and quality can be helpful in understanding the designs generated within these contexts although they may need adjustment based on the application to a constrained design space.