It's Not Just about Accuracy: Metrics That Matter When Modeling Expert Sketching Ability

摘要

It's Not Just about Accuracy: Metrics That Matter When Modeling Expert Sketching Ability TRACY HAMMOND$1SHALINI PRIYA ASHOK KUMAR$1MATTHEW RUNYON$1JOSH CHERIAN$1BLAKE WILLIFORD$1SWARNA KESHAVABHOTLA$1STEPHANIE VALENTINE$1WAYNE LI$2JULIE LINSEY$3; $1Sketch Recognition Lab, Texas A&M University$2College of Design, Georgia Institute of Technology$3 iDREEM Lab, Georgia Institute of Technology; ;;;;;;;; Design sketching is an important skill for designers, engineers, and creative professionals, as it allows them to express their ideas and concepts in a visual medium. Being a critical and versatile skill for many different disciplines, courses on design sketching are often taught in universities. Courses today predominately rely on pen and paper; however, this traditional pedagogy is limited by the availability of human instructors, who ' can provide personalized feedback. Using a stylus-based intelligent tutoring system called SketchTivity, we aim to eventually mimic the feedback given by an instructor and assess student-drawn sketches to give students insight into areas for improvement. To provide effective feedback to users, it is important to identify what aspects of their sketches they should work on to improve their sketching ability. After consulting with several domain experts in sketching, we came up with several classes of features that could potentially differentiate expert and novice sketches. Because improvement on one metric, such as speed, may result in a decrease in another metric, such as accuracy, the creation of a single score may not mean much to the user. We attempted to create a single internal score that represents overall drawing skill so that the system can track improvement over time and found that this score correlates highly with expert rankings. We gathered over 2,000 sketches from 20 novices and four experts for analysis. We identified key metrics for quality assessment that were shown to significantly correlate with the quality of expert sketches and provide insight into providing intelligent user feedback in the future.