In 2008, NASA's Pavilion Lake Research Project (PLRP) began a study to develop a highly detailed map of the morphology, depth, and location of microbialites - underwater formations that provide insight into some of the earliest forms of life on Earth - in Pavilion and Kelly lakes, Canada. Data for this project was collected between 2008 and 2012 in three phases, the first two of which were limited to the PLRP science team. The third phase, called the Morphology Analysis Project for Participatory Exploration and Research (MAPPER), was an online Citizen Science activity open to the general public, and was made available in the run-up to the PLRP 2011 field season. MAPPER was built from the ground up with Citizen Science in mind, and included reference material, tutorials, and quizzes to provide a consistent training experience for new users. Using MAPPER, more than 4,600 people from around the world classified over 1.1 million images and submitted over 1.4 million observations, representing 2,200+ person-hours of effort. Quality of the expert and volunteer data are the prime focus of our study. Classification fields for a given image by a user were divided into three primary "labs": ID, microbialites, algae, each with subfields pertaining to details of that field. Images were classified by multiple users. An "expert group consensus" was formed for each expert classification by determining the mode among all the experts that rated that particular field and image as a basis to compare the expert ratings to the volunteers. For example, for the primary fields identifying the presence of microbialites and algae in a given image, experts unanimously agreed on 46.5%.
展开▼
