Opening Pandora's box: cause and impact of errors on plant pigment studies

摘要

Too many errors in scientific publications, too many in plant sciences, too Today, there is an astonishing volume of scientific information available for researchers, which can be easily accessed through powerful search tools. Yet, the question now is whether all this vast amount of information is reliable. In this sense, a “bad science” controversy arose recently when many Open Access (OA) journals (more than a half) published a false, error-ridden paper, which had been submitted in order to test the publishing ethics of these journals (Bohannon, 2013 ). This fake article was published mainly by fraudulent journals, but it was also accepted by a number of OA journals of renowned publishers with peer-review systems. The failure to reject an article full of errors revealed that the system's gearbox is leaking somewhere. The carelessness of peer-reviews in a number of OA journals has opened a Pandora's Box, and what is more disconcerting, nobody can guarantee that it could not also affect regular journals (non OA). Traditionally, it has been assumed that scientific journals should detect and correct all these failings through the peer review before publication. Regrettably, as we show in this communication, the system is far from being perfect (Pulverer, 2010 ; Székely et al., 2014 ). Whilst the detection of laboratory errors is an issue of great attention in medicine (Bonini et al., 2002 ; Carraro and Plebani, 2007 ; Hammerling, 2012 ), in experimental science, it does not seem to be a crucial task. However, we were aware of this concern when we performed a literature compilation with the aim of providing a comprehensive evaluation of the responses of photosynthetic pigment composition to environmental conditions (Esteban et al., 2015 ). In this survey, we compiled data from 525 papers from the last 21 years (1991–2011). After carrying out a critical analysis of the data, a considerable number of papers, 96 out of 525, were found to have data out of range, errors and inconsistencies in at least one of the parameters reported. In order to detect these errors, we used as an initial screening tool three standard deviations from the mean. Data outside this interval were subsequently examined (Osborne and Overbay, 2004 ), in order to identify whether errors had arisen from the inherent variability of the data or from mistakes in the data (for a detailed description see Esteban et al., 2015 ). We decided to carry out an in-depth analysis of these controversial data, establishing a new background for our study. In this sense, data was re-evaluated and classified on the basis of the type of error: I, II, and III. Error I ( n = 46) includes those articles containing values out of range (outside ±3 standard deviation from the mean), likely due to pre-analytical and analytical flaws. Error II ( n = 37) refers to the presence of wrong values (most of them higher than 1000-fold reference values), most likely caused by post-analytical errors. Error III ( n = 13) includes those articles with mistaken units, probably included during the final phase of publishing and editing. Discarding the non-ethical manipulation of data, errors may occur as a result of experimental (analytical or methodological), mathematical or editing errors. For the case studied here (pigment determinations), we have identified several potential sources of error, which may occur at any stage of the research process: (i) Methodological errors during pre-analytical and analytical phase : inappropriate specimen/sample collection and preservation, labeling errors, wrong biomass/leaf area measurement, incomplete extraction, malfunction of instruments, incorrect compound identification, pipetting errors, etc. (ii) Data analysis errors in the post-analytical phase : handling of mathematics in spreadsheets, mistakes in the preparation of graphics or tables, improper data entry, and failure in reporting; (iii) Publishing and editing errors: confusion in units, typing errors such as Latin instead of Greek letters or errors in graph scales (see Table S1 for complete list of errors, tips and solutions). The data published in regular papers do not allow an assessment to be made of which of these aforementioned error sources is the cause of the mistakes. However, in other fields, such as analytical medicine, in which traceability is easier thanks to the application of quality assurance protocols, most errors occur during the pre- and post-analytical phases (Hammerling, 2012 ). How, where and when these errors appear As an example of the inaccuracy of pigment measurements, we have performed an analysis of data published on pigment composition in the model plant Arabidopsis thaliana ecotype Columbia including all the available literature published in the major journals in this field during the period 1991–2011 (see Esteban et al., 2015 for details). It is assumed that plants cultivated under similar conditions in different laboratories should not differ greatly in th