Statistical significance publication bias, its determination and statistical adjustments in meta-analysis

摘要

Literature review, or research synthesis is an essential part of programs of scientific research. There is no doubt that conclusions based upon combined results of all of studies available are much more reliable and persuasive than those from any single one of them, if the combining is conducted properly. The use of statistical methods to quantitatively synthesize research findings from different studies was not common in fields such as education and psychology for many years, due to lack of a common physical measurement scale to represent the results of studies. Since Glass (1976) introduced standardized mean differences in meta-analysis, broader application of statistical techniques for combining the evidence from individual studies to these fields has become possible.;Because of its attractive nature, meta-analysis has been employed in many fields as a research tool. Perhaps as a consequence, various issues regarding its theoretical, practical and methodological problems have been raised and extensively discussed (Cooper, 1982, 1984; Glass, McGaw and Smith, 1981; Hedges and Olkin, 1985; Hedges, 1986; Light and Pillemer, 1984). In fact meta-analysis itself has become the objective of a great deal of research. Many of the initial problems have now been satisfactorily solved. But a few of them still remain challenging. The problem of publication bias based on statistical significance is one of them.;There is no need to say that published literature plays an important role in literature review. Because researchers always produce many more articles than journal editors can publish (Bakan, 1966; Brackbill & Korten 1970; Newman, 1966), publication process is highly selective. Perhaps because of that, few people seriously challenged the fairness of published literature, as a whole, until Sterling (1959).;Sterling found that published literature was biased against studies with statistically nonsignificant results in psychological journals. He believed that because nonsignificant results were seldom published and the published significant studies were rarely verified, there was a substantial portion of erroneous conclusions in published literature.;Sterling's finding was appalling. Many studies followed and supported his study (Bozarth and Roberts, 1972; Galtung, 1967; Greenwald, 1975; Katzer and Sodt, 1973). That bias against nonsignificant research results in publication selection has now been generally accepted as a serious problem. In meta-analysis, where the main goal is to assess effect sizes quantitatively, the problem becomes more obvious and more significant. Hedges (1984) pointed out that such publication selection produced biased estimators of the population effect size; to be more specific, the estimators overestimated the population parameter.;This dissertation is devoted to an investigation of statistical significance publication bias and to statistical adjustments to correct the bias.