A data-driven method for rating management information systems journals in the same scale of the Association of Business Schools Journal Guide

摘要

An appropriate measurement of journal quality is essential in accreditation, funding allocation, hiring, merit pay, tenure, and promotion decisions in academics. The current best practice to rate journal quality is to combine journal bibliometrics with expert assessment. For example, the Association of Business Schools (ABS) Journal Guide generated by this method is widely used by many business schools. However, different journal bibliometrics calculated in the citation network sometimes provide inconsistent ranking, and it is hard for domain experts to utilize the conflicting information. Therefore, given a journal, if the ABS Scientific Committee members are not familiar with it and different journal bibliometrics provide conflicting information, the given journal is hard to be rated and will not be included in the ABS journal list. In order to solve the above issue and maintain a comprehensive list of journals in the management information systems (MIS) field, this paper proposes a data-driven method to predict the ABS rating based on six popular bibliometrics for any given MIS journal. To the best of our knowledge, our method is the first work on this type to predict ABS ratings, which can serve as a more reliable rating reference and is much easier to be used to generate the rating for a comprehensive list of journals in the MIS field. In this paper, comprehensive experiments are conducted to evaluate the rating performance of our method from four different perspectives, including new journals, top journals, and interdisciplinary journals, and identifying overrated and underrated journals by ABS. Experiment results show our method can provide very reliable estimated ABS ratings for most MIS journals with few exceptions. Since our method is not perfect, expert knowledge is encouraged to be included to correct our estimated ABS ratings. However, such correction must be conducted under the following two constraints. First, domain experts must have sufficient evidences to do the correction. Second, correction can be adding or subtracting 1, but not beyond 1.