背景:作为研究人员展示其学术发现的重要平台,医学期刊的评价取向与发表研究成果的价值取向有着密切的关系。然而,医学期刊的学术影响力和破坏性创新水平之间的差异尚未得到任何研究的检验.
目的:本研究旨在比较学术影响之间的关系和差异,颠覆性创新水平,以及医学期刊和发表的研究论文的同行评审结果。我们还分析了影响评价的异同,颠覆性创新,和同行评审不同类型的医学研究论文及其根本原因。
方法:选择2018年普通和内科学引文索引扩展(SCIE)期刊为研究对象,基于PubMed开放PMID-to-PMID引文索引(POCI)和H1Connect数据库,探讨医学期刊学术影响力和颠覆性创新水平的差异,分别,我们将它们与同行评审的结果进行了比较。
结果:首先,期刊中断指数(JDI)与期刊5年累积引文(JCC5)的相关系数,期刊影响因子(JIF),和期刊引文指标(JCI)分别为0.677、0.585和0.621。绝对中断指数(Dz)与5年累积引文(CC5)的相关系数为0.635。然而,期刊颠覆性创新和学术影响力排名的平均差异达到20位(约17.5%)。研究论文颠覆性创新和影响力排名的平均差异达到约2700位(约17.7%)。差异反映了两种评价体系的本质区别。第二,根据JDI选择的前7种期刊,JCC5,JIF,和JCI是一样的,所有这些都是H期刊。虽然8(8/15,53%),96(96/150,64%),和880名(880/1500,58.67%)的前0.1%,前1%,分别根据Dz和CC5选择的前10%论文,是一样的。第三,带有“改变临床实践”标签的研究论文仅显示了中等程度的创新(4.96)和影响(241.67)水平,但具有较高的同行评审认可度(6.00)和关注度(2.83)。
结论:研究结果表明,基于创新指标的研究评价脱离了传统的影响评价体系。3个评价体系(影响评价、颠覆性创新评估,和同行评审)仅对权威期刊和顶级论文有很高的一致性。单一影响指标和创新指标都不能直接反映医学研究对临床实践的影响。如何建立一个综合,全面,科学,完善现有的医学期刊评价体系仍需进一步研究。
BACKGROUND: As an important platform for researchers to present their academic findings, medical journals have a close relationship between their evaluation orientation and the value orientation of their published research results. However, the differences between the academic impact and level of disruptive innovation of medical journals have not been examined by any study yet.
OBJECTIVE: This study aims to compare the relationships and differences between the academic impact, disruptive innovation levels, and peer review results of medical journals and published research papers. We also analyzed the similarities and differences in the impact evaluations, disruptive innovations, and peer reviews for different types of medical research papers and the underlying reasons.
METHODS: The general and internal medicine Science Citation Index Expanded (SCIE) journals in 2018 were chosen as the study object to explore the differences in the academic impact and level of disruptive innovation of medical journals based on the OpenCitations Index of PubMed open PMID-to-PMID citations (POCI) and H1Connect databases, respectively, and we compared them with the results of peer review.
RESULTS: First, the correlation coefficients of the Journal Disruption Index (JDI) with the Journal Cumulative Citation for 5 years (JCC5), Journal Impact Factor (JIF), and Journal Citation Indicator (JCI) were 0.677, 0.585, and 0.621, respectively. The correlation coefficient of the absolute disruption index (Dz) with the Cumulative Citation for 5 years (CC5) was 0.635. However, the average difference in the disruptive innovation and academic influence rankings of journals reached 20 places (about 17.5%). The average difference in the disruptive innovation and influence rankings of research papers reached about 2700 places (about 17.7%). The differences reflect the essential difference between the two evaluation systems. Second, the top 7 journals selected based on JDI, JCC5, JIF, and JCI were the same, and all of them were H-journals. Although 8 (8/15, 53%), 96 (96/150, 64%), and 880 (880/1500, 58.67%) of the top 0.1%, top 1%, and top 10% papers selected based on Dz and CC5, respectively, were the same. Third, research papers with the \"changes clinical practice\" tag showed only moderate innovation (4.96) and impact (241.67) levels but had high levels of peer-reviewed recognition (6.00) and attention (2.83).
CONCLUSIONS: The results of the study show that research evaluation based on innovative indicators is detached from the traditional impact evaluation system. The 3 evaluation systems (impact evaluation, disruptive innovation evaluation, and peer review) only have high consistency for authoritative journals and top papers. Neither a single impact indicator nor an innovative indicator can directly reflect the impact of medical research for clinical practice. How to establish an integrated, comprehensive, scientific, and reasonable journal evaluation system to improve the existing evaluation system of medical journals still needs further research.