PDF(Pubmed)
Abstract：
BACKGROUND: Objectives of this study were to analyze the impact of including librarians and information specialist as methodological peer-reviewers. We sought to determine if and how librarians\' comments differed from subject peer-reviewers\'; whether there were differences in the implementation of their recommendations; how this impacted editorial decision-making; and the perceived utility of librarian peer-review by librarians and authors.
METHODS: We used a mixed method approach, conducting a qualitative analysis of reviewer reports, author replies and editors\' decisions of submissions to the International Journal of Health Governance. Our content analysis categorized 16 thematic areas, so that methodological and subject peer-reviewers\' comments, decisions and rejection rates could be compared. Categories were based on the standard areas covered in peer-review (e.g., title, originality, etc.) as well as additional in-depth categories relating to the methodology (e.g., search strategy, reporting guidelines, etc.). We developed and used criteria to judge reviewers\' perspectives and code their comments. We conducted two online multiple-choice surveys which were qualitatively analyzed: one of methodological peer-reviewers\' perceptions of peer-reviewing, the other of published authors\' views on the suggested revisions.
RESULTS: Methodological peer-reviewers assessed 13 literature reviews submitted between September 2020 and March 2023. 55 reviewer reports were collected: 25 from methodological peer-reviewers, 30 from subject peer-reviewers (mean: 4.2 reviews per manuscript). Methodological peer-reviewers made more comments on methodologies, with authors more likely to implement their changes (52 of 65 changes, vs. 51 of 82 by subject peer-reviewers); they were also more likely to reject submissions (seven vs. four times, respectively). Where there were differences in recommendations to editors, journal editors were more likely to follow methodological peer-reviewers (nine vs. three times, respectively). The survey of published authors (87.5% response rate) revealed four of seven found comments on methodologies helpful. Librarians\' survey responses (66.5% response rate) revealed those who conducted peer-reviews felt they improved quality of publications.
CONCLUSIONS: Librarians can enhance evidence synthesis publications by ensuring methodologies have been conducted and reported appropriately. Their recommendations helped authors revise submissions and facilitated editorial decision-making. Further research could determine if sharing reviews with subject peer-reviewers and journal editors could benefit them in better understanding of evidence synthesis methodologies.
METHODS: We used a mixed method approach, conducting a qualitative analysis of reviewer reports, author replies and editors\' decisions of submissions to the International Journal of Health Governance. Our content analysis categorized 16 thematic areas, so that methodological and subject peer-reviewers\' comments, decisions and rejection rates could be compared. Categories were based on the standard areas covered in peer-review (e.g., title, originality, etc.) as well as additional in-depth categories relating to the methodology (e.g., search strategy, reporting guidelines, etc.). We developed and used criteria to judge reviewers\' perspectives and code their comments. We conducted two online multiple-choice surveys which were qualitatively analyzed: one of methodological peer-reviewers\' perceptions of peer-reviewing, the other of published authors\' views on the suggested revisions.
RESULTS: Methodological peer-reviewers assessed 13 literature reviews submitted between September 2020 and March 2023. 55 reviewer reports were collected: 25 from methodological peer-reviewers, 30 from subject peer-reviewers (mean: 4.2 reviews per manuscript). Methodological peer-reviewers made more comments on methodologies, with authors more likely to implement their changes (52 of 65 changes, vs. 51 of 82 by subject peer-reviewers); they were also more likely to reject submissions (seven vs. four times, respectively). Where there were differences in recommendations to editors, journal editors were more likely to follow methodological peer-reviewers (nine vs. three times, respectively). The survey of published authors (87.5% response rate) revealed four of seven found comments on methodologies helpful. Librarians\' survey responses (66.5% response rate) revealed those who conducted peer-reviews felt they improved quality of publications.
CONCLUSIONS: Librarians can enhance evidence synthesis publications by ensuring methodologies have been conducted and reported appropriately. Their recommendations helped authors revise submissions and facilitated editorial decision-making. Further research could determine if sharing reviews with subject peer-reviewers and journal editors could benefit them in better understanding of evidence synthesis methodologies.
摘要：
背景：本研究的目的是分析纳入图书馆员和信息专家作为方法同行评审员的影响。我们试图确定图书馆员的评论是否以及如何与主题同行评审者不同；他们的建议的实施是否存在差异；这如何影响编辑决策；以及图书馆员和作者对图书馆员同行评审的感知效用。
方法：我们使用了混合方法方法，对审稿人的报告进行定性分析，作者回复和编辑对“国际卫生治理杂志”投稿的决定。我们的内容分析对16个主题领域进行了分类，所以方法论和主题同行评议者的评论，可以比较决策和拒绝率。类别基于同行评审涵盖的标准领域(例如，title,独创性,等。)以及与方法相关的其他深入类别(例如，搜索策略,报告准则，等。).我们开发并使用标准来判断审稿人的观点并对他们的评论进行编码。我们进行了两项在线多项选择调查，并进行了定性分析：一项是方法学同行评审者对同行评审的看法，其他已发表的作者对建议的修订的看法。
结果：方法同行评审员评估了2020年9月至2023年3月之间提交的13篇文献综述。收集了55份审稿人报告：25份来自方法学同行审稿人，30位来自主题同行评审员（平均：每篇手稿4.2条评论）。方法学同行评审员对方法学发表了更多评论，作者更有可能实施他们的更改(65个更改中的52个，vs.受试者同行评审者中的51人)；他们也更有可能拒绝提交(7与四次,分别)。如果对编辑的建议存在差异，期刊编辑更有可能遵循方法论的同行评审(9与三次,分别)。对已发表的作者的调查（回应率为87.5％）显示，对方法有帮助的七个评论中有四个。图书馆员的调查回应（66.5％的回应率）显示，进行同行评审的人认为他们提高了出版物的质量。
结论：图书馆员可以通过确保方法得到适当的实施和报告来加强证据综合出版物。他们的建议帮助作者修改提交的作品并促进编辑决策。进一步的研究可以确定与主题同行评审和期刊编辑共享评论是否可以使他们更好地理解证据综合方法。
方法：我们使用了混合方法方法，对审稿人的报告进行定性分析，作者回复和编辑对“国际卫生治理杂志”投稿的决定。我们的内容分析对16个主题领域进行了分类，所以方法论和主题同行评议者的评论，可以比较决策和拒绝率。类别基于同行评审涵盖的标准领域(例如，title,独创性,等。)以及与方法相关的其他深入类别(例如，搜索策略,报告准则，等。).我们开发并使用标准来判断审稿人的观点并对他们的评论进行编码。我们进行了两项在线多项选择调查，并进行了定性分析：一项是方法学同行评审者对同行评审的看法，其他已发表的作者对建议的修订的看法。
结果：方法同行评审员评估了2020年9月至2023年3月之间提交的13篇文献综述。收集了55份审稿人报告：25份来自方法学同行审稿人，30位来自主题同行评审员（平均：每篇手稿4.2条评论）。方法学同行评审员对方法学发表了更多评论，作者更有可能实施他们的更改(65个更改中的52个，vs.受试者同行评审者中的51人)；他们也更有可能拒绝提交(7与四次,分别)。如果对编辑的建议存在差异，期刊编辑更有可能遵循方法论的同行评审(9与三次,分别)。对已发表的作者的调查（回应率为87.5％）显示，对方法有帮助的七个评论中有四个。图书馆员的调查回应（66.5％的回应率）显示，进行同行评审的人认为他们提高了出版物的质量。
结论：图书馆员可以通过确保方法得到适当的实施和报告来加强证据综合出版物。他们的建议帮助作者修改提交的作品并促进编辑决策。进一步的研究可以确定与主题同行评审和期刊编辑共享评论是否可以使他们更好地理解证据综合方法。
