Many authors used reporting checklists as an assessment tool to analyze the reporting quality of diverse types of evidence. We aimed to analyze methodological approaches used by researchers assessing reporting quality of evidence in randomized controlled trials, systematic reviews, and observational studies.
We analyzed articles reporting quality assessment of evidence with Preferred Reporting Items of Systematic Reviews and Meta-Analyses (PRISMA), CONsolidated Standards of Reporting Trials (CONSORT), or the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) checklists published up to 18 July 2021. We analyzed methods used for assessing reporting quality.
Among 356 analyzed articles, 293 (88%) investigated a specific thematic field. The CONSORT checklist (N = 225; 67%) was most often used, in its original, modified, partial form, or its extension. Numerical scores were given for adherence to checklist items in 252 articles (75%), of which 36 articles (11%) used various reporting quality thresholds. In 158 (47%) articles, predictors of adherence to reporting checklist were analyzed. The most studied factor associated with adherence to reporting checklist was the year of article publication (N = 82; 52%).
The methodology used for assessing reporting quality of evidence varied considerably. The research community needs a consensus on a consistent methodology for assessing the quality of reporting.

暂无摘要。

BACKGROUND: The central line bundle to reduce central line-associated bloodstream infections (CLABSI) is widely regarded as one of the most evidence-based quality improvement (QI) interventions. Yet, two high-quality trials reached different conclusions about its effectiveness.
OBJECTIVE: To assess the overall evidence on the effectiveness of the central line bundle and also to illustrate issues related to appraising the effectiveness of QI interventions.
METHODS: We searched the English-language literature (MEDLINE to Sept 2014) for prospective evaluations of the central line bundle (hand hygiene, chlorhexidine skin antisepsis, maximum sterile barrier precautions, optimal catheter site selection, daily review of line necessity) on CLABSI. Mantel-Haenszel risk ratios were calculated using a random effects model. Risk of bias was assessed on five domains: comparability of subjects, definition of intervention, assessment of outcome, statistical analysis and co-interventions/heterogeneity. Strength of the evidence was assessed following the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) approach, a widely recommended framework for assessing the robustness of treatment effect and the likelihood of change as a result of future studies.
RESULTS: Across 59 studies, the central line bundle effectively reduced CLABSI by 56% (relative risk 0.44 (95% CI 0.39 to 0.50)). Studies that assessed bundle compliance at the individual patient level reported slightly higher reductions than other studies. Considerable heterogeneity was present in most subgroups. Most studies had unclear or high risk of bias, with only six (10%) studies exhibiting low risk of bias on at least four domains without any high risk. In this subset of higher-quality studies, the reduction was 52% (95% CI 32% to 66%) without heterogeneity. Applying the GRADE framework, the overall strength of the evidence was low, but moderate in quality for the six high-quality studies. This rating is typically interpreted as meaning that further research is likely to have an important impact on our confidence in the effect estimate and may change the estimate.
CONCLUSIONS: That the central line bundle could receive only a moderate evidence rating may suggest that the GRADE framework, developed mostly for traditional clinical therapies, requires modification for QI interventions. GRADE does not distinguish prospective trials (eg, controlled before-after studies and interrupted time series) from lower-level observational studies. On the other hand, that the two highest quality studies reached different conclusions makes it difficult to conclude that future research would not change the effect estimate, especially given evidence of secular trends and the variability of co-interventions to ensure bundle compliance, which created heterogeneity across studies.

HEALTH CARE POLICY BACKGROUND: Findings from scientific studies form the basis for evidence-based health policy decisions.
BACKGROUND: Quality assessments to evaluate the credibility of study results are an essential part of health technology assessment reports and systematic reviews. Quality assessment tools (QAT) for assessing the study quality examine to what extent study results are systematically distorted by confounding or bias (internal validity). The tools can be divided into checklists, scales and component ratings.
OBJECTIVE: What QAT are available to assess the quality of interventional studies or studies in the field of health economics, how do they differ from each other and what conclusions can be drawn from these results for quality assessments?
METHODS: A systematic search of relevant databases from 1988 onwards is done, supplemented by screening of the references, of the HTA reports of the German Agency for Health Technology Assessment (DAHTA) and an internet search. The selection of relevant literature, the data extraction and the quality assessment are carried out by two independent reviewers. The substantive elements of the QAT are extracted using a modified criteria list consisting of items and domains specific to randomized trials, observational studies, diagnostic studies, systematic reviews and health economic studies. Based on the number of covered items and domains, more and less comprehensive QAT are distinguished. In order to exchange experiences regarding problems in the practical application of tools, a workshop is hosted.
RESULTS: A total of eight systematic methodological reviews is identified as well as 147 QAT: 15 for systematic reviews, 80 for randomized trials, 30 for observational studies, 17 for diagnostic studies and 22 for health economic studies. The tools vary considerably with regard to the content, the performance and quality of operationalisation. Some tools do not only include the items of internal validity but also the items of quality of reporting and external validity. No tool covers all elements or domains. Design-specific generic tools are presented, which cover most of the content criteria.
CONCLUSIONS: The evaluation of QAT by using content criteria is difficult, because there is no scientific consensus on the necessary elements of internal validity, and not all of the generally accepted elements are based on empirical evidence. Comparing QAT with regard to contents neglects the operationalisation of the respective parameters, for which the quality and precision are important for transparency, replicability, the correct assessment and interrater reliability. QAT, which mix items on the quality of reporting and internal validity, should be avoided.
CONCLUSIONS: There are different, design-specific tools available which can be preferred for quality assessment, because of its wider coverage of substantive elements of internal validity. To minimise the subjectivity of the assessment, tools with a detailed and precise operationalisation of the individual elements should be applied. For health economic studies, tools should be developed and complemented with instructions, which define the appropriateness of the criteria. Further research is needed to identify study characteristics that influence the internal validity of studies.