OBJECTIVE: In a time of exponential growth of new evidence supporting clinical decision-making, combined with a labor-intensive process of selecting this evidence, methods are needed to speed up current processes to keep medical guidelines up-to-date. This study evaluated the performance and feasibility of active learning to support the selection of relevant publications within medical guideline development and to study the role of noisy labels.
METHODS: We used a mixed-methods design. Two independent clinicians\' manual process of literature selection was evaluated for 14 searches. This was followed by a series of simulations investigating the performance of random reading versus using screening prioritization based on active learning. We identified hard-to-find papers and checked the labels in a reflective dialogue.
METHODS: Inter-rater reliability was assessed using Cohen\'s Kappa (ĸ). To evaluate the performance of active learning, we used the Work Saved over Sampling at 95% recall (WSS@95) and percentage Relevant Records Found at reading only 10% of the total number of records (RRF@10). We used the average time to discovery (ATD) to detect records with potentially noisy labels. Finally, the accuracy of labeling was discussed in a reflective dialogue with guideline developers.
RESULTS: Mean ĸ for manual title-abstract selection by clinicians was 0.50 and varied between - 0.01 and 0.87 based on 5.021 abstracts. WSS@95 ranged from 50.15% (SD = 17.7) based on selection by clinicians to 69.24% (SD = 11.5) based on the selection by research methodologist up to 75.76% (SD = 12.2) based on the final full-text inclusion. A similar pattern was seen for RRF@10, ranging from 48.31% (SD = 23.3) to 62.8% (SD = 21.20) and 65.58% (SD = 23.25). The performance of active learning deteriorates with higher noise. Compared with the final full-text selection, the selection made by clinicians or research methodologists deteriorated WSS@95 by 25.61% and 6.25%, respectively.
CONCLUSIONS: While active machine learning tools can accelerate the process of literature screening within guideline development, they can only work as well as the input given by human raters. Noisy labels make noisy machine learning.

BACKGROUND: Development of guidelines for public health, health system, and health policy interventions demands complex systems thinking to understand direct and indirect effects of interventions within dynamic systems. The WHO-INTEGRATE framework, an evidence-to-decision framework rooted in the norms and values of the World Health Organization (WHO), provides a structured method to assess complexities in guidelines systematically, such as the balance of an intervention\'s health benefits and harms and their human rights and socio-cultural acceptability. This paper provides a worked example of the application of the WHO-INTEGRATE framework in developing the WHO guidelines on parenting interventions to prevent child maltreatment, and shares reflective insights regarding the value added, challenges encountered, and lessons learnt.
METHODS: The methodological approach comprised describing the intended step-by-step application of the WHO-INTEGRATE framework and gaining reflective insights from introspective sessions within the core team guiding the development of the WHO guidelines on parenting interventions and a methodological workshop.
RESULTS: The WHO-INTEGRATE framework was used throughout the guideline development process. It facilitated reflective deliberation across a broad range of decision criteria and system-level aspects in the following steps: (1) scoping the guideline and defining stakeholder engagement, (2) prioritising WHO-INTEGRATE sub-criteria and guideline outcomes, (3) using research evidence to inform WHO-INTEGRATE criteria, and (4) developing and presenting recommendations informed by WHO-INTEGRATE criteria. Despite the value added, challenges, such as substantial time investment required, broad scope of prioritised sub-criteria, integration across diverse criteria, and sources of evidence and translation of insights into concise formats, were encountered.
CONCLUSIONS: Application of the WHO-INTEGRATE framework was crucial in the integration of effectiveness evidence with insights into implementation and broader implications of parenting interventions, extending beyond health benefits and harms considerations and fostering a whole-of-society-perspective. The evidence reviews for prioritised WHO-INTEGRATE sub-criteria were instrumental in guiding guideline development group discussions, informing recommendations and clarifying uncertainties. This experience offers important lessons for future guideline panels and guideline methodologists using the WHO-INTEGRATE framework.

Currently, there is no international standard for the methodology of patient versions of guideline development. In China, the development of patient versions of guidelines is still in its infancy, and there are no registered or published patient versions of guidelines in the field of acupuncture. This paper introduces two methods of developing patient versions of guidelines: directly converting clinical practice guidelines into patient versions guidelines and developing patient versions of guidelines independently. The relationship and differences between patient guidelines and clinical practice versions of guidelines are compared. By integrating the unique characteristics of acupuncture, this paper analyzes and discusses the significance, problems, and challenges of developing patient versions of guidelines in the field of acupuncture, aiming to provide methodological references for the future development of acupuncture patient versions of guidelines.
患者指南的制定方法目前国际上尚无统一的标准。我国患者指南的制定尚处于起步阶段，针灸领域亦无患者指南注册和发布。本文介绍了由临床实践指南直接转化的患者指南以及单独制定的患者指南两种制定方法，比较患者指南和临床实践指南的联系与区别。并结合针灸学自身特点，分析针灸领域制定患者指南的意义及其面临的问题与挑战，以期为未来针灸患者指南的制定提供方法学参考。.

BACKGROUND: Guidelines depend on effect estimates, usually derived from randomised controlled trials, to inform their decisions. Qualitative research evidence may improve decisions made but where in the process and the methods to do this have not been so clearly established. We sought to describe and appraise how qualitative research has been used to inform World Heath Organization guidance since 2020.
METHODS: We conducted a document analysis of WHO guidelines from 2020 to 2022. We purposely sampled guidelines on the topics of maternal and newborn health (MANH) and infectious diseases, as most of the qualitative synthesis to date has been conducted on these topics, likely representing the \'best case\' scenario. We searched the in-built repository feature of the WHO website and used standardised search terms to identify qualitative reporting. Using deductive frameworks, we described how qualitative evidence was used to inform guidelines and appraised the standards of this use.
RESULTS: Of the 29 guidelines, over half used qualitative research to help guide decisions (18/29). A total of 8 of these used qualitative research to inform the guideline scope, all 18 to inform recommendations, and 1 to inform implementation considerations. All guidelines drew on qualitative evidence syntheses (QES), and five further supplemented this with primary qualitative research. Qualitative findings reported in guidelines were typically descriptive, identifying people\'s perception of the benefits and harms of interventions or logistical barriers and facilitators to programme success. No guideline provided transparent reporting of how qualitative research was interpreted and weighed used alongside other evidence when informing decisions, and only one guideline reported the inclusion of qualitative methods experts on the panel. Only a few guidelines contextualised their recommendations by indicating which populations and settings qualitative findings could be applied.
CONCLUSIONS: Qualitative research frequently informed WHO guideline decisions particularly in the field of MANH. However, the process often lacked transparency. We identified unmet potential in informing implementation considerations and contextualisation of the recommendations. Use in these areas needs further methods development.

BACKGROUND: The success of abdominal cancer surgery depends not only on the surgery itself but is influenced by the overall perioperative management. Given the multitude of perioperative measures and the ever-increasing number of studies on perioperative management, it is difficult to keep track and provide evidence-based perioperative management. The planned guideline on perioperative management will review the existing evidence and derive treatment recommendations.
METHODS: The processing of the evidence is carried out by 6 working groups according to an 8-step scheme: after drafting the guideline questions in PICO format (1), a systematic literature search is carried out (2), and the records found are screened by two independent reviewers from the coordination team. Subsequently, the full texts of the potentially relevant articles are made available to the working groups for full text screening (3). All articles to be included are reviewed for methodological quality (4) before summary of findings tables are generated (5). In line with the GRADE approach, confidence in the evidence is assessed (6) before a recommendation is derived from the evidence, using a modified GRADE Evidence to Decision Framework (7). Finally, all recommendations are compiled and agreed within the guideline group (8).
CONCLUSIONS: Guidelines serve as foundation for therapy decisions in everyday clinical practice and should therefore be based on up-to-date research results. However, while primary studies and systematic reviews are critically reviewed for their methodological quality, the process of guideline development is often not comprehensible. A protocol with predefined methodology should therefore create transparency and strengthen confidence in the recommendations.
BACKGROUND: The guideline is registered in the AWMF (Association of the Scientific Medical Societies) Guideline Register (088-010OL).

OBJECTIVE: A core outcome set (COS) is an agreed standardized set of outcomes that should be measured and reported, as a minimum, in specific areas of health or health care. A COS is developed through a consensus process to ensure health care outcomes to be measured are relevant to decision-makers, including patients and health-care professionals. Use of COS in guideline development is likely to increase the relevance of the guideline to those decision-makers. Previous work has looked at the uptake of COS in trials, systematic reviews, health technology assessments and regulatory guidance but to date there has been no evaluation of the use of COS in practice guideline development. The objective of this study was to investigate the representation of core outcomes in a set of international practice guidelines.
METHODS: We searched for clinical guidelines relevant to ten high-quality COS (with focus on the United Kingdom, Germany, China, India, Canada, Denmark, United States and World Health Organisation). We matched scope between COS and guideline in terms of condition, population and outcome. We calculated the proportion of guidelines mentioning or referencing COS and the proportion of COS domains specifically, or generally, matching to outcomes specified in each guideline populations, interventions, comparators and outcome (PICO) statement.
RESULTS: We found 38 guidelines that contained 170 PICO statements matching the scope of the ten COS and of sufficient quality to allow data extraction. None of the guidelines reviewed explicitly mentioned or referenced the relevant COS. The median (range) of the proportion of core outcomes covered either specifically or generally by the guideline PICO was 30% (0%-100%).
CONCLUSIONS: There is no evidence that COS are being used routinely to inform the guideline development process, and concordance between outcomes in published guidelines and those in COS is limited. Further work is warranted to explore barriers and facilitators in the use of COS when developing clinical guidelines.

BACKGROUND: PANELVIEW is an instrument for evaluating the appropriateness of the process, methods, and outcome of guideline development and the satisfaction of the guideline group with these steps.
OBJECTIVE: To evaluate the guideline development process of the German guideline on the treatment of patients with severe/multiple injuries (\'German polytrauma guideline\') from the perspective of the guideline group, and to identify areas where this process may be improved in the future.
METHODS: We administered PANELVIEW to the participants of the 2022 update of the German polytrauma guideline. All guideline group members, including delegates of participating medical societies, steering group members, authors of guideline chapters, the chair, and methodological lead, were invited to participate. Responses were analysed using descriptive statistics. Comments received were categorised by domains/items of the tool.
RESULTS: After the first, second, and last consensus conference, the guideline group was invited via email to participate in a web-based survey. Response rates were 36% (n/N = 13/36), 40% (12/30), and 37% (20/54), respectively. The mean scores for items ranged between 5.1 and 6.9 on a scale from 1 (fully disagree) to 7 (fully agree). Items with mean scores below 6.0 were related to (1) administration, (2) consideration of patients\' views, perspectives, values, and preferences, and (3) the discussion of research gaps and needs for future research.
CONCLUSIONS: The PANELVIEW tool showed that the guideline group was satisfied with most aspects of the guideline development process. Areas for improvement of the process were identified. Strategies to improve response rates should be explored.

BACKGROUND: Several guideline organizations produce patient versions of clinical practice guidelines (PVGs) which translate recommendations into simple language. A former study of our working group revealed that few guideline organizations publish their methods used to develop PVGs. Clear definitions of PVGs do not prevail and their purposes often remain unclear. We aimed to explore experts\' perspectives on developing, disseminating and implementing PVGs to discuss and incorporate these experiences when consenting on methodological guidance and further improving PVGs.
METHODS: We conducted 17 semi-structured telephone interviews with international experts working with PVGs from September 2021 through January 2022. We conducted the interviews in English or German, they were recorded and transcribed verbatim. We utilized Mayring\'s qualitative content analysis with MAXQDA software to analyze the data.
RESULTS: In two interviews two participants were interviewed at the same time. This resulted in a total of 19 participants from 16 different organizations and eight different countries participated. Most were female (16/19) and their experience in working with PVGs ranged from 1 to 20 years. All follow methodological standards when developing PVGs, but the extent of these standards and their public accessibility differs. Aims and target groups of PVGs vary between organizations. Facilitators for developing PVGs are working with a multidisciplinary team, financial resources, consultation processes and a high-quality underlying CPG. Facilitators for disseminating and implementing PVGs are using various strategies. Barriers, on the other hand, are the lack of these factors. All participants mentioned patient involvement as a key aspect in PVG development.
CONCLUSIONS: The steps in the PVG development process are largely similar across the countries. Focus is placed on the involvement of patients in the development process, although the extent of participation varies. The experts collectively attribute great importance to PVGs overall, but in order to constantly adapt to medical progress and changing conditions, the focus in the future may be more on formats like living guidelines. Although there are different views on the mandatory development of PVGs, there is a consistent call for more transparency regarding the methodology used for PVGs.

OBJECTIVE: To formally evaluate the uptake and reporting of the Grading of Recommendation Assessments, Development and Evaluation (GRADE) approach in clinical practice guidelines (CPGs) developed by the Eastern Association for the Surgery of Trauma (EAST).
METHODS: Based on an a priori, written protocol, we developed a dedicated data abstraction form that included the six suggested criteria for using and applying GRADE. By searching the EAST website, we identified all EAST guidelines that referenced the use of GRADE. All steps of the data abstraction process were completed independently and in duplicate by two members of the research team.
RESULTS: We identified a total of 48 CPGs that used GRADE. Trauma and violence prevention (n = 11; 23.9%) was the most common topic. The median number of patient/population, intervention, comparison, and outcomes (PICO) questions addressed was 3 (interquartile range: 2; 4) with a median of 2.5 (interquartile range: 1; 4) critical outcomes. A conditional/weak recommendation was provided for n = 79 (51.4%) PICOs, whereas a strong recommendation was provided for 33 PICOs (23.9%). For 22 PICOs (15.9%), no recommendation was made. Nearly all guideline documents provided search dates (n = 44; 95.7%) and a Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram (n = 44; 95.7%). Most described categories for rating down (n = 35; 76.1%). GRADE decision-making domains related to the ratio of benefits to harms, values and preferences, and resource utilization were referenced by 43.5% (n = 20), 43.5% (n = 20), and 30.4% (n = 14) of CPGs, respectively. For nearly half of PICO questions (n = 59; 44.2%) authors did not provide an evidence profile or summary of findings table. Comparing time periods from 2014-2018 to 2019-2022, the proportion of recommendations with an overall certainty of evidence increased (52.4% vs 83.9%; P < 0.001).
CONCLUSIONS: EAST has successfully adopted GRADE to develop many trauma-related guidelines, each addressing a finite number of focused clinical questions based on systematic reviews conducted in-house. Overall reporting improved over time. There is for improvement when it comes to consistent provision of an overall certainty of evidence, the reporting of the evidence to decision-making process, and the justification of strong recommendations based on low/very low certainty evidence.

The development of the new S3 guideline \"Epidemiology, diagnosis, treatment and follow-up of the bladder exstrophy-epispadias complex\" was funded by the German Innovation Fund of the Federal Joint Committee (G-BA). Despite the relatively low level of evidence of the identified literature, a systematic approach and consistent evaluation of the literature enabled the formulation of a large number of evidence-based recommendations across a variety of topics. In addition, a patient guideline is under development in order to disseminate the guideline recommendations and to enhance self-management and understanding among patients and their relatives. A needs analysis had been carried out in order to adequately assess the topics that are most important for patients and relatives. Upon completion of the German guideline, an English translation in cooperation with the e‑UROGEN network is planned.
UNASSIGNED: Die Erstellung der neuen S3-Leitlinie „Epidemiologie, Diagnostik, Therapie und Nachsorge des Blasenekstrophie-Epispadie Komplexes“ wurde durch 2021 beantragte Forschungsgelder vom Innovationsfonds des Gemeinsamen Bundesausschusses (G-BA) ermöglicht. Wenngleich das Evidenzlevel der identifizierten Literatur vergleichsweise niedrig ist, konnten durch ein systematisches Vorgehen und eine konsequente Bewertung der Literatur eine Vielzahl von evidenzbasierten Empfehlungen zu vielen Themen formuliert werden. Zudem wird eine Patient*innenleitlinie erstellt, um die Leitlinienempfehlungen in die Breite zu bringen und das Selbstmanagement und Verständnis der Betroffenen und Zugehörigen zu fördern. Um die für Patient*innen wichtigen Themen in der Patient*innenleitlinie adäquat abzubilden, wurde im Vorfeld eine Bedarfsanalyse durchgeführt. Nach Abschluss der Arbeiten an der deutschen Leitlinie ist mit dem e‑UROGEN-Netzwerk eine Kooperation für eine englische Übersetzung geplant.