BACKGROUND: Digital health interventions (DHIs) have the potential to enable public end users, such as citizens and patients, to manage and improve their health. Although the number of available DHIs is increasing, examples of successfully established DHIs in public health systems are limited. To counteract the nonuse of DHIs, they should be comprehensively evaluated while integrating end users. Unfortunately, there is a wide variability and heterogeneity according to the approaches of evaluation, which creates a methodological challenge.
OBJECTIVE: This scoping review aims to provide an overview of the current established processes for evaluating DHIs, including methods, indicators, and end-user involvement. The review is not limited to a specific medical field or type of DHI but offers a holistic overview.
METHODS: This scoping review was conducted following the JBI methodology for scoping reviews based on the framework by Arksey & O\'Malley and complies with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Three scientific databases (PubMed, Scopus, and Science Direct) were searched in April 2023. English and German studies between 2008 and 2023 were considered when evaluating DHIs that explicitly address public end users. The process of study selection was carried out by several researchers to avoid reviewer bias.
RESULTS: The search strategy identified 9618 publications, of which 160 were included. Among these included articles, 200 evaluations were derived and analyzed. The results showed that there is neither a consensus on the methods to evaluate DHIs nor a commonly agreed definition or usage of the evaluated indicators, which results in a broad variety of evaluation practices. This aligns with observations of the existing literature. It was found that there is a lack of references to existing frameworks for the evaluation of DHIs. The majority of the included studies referred to user-centered approaches and involved end users in the evaluation process. As assistance for people developing and evaluating DHIs and as a basis for thinking about appropriate ways to evaluate DHIs, a results matrix was created where the findings were combined per DHI cluster. Additionally, general recommendations for the evaluators of DHIs were formulated.
CONCLUSIONS: The findings of this scoping review offer a holistic overview of the variety and heterogeneity according to the approaches of evaluation of DHIs for public end users. Evaluators of these DHIs should be encouraged to reference established frameworks or measurements for justification. This would ease the transferability of the results among similar evaluation studies within the digital health sector, thereby enhancing the coherence and comparability of research in this area.

BACKGROUND: The exponential growth in the use of mobile health (mHealth) applications in patient care has highlighted the importance of understanding end-users\' needs for successful interventions, achievable through the process of co-design. This review aims to summarize previously published uses of co-design in mHealth applications. It will inform the development of a local mHealth tool in pediatric surgery.
METHODS: The rapid review was conducted following Cochrane Rapid Review guidelines. We searched four databases from inception until August 2021 for collaborative design in mHealth apps. Two reviewers independently screened titles and abstracts using Rayyan, with a third reviewer resolving any conflicts. Studies were excluded if they were non-empirical or in other languages than English.
RESULTS: Out of 984 records screened, 175 were included for full-text screening, and 27 met all eligibility criteria. The purpose of the studies related to care support, communication, self-management or information management. All interventions involved their target audience in the co-design process, and 85% included healthcare professionals for expert opinion. Stakeholder participation was solicited via interviews, surveys, focus groups, design sessions, and prototype evaluation. To initiate the process, 15 studies included needs identification, 8 did background research, and 7 performed literature reviews.
CONCLUSIONS: Co-design has a positive impact on mHealth development, research processes and outcomes. Its success relies on a clearly identified framework of terminology, activities involved, and evaluation techniques. The findings of this review will be integrated into a novel co-design framework, which will be used to inform the development of a pediatric surgery mHealth application.
METHODS: This review uses manuscripts that are rated level III-V.

Young handball players experience high injury rates. Specific injury prevention programs reduce injury rates but are not well implemented into youth players\' training. The \'Implementing injury Prevention training ROutines in TEams and Clubs in youth Team handball (I-PROTECT)\' project addresses this challenge. The aim of this study was to investigate how youth handball coaches and players experienced the recently developed I-PROTECT GO pilot program, by focusing on barriers and facilitators. Three focus group interviews were conducted with coaches and players, and their answers were analyzed using a general inductive approach. The participants appreciated the program and found it useful for their context. The participants\' statements about facilitators and barriers centered around the categories of resources, exercises, program design, and learning. Facilitators that emerged were motivating exercises (e.g., handball-specific), a helpful set-up (e.g., variation), having a clear purpose of exercises, the possibility to fulfil basic psychological needs while training, receiving instructions and feedback, and having role models. Barriers that emerged were limited space and material, difficulties with exercises, an unhelpful set-up (e.g., too repetitive), and undisciplined training. It is important to address perceived barriers and facilitators among coaches and players when developing injury prevention training programs to enhance the uptake of such training.

To design and evaluate a mental health treatment program and internet-based delivery platform for patients with ischemic heart disease (IHD) attending cardiac rehabilitation with the aim of reducing the risks associated with anxiety and/or depression. Patients diagnosed with IHD and comorbid anxiety and/or depression. Participatory design of treatment program and internet platform through staged inclusion of participants in two groups. Group 1 was enrolled as co-researchers with prolonged engagement in the project. Group 2 participated only in the pilot evaluation workshop. Three patients were included in Group 1, two patients in Group 2. Inclusion of patients proved challenging, but the extended collaboration with co-researchers yielded valuable circumstantial insight and resulted in the design of a novel nine-module treatment program. Additionally, the inclusion of two participant groups helped shape the development of an internet platform based on an open-source content management system. Our grouped participation method contributes with several recommendations and reflections of advantages of this approach. Collaboration with co-researchers helped us gain a deeper understanding of the impact of language on self-perception and potential stigma. Prolonged participation led to a higher level of trust and familiarity, which enabled uncovering of issues otherwise hidden.

The PEDI-CAT measures daily functioning of children and youth, aged 1 to 21 years, with a variety of physical, cognitive and/or behavioral disabilities. In order to use an instrument in another culture or language, translation and cross-cultural validation are important, particularly for end-users. This study describes the process of translation and cross-cultural adaptation of the Dutch version of the PEDI-CAT.
End-users were involved in all steps. First, the PEDI-CAT items were reviewed to determine whether the items were relevant and acceptable in the Dutch culture. Then, the PEDI-CAT was translated into Dutch using specific guidelines. Finally, the wording of the Dutch items and response options were reviewed and tested with 22 parents of children and adolescents with and without disabilities.
All 267 items and response options of the original PEDI-CAT were assessed as relevant and translated into Dutch. A selection of 175 items was tested with Think Aloud interviews which revealed that the translation of 46 items could be improved.
The role of end-users in the process of translation and cross-cultural adaptation was crucial. This collaborative process resulted in a Dutch version of the PEDI-CAT that has been optimally adapted to the Dutch language and culture.

Many researchers and practitioners argue the importance of end-user involvement in workplace safety management, but the research literature and practices remain fractured across orientations. The primary aim of this article is to bridge the gap between two major participatory safety management approaches: behavioral safety and participatory ergonomics. First, an overview and brief history of participative management is presented to provide context for its application to workplace safety. Next, behavioral safety and participatory ergonomics are separately examined in terms of their origins and core elements. Finally, based on this examination, unifying elements between behavioral safety and participatory ergonomics will be presented to provide a comprehensive account of participatory safety management.

UNASSIGNED: For the elderly to get the care and services they need, they may need to make the difficult decision about staying in their home or moving to another home. Many other people may be involved in their care too (friends, family and healthcare providers), and can support them in making the decision. We asked informal caregivers of elderly people to help us develop a decision guide to support them and their loved ones in making this decision. This guide will be used by health providers in home care who are trained to help people make decisions. The guide is in French and English. To design and test this decision guide we involved elderly people, their caregivers and health administrators. We first asked them what they needed for making the decision, and then designed a first version of the guide. Then we asked them to look at it and give feedback, which was used to make the final version. We then used scientific criteria to check its content and the language used. The final decision guide was acceptable to the caregivers, their elderly loved ones, and the health administrators. The guide is currently being evaluated in a large research project with home care teams in the province of Quebec.
UNASSIGNED: Background As they grow older, many elderly people are faced with the difficult and preference-sensitive decision about staying in their home or moving to a residence better adapted to their evolving care needs. We aimed to develop an English and French decision aid (DA) for elderly people facing this decision, and to involve end-users in all phases of the development process. Methods A three-cycle design with involvement of end-users in Quebec. End-users were elderly people (n = 4) caregivers of the elderly (n = 5), health administrators involved in home-care service delivery or policy (n = 6) and an interprofessional research team (n = 19). Cycle 1: Decisional needs assessment and development of the first prototype based on existing tools and input from end-users; overview of reviews examining the impact of location of care on elderly people\'s health outcomes. Cycle 2: Usability testing with end-users, adaptation of prototype. Cycle 3: Refinement of the prototype with a linguist, graphic designer and end-users. The final prototype underwent readability testing and an International Patient Decision Aids (IPDAS) criteria compatibility assessment to verify minimal requirements for decision aids and was tested for usability by the elderly. ResultsCycle 1: We used the Ottawa Personal Decision Guide to design a first prototype. As the overview of reviews did not find definitive evidence regarding optimal locations of care for elderly people, we were not able to add evidence-based advantages and disadvantages to the guide. Cycle 2: Overall, the caregivers and health administrators who evaluated the prototype (n = 10) were positive. In response to their suggestions, we deleted some elements (overview of pros, cons, and consequences of the options) that were necessary to qualify the tool as a DA and renamed it a \"decision guide\". Cycle 3: We developed French and English versions of the guide, readable at a primary school level. The elderly judged the guide as acceptable. Conclusion We developed a decision guide to support elderly people and their caregivers in decision making about location of care. This paper is one of few to report on a fully collaborative approach to decision guide development that involves end-users at every stage (caregivers and health administrators early on, the frail elderly in the final stages). The guide is currently being evaluated in a cluster randomized trial. Trial registration: NCT02244359.

The benefits of end-user involvement in health-care research are widely recognized by research agencies. There are few published evaluations of end-user involvement in systematic reviews.
(i) Describe end-user involvement in a complex mixed-methods systematic review of ADHD in schools, (ii) reflect on the impact of end-user involvement, (iii) highlight challenges and benefits experienced and (iv) provide suggestions to inform future involvement.
End-users were involved in all stages of the project, both as authors and as members of an advisory group. In addition, several events were held with groups of relevant end-users during the project.
End-user input (i) guided the direction of the research, (ii) contributed to a typology of interventions and outcomes, (iii) contributed to the direction of data analysis and (iv) contributed to the robustness of the syntheses by demonstrating the alignment of interim findings with lived experiences. Challenges included (i) managing expectations, (ii) managing the intensity of emotion, (iii) ensuring that involvement was fruitful for all not just the researcher, (iv) our capacity to communicate and manage the process and (v) engendering a sense of involvement amongst end-users.
End-user involvement was an important aspect of this project. To minimize challenges in future projects, a recognition by the project management team and the funding provider that end-user involvement even in evidence synthesis projects is resource intensive is essential to allow appropriate allocation of time and resources for meaningful engagement.