BACKGROUND: Digital health interventions (DHIs) have the potential to enable public end users, such as citizens and patients, to manage and improve their health. Although the number of available DHIs is increasing, examples of successfully established DHIs in public health systems are limited. To counteract the nonuse of DHIs, they should be comprehensively evaluated while integrating end users. Unfortunately, there is a wide variability and heterogeneity according to the approaches of evaluation, which creates a methodological challenge.
OBJECTIVE: This scoping review aims to provide an overview of the current established processes for evaluating DHIs, including methods, indicators, and end-user involvement. The review is not limited to a specific medical field or type of DHI but offers a holistic overview.
METHODS: This scoping review was conducted following the JBI methodology for scoping reviews based on the framework by Arksey & O\'Malley and complies with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Three scientific databases (PubMed, Scopus, and Science Direct) were searched in April 2023. English and German studies between 2008 and 2023 were considered when evaluating DHIs that explicitly address public end users. The process of study selection was carried out by several researchers to avoid reviewer bias.
RESULTS: The search strategy identified 9618 publications, of which 160 were included. Among these included articles, 200 evaluations were derived and analyzed. The results showed that there is neither a consensus on the methods to evaluate DHIs nor a commonly agreed definition or usage of the evaluated indicators, which results in a broad variety of evaluation practices. This aligns with observations of the existing literature. It was found that there is a lack of references to existing frameworks for the evaluation of DHIs. The majority of the included studies referred to user-centered approaches and involved end users in the evaluation process. As assistance for people developing and evaluating DHIs and as a basis for thinking about appropriate ways to evaluate DHIs, a results matrix was created where the findings were combined per DHI cluster. Additionally, general recommendations for the evaluators of DHIs were formulated.
CONCLUSIONS: The findings of this scoping review offer a holistic overview of the variety and heterogeneity according to the approaches of evaluation of DHIs for public end users. Evaluators of these DHIs should be encouraged to reference established frameworks or measurements for justification. This would ease the transferability of the results among similar evaluation studies within the digital health sector, thereby enhancing the coherence and comparability of research in this area.

BACKGROUND: Use of participatory research methods is increasing in research trials. Once partnerships are established with end-users, there is less guidance about processes research teams can use to successfully incorporate end-user feedback. The current study describes the use of a brief reflections process to systematically examine and evaluate the impact of end-user feedback on study conduct.
METHODS: The Comparative Effectiveness of Trauma-Focused and Non-Trauma- Focused Treatment Strategies for PTSD among those with Co-Occurring SUD (COMPASS) study was a randomized controlled trial to determine the effectiveness of trauma-focused psychotherapy versus non-trauma-focused psychotherapy for Veterans with co-occurring posttraumatic stress disorder and substance use disorder who were entering substance use treatment within the Department of Veterans Affairs. We developed and paired a process of \"brief reflections\" with our end-user engagement methods as part of a supplemental evaluation of the COMPASS study engagement plan. Brief reflections were 30-minute semi-structured discussions with the COMPASS Team following meetings with three study engagement panels about feedback received regarding study issues. To evaluate the impact of panel feedback, 16 reflections were audio-recorded, transcribed, rapidly analyzed, and integrated with other study data sources.
RESULTS: Brief reflections revealed that the engagement panels made recommended changes in eight areas: enhancing recruitment; study assessment completion; creating uniformity across Study Coordinators; building Study Coordinator connection to Veteran participants; mismatch between study procedures and clinical practice; therapist skill with patients with active substance use; therapist burnout; and dissemination of study findings. Some recommendations positively impact study conduct while others had mixed impact. Reflections were iterative and led to emergent processes that included revisiting previously discussed topics, cross-pollination of ideas across panels, and sparking solutions amongst the Team when the panels did not make any recommendations or recommendations were not feasible.
CONCLUSIONS: When paired with end-user engagement methods, brief reflections can facilitate systematic examination of end-user input, particularly when the engagement strategy is robust. Reflections offer a forum of accountability for researchers to give careful thought to end-user recommendations and make timely improvements to the study conduct. Reflections can also facilitate evaluation of these recommendations and reveal end-user-driven strategies that can effectively improve study conduct.
BACKGROUND: ClinicalTrials.gov (NCT04581434) on October 9, 2020; https://clinicaltrials.gov/ct2/show/study/NCT04581434?term=NCT04581434&draw=2&rank=1 .

In seasonally frozen regions, concrete pavement is exposed to cycles of freeze-thaw and erosion from de-icing salt, which can lead to unfavorable service conditions and vulnerability to damage. This paper examines the compressive strength, flexural-tensile strength, abrasion resistance, permeability, and spacing factor of concrete, taking into account the impact of various curing conditions, de-icing salt solutions, and mass fractions on the concrete\'s freeze-thaw resistance. Two test methods, the single-face method and the fast-freezing method, were used to comparatively analyze the freeze-thaw resistance of concrete. The analysis was based on the surface scaling, water absorption rate, mass loss rate, relative dynamic elastic modulus, and relative durability index. The results indicate that the presence of salt solution significantly worsened the degree of concrete damage caused by freeze-thaw cycles. The use of freeze-thaw media, specifically sodium chloride (NaCl), calcium chloride (CaCl2), and potassium acetate (KAc) at mass fractions of 5%, 4.74%, and 5%, respectively, had the greatest impact on the surface scaling of concrete. However, their effect on the water absorption rate was inconsistent. When the freeze-thaw medium was water, the concrete\'s relative dynamic elastic modulus and relative durability index were 9.6% and 75.3% higher, respectively, for concrete cured in 20 °C-95% RH conditions compared to those cured in 0 °C-50% RH conditions. We propose a comprehensive relative durability index (DFw) by combining the results of two methods of freeze-thaw tests. The DFw of concrete cured in 0 °C-50% RH conditions was 83.8% lower than that of concrete cured in 20 °C-95% RH conditions when exposed to a freeze-thaw medium of 5% mass fraction NaCl solution. To evaluate the salt freeze-thaw resistance of concrete pavement, it is recommended to use surface scaling and DFw together.

暂无摘要。

Digital interventions often suffer from low usage, which may reflect insufficient attention to user experience. Moreover, the existing evaluation methods have limited applicability in the remote study of user experience of complex interventions that have expansive content and that are used over an extensive period of time. To alleviate these challenges, we describe here a novel qualitative Ecological Momentary Assessment (EMA) method: the CORTO method (Contextual, One-item, Repeated, Timely, Open-ended). We used it to gather digital intervention user experience data from Finnish adults (n = 184) who lived with interview-confirmed major depressive disorder (MDD) and took part in a randomized controlled trial (RCT) that studied the efficacy of a novel 12-week game-based digital intervention for depression. A second dataset on user experience was gathered with retrospective interviews (n = 22). We inductively coded the CORTO method and retrospective interview data, which led to four user experience categories: (1) contextual use, (2) interaction-elicited emotional experience, (3) usability, and (4) technical issues. Then, we used the created user experience categories and Template Analysis to analyze both datasets together, and reported the results qualitatively. Finally, we compared the two datasets with each other. We found that the data generated with the CORTO method offered more insights into usability and technical categories than the interview data that particularly illustrated the contextual use. The emotional valence of the interview data was more positive compared with the CORTO data. Both the CORTO and interview data detected 55 % of the micro-level categories; 20 % of micro-level categories were only detected by the CORTO data and 25 % only by the interview data. We found that the during-intervention user experience measurement with the CORTO method can provide intervention-specific insights, and thereby further the iterative user-centered intervention development. Overall, these findings highlight the impact of evaluation methods on the categories and qualities of insights acquired in intervention research.

National health services in Ireland and the UK fund the majority of social prescribing services and have issued recommendations for evaluation. However, it is not known what outcomes are prioritised for evaluation within individual services and what evaluation methods are used to capture recommended outcomes. A survey was carried out to examine evaluation practices of social prescribing services on the island of Ireland. This study used a cross-sectional observational design. The sample was all the staff involved in delivering and/or managing SP services on the island of Ireland. Questionnaires were distributed at a national SP conference and online. Closed-response questions were analysed using descriptive statistics. Content analysis was used for open-ended questions. Eighty-four usable surveys were returned (50% from the Republic of Ireland and 50% from Northern Ireland). All respondents (100%) agreed on the importance of measuring SP outcomes. The most frequently measured outcomes were health and well-being (89.2%) and loneliness (84%). The least frequently measured outcome was the satisfaction of healthcare professionals referring to SP: 78.4% of respondents never measured this outcome. The most frequently used measurement tool was the Short Warwick Edinburgh Mental Well-Being Scale, with 38/76 (50%) respondents using this measure. There was a lack of standardised measures identified for some outcomes. For example, 70% of respondents reported always measuring physical activity (PA), but only four respondents identified a specific PA measure. In open-ended questions, respondents recommended flexibility in evaluation methods to reflect the complexity and individualised focus of SP. They also identified the need for protected time to complete evaluations and recommended a national strategy to inform priorities in evaluations. This study demonstrates a wide variation on the island of Ireland on how SP services are measuring outcomes, with many outcomes rarely or never measured using standardised measures. Agreement is needed on a core outcome set for social prescribing in order to guide service delivery and evaluations.

In recent years, exoskeleton test methods for industrial exoskeletons have evolved to include simulated laboratory and field environments. Physiological, kinematic, and kinetic metrics, as well as subjective surveys, are used to evaluate exoskeleton usability. In particular, exoskeleton fit and usability can also impact the safety of exoskeletons and their effectiveness at reducing musculoskeletal injuries. This paper surveys the state of the art in measurement methods applied to exoskeleton evaluation. A notional classification of the metrics based on exoskeleton fit, task efficiency, comfort, mobility, and balance is proposed. In addition, the paper describes the test and measurement methods used in supporting the development of exoskeleton and exosuit evaluation methods to assess their fit, usability, and effectiveness in industrial tasks such as peg in hole, load align, and applied force. Finally, the paper includes a discussion of how the metrics can be applied towards a systematic evaluation of industrial exoskeletons, current measurement challenges, and future research directions.

Price, Cost and Income (PCI) are distinct economic indicators intrinsically linked to the values they denote. These observables take center stage in the multi-criteria decision-making process that enables economic agents to convey subjective utilities of market-exchanged commodities objectively. The valuation of these commodities heavily relies on PCI-based empirical observables and their supported methodologies. This valuation measure\'s accuracy is critical, as it influences subsequent decisions within the market chain. However, measurement errors often arise due to inherent uncertainties in the value state, impacting economic agents\' wealth, particularly when trading significant commodities such as real estate properties. This paper addresses this issue by incorporating entropy measurements into real estate valuation. This mathematical technique adjusts and integrates triadic PCI estimates, improving the final stage of appraisal systems where definitive value decisions are crucial. Employing entropy within the appraisal system can also aid market agents in devising informed production/trading strategies for optimal returns. The results from our practical demonstration indicate promising implications. The entropy\'s integration with PCI estimates significantly improved the value measurement\'s precision and reduced economic decision-making errors.

Although experiences of loss and the consequent grief are natural in human life, some individuals may have difficulty managing these events, to the point of developing significant impairment in their functioning in important life areas. Given this, the present research aimed to explore the psychometric properties of the Italian version of the Adult Attitude to Grief scale (AAG) to facilitate research on adult vulnerability to grief among Italian-speaking populations. A sample of 367 participants (Mage = 30.44, SD = 11.21; 78% females) participated in this research. A back-translation procedure was implemented to develop the Italian AAG. Then, participants completed the Italian AAG alongside a battery of other self-report psychometric scales in order to assess aspects of the construct validity of the AAG: the Forty-Item Defense Style Questionnaire, the Impact of Event Scale-Revised, and the Beck Depression Inventory-II. A bifactor structure was found to have the best fit to the data, supporting the possibility of using both the general factor (i.e., vulnerability) and three dimensions (i.e., overwhelmed, controlled, and resilient). Unlike the original version, the control dimension emerged as a \"protective\" factor in the Italian population, together with the resilient factor. Furthermore, results provided satisfactory indications of internal consistency and construct validity. In conclusion, the Italian AAG was shown to be a valid, reliable, quick, and easy-to-use scale that can be used both for research and clinical practice in the Italian context.

Novel single-cell-based technologies hold the promise of matching T cell receptor (TCR) sequences with their cognate peptide-MHC recognition motif in a high-throughput manner. Parallel capture of TCR transcripts and peptide-MHC is enabled through the use of reagents labeled with DNA barcodes. However, analysis and annotation of such single-cell sequencing (SCseq) data are challenged by dropout, random noise, and other technical artifacts that must be carefully handled in the downstream processing steps. We here propose a rational, data-driven method termed ITRAP (improved T cell Receptor Antigen Paring) to deal with these challenges, filtering away likely artifacts, and enable the generation of large sets of TCR-pMHC sequence data with a high degree of specificity and sensitivity, thus outputting the most likely pMHC target per T cell. We have validated this approach across 10 different virus-specific T cell responses in 16 healthy donors. Across these samples, we have identified up to 1494 high-confident TCR-pMHC pairs derived from 4135 single cells.