UNASSIGNED: To assess the external validity of randomized controlled trials (RCTs) of bariatric surgical treatment on diabetes control.
UNASSIGNED: Multisite RCTs provide the strongest evidence supporting clinical treatments and have the greatest internal validity. However, characteristics of trial participants may not be representative of patients receiving treatment in the real world. There is a need to assess how the results of RCTs generalize to all contemporary patient populations undergoing treatments.
UNASSIGNED: All patients undergoing sleeve gastrectomy at University of California Los Angeles (UCLA) between January 8, 2018 and May 19, 2023 had their baseline characteristics, weight change, and diabetes control compared with those enrolled in the surgical treatment and medications potentially eradicate diabetes efficiently (STAMPEDE) and diabetes surgery study (DSS) RCTs of bariatric surgery\'s effect on diabetes control. Weight loss and diabetes control were compared between UCLA patients who did and did not fit the entry criteria for these RCTs.
UNASSIGNED: Only 65 (17%) of 387 patients with diabetes fulfilled the eligibility criteria for STAMPEDE, and 29 (7.5%) fulfilled the criteria for DSS due to being older, having higher body mass index, and lower HbA1c. UCLA patients experienced slightly less weight loss than patients in the RCTs but had similar diabetes control. The 313 (81%) patients not eligible for study entry into either RCT had similar long-term diabetes control as those who were eligible for the RCTs.
UNASSIGNED: Even though only a very small proportion of patients undergoing bariatric surgery met the eligibility criteria for the 2 major RCTs, most patients in this contemporary cohort had similar outcomes. Diabetes outcomes from STAMPEDE and DSS generalize to most patients undergoing bariatric surgery for diabetes control.

BACKGROUND: Educational neuroscience research, which investigates the neurobiological mechanisms of learning, has historically incorporated samples drawn mostly from white, middle-class, and/or suburban populations. However, sampling in research without attending to representation can lead to biased interpretations and results that are less generalizable to an intended target population. Prior research revealing differences in neurocognitive outcomes both within- and across-groups further suggests that such practices may obscure significant effects with practical implications.
UNASSIGNED: Negative attitudes among historically marginalized communities, stemming from historical mistreatment, biased research outcomes, and implicit or explicit attitudes among research teams, can hinder diverse participation. Qualities of the research process including language requirements, study locations, and time demands create additional barriers.
METHODS: Flexible data collection approaches, community engaugement, and transparent reporting could build trust and enhance sampling diversity. Longer-term solutions include prioritizing research questions relevant to marginalized communities, increasing workforce diversity, and detailed reporting of sample demographics. Such concerted efforts are essential for robust educational neuroscience research to maximize positive impacts broadly across learners.

UNASSIGNED: The Adult Changes in Thought (ACT) study is a cohort of Kaiser Permanente Washington members ages 65+ that began in 1994.
UNASSIGNED: We wanted to know how well ACT participants represented all older adults in the region, and how well ACT findings on eye disease and its relationship with Alzheimer\'s disease generalized to all older adults in the Seattle Metropolitan Region.
UNASSIGNED: We used participation weights derived from pooling ACT and Behavioral Risk Factor Surveillance System (BRFSS) data to estimate prevalences of common eye diseases and their associations with Alzheimer\'s disease incidence. Cox proportional hazards models accounted for age, education, smoking, sex, and APOE genotype. Confidence intervals for weighted analyses were bootstrapped to account for error in estimating the weights.
UNASSIGNED: ACT participants were fairly similar to older adults in the region. The largest differences were more self-reported current cholesterol medication use in BRFSS and higher proportions with low education in ACT. Incorporating the weights had little impact on prevalence estimates for age-related macular degeneration or glaucoma. Weighted estimates were slightly higher for diabetic retinopathy (weighted 5.7% (95% Confidence Interval 4.3, 7.1); unweighted 4.1% (3.6, 4.6)) and cataract history (weighted 51.8% (49.6, 54.3); unweighted 48.6% (47.3, 49.9)). The weighted hazard ratio for recent diabetic retinopathy diagnosis and Alzheimer\'s disease was 1.84 (0.34, 4.29), versus 1.32 (0.87, 2.00) in unweighted ACT.
UNASSIGNED: Most, but not all, associations were similar after participation weighting. Even in community-based cohorts, extending inferences to broader populations may benefit from evaluation with participation weights.

BACKGROUND: The PRIME-NL study prospectively evaluates a new integrated and personalized care model for people with parkinsonism, including Parkinson\'s disease, in a selected region (PRIME) in the Netherlands. We address the generalizability and sources of selection and confounding bias of the PRIME-NL study by examining baseline and 1-year compliance data.
METHODS: First, we assessed regional baseline differences between the PRIME and the usual care (UC) region using healthcare claims data of almost all people with Parkinson\'s disease in the Netherlands (the source population). Second, we compared our questionnaire sample to the source population to determine generalizability. Third, we investigated sources of bias by comparing the PRIME and UC questionnaire sample on baseline characteristics and 1-year compliance.
RESULTS: Baseline characteristics were similar in the PRIME (n = 1430) and UC (n = 26,250) source populations. The combined questionnaire sample (n = 920) was somewhat younger and had a slightly longer disease duration than the combined source population. Compared to the questionnaire sample in the PRIME region, the UC questionnaire sample was slightly younger, had better cognition, had a longer disease duration, had a higher educational attainment and consumed more alcohol. 1-year compliance of the questionnaire sample was higher in the UC region (96%) than in the PRIME region (92%).
CONCLUSIONS: The generalizability of the PRIME-NL study seems to be good, yet we found evidence of some selection bias. This selection bias necessitates the use of advanced statistical methods for the final evaluation of PRIME-NL, such as inverse probability weighting or propensity score matching. The PRIME-NL study provides a unique window into the validity of a large-scale care evaluation for people with a chronic disease, in this case parkinsonism.

BACKGROUND: Deep learning has been increasingly investigated for assisting clinical in vitro fertilization (IVF). The first technical step in many tasks is to visually detect and locate sperm, oocytes, and embryos in images. For clinical deployment of such deep learning models, different clinics use different image acquisition hardware and different sample preprocessing protocols, raising the concern over whether the reported accuracy of a deep learning model by one clinic could be reproduced in another clinic. Here we aim to investigate the effect of each imaging factor on the generalizability of object detection models, using sperm analysis as a pilot example.
METHODS: Ablation studies were performed using state-of-the-art models for detecting human sperm to quantitatively assess how model precision (false-positive detection) and recall (missed detection) were affected by imaging magnification, imaging mode, and sample preprocessing protocols. The results led to the hypothesis that the richness of image acquisition conditions in a training dataset deterministically affects model generalizability. The hypothesis was tested by first enriching the training dataset with a wide range of imaging conditions, then validated through internal blind tests on new samples and external multi-center clinical validations.
RESULTS: Ablation experiments revealed that removing subsets of data from the training dataset significantly reduced model precision. Removing raw sample images from the training dataset caused the largest drop in model precision, whereas removing 20x images caused the largest drop in model recall. by incorporating different imaging and sample preprocessing conditions into a rich training dataset, the model achieved an intraclass correlation coefficient (ICC) of 0.97 (95% CI: 0.94-0.99) for precision, and an ICC of 0.97 (95% CI: 0.93-0.99) for recall. Multi-center clinical validation showed no significant differences in model precision or recall across different clinics and applications.
CONCLUSIONS: The results validated the hypothesis that the richness of data in the training dataset is a key factor impacting model generalizability. These findings highlight the importance of diversity in a training dataset for model evaluation and suggest that future deep learning models in andrology and reproductive medicine should incorporate comprehensive feature sets for enhanced generalizability across clinics.

BACKGROUND: Radiomics models trained on data from one center typically show a decline of performance when applied to data from external centers, hindering their introduction into large-scale clinical practice. Current expert recommendations suggest to use only reproducible radiomics features isolated by multiscanner test-retest experiments, which might help to overcome the problem of limited generalizability to external data.
OBJECTIVE: To evaluate the influence of using only a subset of robust radiomics features, defined in a prior in vivo multi-MRI-scanner test-retest-study, on the performance and generalizability of radiomics models.
METHODS: Retrospective.
METHODS: Patients with monoclonal plasma cell disorders. Training set (117 MRIs from center 1); internal test set (42 MRIs from center 1); external test set (143 MRIs from center 2-8).
UNASSIGNED: 1.5T and 3.0T; T1-weighted turbo spin echo.
RESULTS: The task for the radiomics models was to predict plasma cell infiltration, determined by bone marrow biopsy, noninvasively from MRI. Radiomics machine learning models, including linear regressor, support vector regressor (SVR), and random forest regressor (RFR), were trained on data from center 1, using either all radiomics features, or using only reproducible radiomics features. Models were tested on an internal (center 1) and a multicentric external data set (center 2-8).
METHODS: Pearson correlation coefficient r and mean absolute error (MAE) between predicted and actual plasma cell infiltration. Fisher\'s z-transformation, Wilcoxon signed-rank test, Wilcoxon rank-sum test; significance level P < 0.05.
RESULTS: When using only reproducible features compared with all features, the performance of the SVR on the external test set significantly improved (r = 0.43 vs. r = 0.18 and MAE = 22.6 vs. MAE = 28.2). For the RFR, the performance on the external test set deteriorated when using only reproducible instead of all radiomics features (r = 0.33 vs. r = 0.44, P = 0.29 and MAE = 21.9 vs. MAE = 20.5, P = 0.10).
CONCLUSIONS: Using only reproducible radiomics features improves the external performance of some, but not all machine learning models, and did not automatically lead to an improvement of the external performance of the overall best radiomics model.
METHODS:
UNASSIGNED: Stage 2.

UNASSIGNED: Generalizability of registrative clinical trials to real-world clinical practice is influenced by comparability of patients in the two settings. We compared characteristics of cancer patients in registrative trials with real-world clinical practice in Italy.
UNASSIGNED: Data on age, sex and performance status (PS) were derived from web-based monitoring registries developed by Italian Medicines Agency (AIFA) and corresponding registrative trials reported in the European Public Assessment Reports (EPAR) of European Medicines Agency (EMA). Weighted means were calculated in registries and trials and differences were described. Multivariate analysis was performed using Principal Component Analysis and Cluster Analysis.
UNASSIGNED: From January, 2013 to April, 2023, 419,461 unique pairs of patients and therapeutic indications were recorded in 129 AIFA registries. Within 140 related trials, 87,452 patients had been enrolled. Median age and rate of elderly (≥65 years old) patients were higher in monitoring registries than in clinical trials [mean difference of median age 5.3 years, p < 0.001; mean difference of elderly rate 17.17% (95% CI 1.06, 1.48)]. Overall, rate of female patients was not different between registries and trials [mean difference -0.55% (95% CI -1.06, -0.05)]. Mean rate of patients with deteriorated PS was low both in trials (3.1%) and in registries (4.3%) with a mean difference of 1.27% (95% CI 1.06, 1.48). Two clusters were identified with multivariate analysis: one including more registries (higher median age and elderly rate, lower female rate, higher rate of deteriorated patients), the other more trials (lower median age and elderly rate, higher female rate, lower rate of deteriorated patients).
UNASSIGNED: This study supports that cancer patients enrolled in trials do only partially represent those who have been treated in Italy in clinical practice. Inclusiveness of registrative trials should be increased to ensure generalizability of results to real-world population.
UNASSIGNED: Partially supported by Italian Ministry of Health.

To date, no population-based studies have specifically explored the external validity of pivotal randomized clinical trials (RCTs) of biologics simultaneously for a broad spectrum of immuno-mediated inflammatory diseases (IMIDs). The aims of this study were, firstly, to compare the patients\' characteristics and median treatment duration of biologics approved for IMIDs between RCTs\' and real-world setting (RW); secondly, to assess the extent of biologic users treated for IMIDs in the real-world setting that would not have been eligible for inclusion into pivotal RCT for each indication of use. Using the Italian VALORE distributed database (66,639 incident biologic users), adult patients with IMIDs treated with biologics in the Italian real-world setting were substantially older (mean age ± SD: 50 ± 15 years) compared to those enrolled in pivotal RCTs (45 ± 15 years). In the real-world setting, certolizumab pegol was more commonly used by adult women with psoriasis/ankylosing spondylitis (F/M ratio: 1.8-1.9) compared to RCTs (F/M ratio: 0.5-0.6). The median treatment duration (weeks) of incident biologic users in RW was significantly higher than the duration of pivotal RCTs in almost all indications for use and most biologics (4-100 vs. 6-167). Furthermore, almost half (46.4%) of biologic users from RW settings would have been ineligible for inclusion in the respective indication-specific pivotal RCTs. The main reasons were: advanced age, recent history of cancer and presence of other concomitant IMIDs. These findings suggest that post-marketing surveillance of biologics should be prioritized for those patients.

While randomized controlled trials (RCTs) are critical for establishing the efficacy of new therapies, there are limitations regarding what comparisons can be made directly from trial data. RCTs are limited to a small number of comparator arms and often compare a new therapeutic to a standard of care which has already proven efficacious. It is sometimes of interest to estimate the efficacy of the new therapy relative to a treatment that was not evaluated in the same trial, such as a placebo or an alternative therapy that was evaluated in a different trial. Such dual-study comparisons are challenging because of potential differences between trial populations that can affect the outcome. In this article, two bridging estimators are considered that allow for comparisons of treatments evaluated in different trials, accounting for measured differences in trial populations. A \"multi-span\" estimator leverages a shared arm between two trials, while a \"single-span\" estimator does not require a shared arm. A diagnostic statistic that compares the outcome in the standardized shared arms is provided. The two estimators are compared in simulations, where both estimators demonstrate minimal empirical bias and nominal confidence interval coverage when the identification assumptions are met. The estimators are applied to data from the AIDS Clinical Trials Group 320 and 388 to compare the efficacy of two-drug vs four-drug antiretroviral therapy on CD4 cell counts among persons with advanced HIV. The single-span approach requires weaker identification assumptions and was more efficient in simulations and the application.

Decentralized clinical trial (DCT) approaches are clinical trials in which some or all trial activities take place closer to participants\' proximities instead of a traditional investigative site. Data from DCTs may be used for clinical and economic evaluations by health technology assessment (HTA) bodies to support reimbursement decision making. This study aimed to explore the opportunities and challenges for DCT approaches from an HTA perspective by interviewing representatives from European HTA bodies.
We conducted semistructured interviews with 25 European HTA representatives between September 2022 and February 2023, and transcripts were analyzed after thematic analysis.
Two main themes were identified from the data relating to (1) DCT approaches in HTA and (2) trial-level acceptance and relevance. Experience with assessing DCTs was limited and a variety of knowledge about DCTs was observed. The respondents recognized the opportunity of DCTs to reduce recall bias when participant-reported outcome data can be collected more frequently and conveniently from home. Concerns were expressed about the data quality when participants become responsible for data collection. Despite this challenge, the respondents recognized the potential of DCTs to increase the generalizability of results because data can be collected in a setting reflective of the everyday situation potentially from a more diverse participant group.
DCTs could generate relevant results for HTA decision making when data are collected in a real-world setting from a diverse participant group. Increased awareness of the opportunities and challenges could help HTA assessors in their appraisal of DCT approaches.