Semiparametric probabilistic index models allow for the comparison of two groups of observations, whilst adjusting for covariates, thereby fitting nicely within the framework of generalized pairwise comparisons (GPC). As with most regression approaches in this setting, the limited amount of data results in invalid inference as the asymptotic normality assumption is not met. In addition, separation issues might arise when considering small samples. In this article, we show that the parameters of the probabilistic index model can be estimated using generalized estimating equations, for which adjustments exist that lead to estimators of the sandwich variance-covariance matrix with improved finite sample properties and that can deal with bias due to separation. In this way, appropriate inference can be performed as is shown through extensive simulation studies. The known relationships between the probabilistic index and other GPC statistics allow to also provide valid inference for example, the net treatment benefit or the success odds.

OBJECTIVE: The restricted Net Treatment Benefit (rNTB) is a clinically meaningful and tractable estimand of the overall treatment effect assessed in randomized trials when at least one survival endpoint with time restriction is used. Its interpretation does not rely on parametric assumptions such as proportional hazards, can be estimated without bias even in the presence of independent right-censoring, and can include a prespecified threshold of minimal clinically relevant difference. To demonstrate that the rNTB, corresponding to the NTB during a predefined time interval, is a meaningful and adaptable measure of treatment effect in clinical trials.
METHODS: In this simulation study, we tested the impact on the rNTB value, estimation, and power of several factors including the presence of a delayed treatment effect, minimal clinically relevant difference threshold value, restriction time value, and the inclusion of both efficacy and toxicity in the rNTB definition. The impact of right censoring on rNTB was assessed in terms of bias. rNTB-derived statistical tests and log rank (LR) tests were compared in terms of power.
RESULTS: RNTB estimates are unbiased even in case of right-censoring. rNTB may be used to estimate the benefit/risk ratio of a new treatment, for example, taking into account both survival and toxicity and include several prioritized outcomes. The estimated rNTB is much easier to interpret in this context compared to NTB in the presence of censoring since the latter is intrinsically dependent on the follow-up duration. Including toxicity increases the test power when the experimental treatment is less toxic. rNTB-derived test power increases when the experimental treatment is associated with longer survival and lower toxicity and might increase in the presence of a cure rate or a delayed treatment effect. Case applications on the PRODIGE, Checkmate-066, and Checkmate-067 trials are provided.
CONCLUSIONS: RNTB is an interesting alternative to describe and test the treatment\'s effect in a clear and understandable way in case of restriction, particularly in scenarios with nonproportional hazards or when trying to balance benefit and safety. It can be tuned to take into consideration short- or long-term survival differences and one or more prioritized outcomes.

Showing \"similar efficacy\" of a less intensive treatment typically requires a non-inferiority trial. Yet such trials may be challenging to design and conduct. In acute promyelocytic leukemia, great progress has been achieved with the introduction of targeted therapies, but toxicity remains a major clinical issue. There is a pressing need to show the favorable benefit/risk of less intensive treatment regimens.
We designed a clinical trial that uses generalized pairwise comparisons of five prioritized outcomes (alive and event-free at 2 years, grade 3/4 documented infections, differentiation syndrome, hepatotoxicity, and neuropathy) to confirm a favorable benefit/risk of a less intensive treatment regimen. We conducted simulations based on historical data and assumptions about the differences expected between the standard of care and the less intensive treatment regimen to calculate the sample size required to have high power to show a positive Net Treatment Benefit in favor of the less intensive treatment regimen.
Across 10,000 simulations, average sample sizes of 260 to 300 patients are required for a trial using generalized pairwise comparisons to detect typical Net Treatment Benefits of 0.19 (interquartile range 0.14-0.23 for a sample size of 280). The Net Treatment Benefit is interpreted as a difference between the probability of doing better on the less intensive treatment regimen than on the standard of care, minus the probability of the opposite situation. A Net Treatment Benefit of 0.19 translates to a number needed to treat of about 5.3 patients (1/0.19 ≃ 5.3).
Generalized pairwise comparisons allow for simultaneous assessment of efficacy and safety, with priority given to the former. The sample size required would be of the order of 300 patients, as compared with more than 700 patients for a non-inferiority trial using a margin of 4% against the less intensive treatment regimen for the absolute difference in event-free survival at 2 years, as considered here.

BACKGROUND: Generalized pairwise comparisons (GPC) can be used to assess the net benefit of new treatments for rare diseases. We show the potential of GPC through simulations based on data from a natural history study in mucopolysaccharidosis type IIIA (MPS IIIA).
METHODS: Using data from a historical series of untreated children with MPS IIIA aged 2 to 9 years at the time of enrolment and followed for 2 years, we performed simulations to assess the operating characteristics of GPC to detect potential (simulated) treatment effects on a multi-domain symptom assessment. Two approaches were used for GPC: one in which the various domains were prioritized, the other with all domains weighted equally. The net benefit was used as a measure of treatment effect. We used increasing thresholds of clinical relevance to reflect the magnitude of the desired treatment effects, relative to the standard deviation of the measurements in each domain.
RESULTS: GPC were shown to have adequate statistical power (80% or more), even with small sample sizes, to detect treatment effects considered to be clinically worthwhile on a symptom assessment covering five domains (expressive language, daily living skills, and gross-motor, sleep and pain). The prioritized approach generally led to higher power as compared with the non-prioritized approach.
CONCLUSIONS: GPC of prioritized outcomes is a statistically powerful as well as a patient-centric approach for the analysis of multi-domain scores in MPS IIIA and could be applied to other heterogeneous rare diseases.

A time-to-first-event composite endpoint analysis has well-known shortcomings in evaluating a treatment effect in cardiovascular clinical trials. It does not fully describe the clinical benefit of therapy because the severity of the events, events repeated over time, and clinically relevant nonsurvival outcomes cannot be considered. The generalized pairwise comparisons (GPC) method adds flexibility in defining the primary endpoint by including any number and type of outcomes that best capture the clinical benefit of a therapy as compared with standard of care. Clinically important outcomes, including bleeding severity, number of interventions, and quality of life, can easily be integrated in a single analysis. The treatment effect in GPC can be expressed by the net treatment benefit, the success odds, or the win ratio. This review provides guidance on the use of GPC and the choice of treatment effect measures for the analysis and reporting of cardiovascular trials.

When assessing the efficacy of a treatment in any clinical trial, it is recommended by the International Conference on Harmonisation to select a single meaningful endpoint. However, a single endpoint is often not sufficient to reflect the full clinical benefit of a treatment in multifaceted diseases, which is often the case in rare diseases. Therefore, the use of a combination of several clinically meaningful outcomes is preferred. Many methodologies that allow for combining outcomes in a so-called composite endpoint are however limited in a number of ways, not in the least in the number and type of outcomes that can be combined and in the poor small-sample properties. Moreover, patient reported outcomes, such as quality of life, often cannot be integrated in a composite analysis, in spite of their intrinsic value.
Recently, a class of non-parametric generalized pairwise comparisons tests have been proposed, which members do allow for any number and type of outcomes, including patient reported outcomes. The class enjoys good small-sample properties. Moreover, this very flexible class of methods allows for prioritizing the outcomes by clinical severity, allows for matched designs and for adding a threshold of clinical relevance. Our aim is to introduce the generalized pairwise comparison ideas and concepts for rare disease clinical trial analysis, and demonstrate their benefit in a post-hoc analysis of a small-sample trial in epidermolysis bullosa. More precisely, we will include a patient relevant outcome (Quality of life), in a composite endpoint. This publication is part of the European Joint Programme on Rare Diseases (EJP RD) series on innovative methodologies for rare diseases clinical trials, which is based on the webinars presented within the educational activity of EJP RD. This publication covers the webinar topic on composite endpoints in rare diseases and includes participants\' response to a questionnaire on this topic.
Generalized pairwise comparisons is a promising statistical methodology for evaluating any type of composite endpoints in rare disease trials and may allow a better evaluation of therapy efficacy including patients reported outcomes in addition to outcomes related to the diseases signs and symptoms.

The net benefit is an effect measure for any type of endpoint, including the time-to-event outcome, and can provide intuitive and clinically meaningful interpretation. It is defined as the probability of a randomly selected subject from the experimental arm surviving by at least a clinically relevant time longer than a randomly selected subject from the control arm. In oncology clinical trials, an intercurrent event such as treatment switching is common, which potentially causes informative censoring; nevertheless, conventional methods for the net benefit are not able to deal with it. In this study, we proposed a new estimator using the inverse probability of censoring weighting (IPCW) method and illustrated an oncology clinical trial with treatment switching (the SHIVA study) to apply the proposed method under the estimand framework.
The net benefit can be estimated using the survival functions of each treatment group. The proposed estimator was based on the survival functions estimated by the inverse probability of the censoring weighting method that can handle covariate-dependent censoring. The simulation study was undertaken to evaluate the operating characteristics of the proposed estimator under several scenarios; we varied the shapes of the survival curves, treatment effect, covariates effect on censoring, proportion of the censoring, threshold of the net benefit, and sample size. We also applied conventional methods (the scoring rules by Péron or Gehan) and the proposed method to the SHIVA study.
Our simulation study showed that the proposed estimator provided less biased results under the covariate-dependent censoring than existing estimators. When applying the proposed method to the SHIVA study, we were able to estimate the net benefit by incorporating the information of the covariates with different estimand strategies to address the intercurrent event of the treatment switching. However, the estimates of the proposed method and those of the aforementioned conventional methods were similar under the hypothetical strategy.
We proposed a new estimator of the net benefit that can include covariates to account for the possibly informative censoring. We also provided an illustrative analysis of the proposed method for the oncology clinical trial with treatment switching using the estimand framework. Our proposed new estimator is suitable for handling the intercurrent events that can potentially cause covariate-dependent censoring.

Benefit-risk balance is gaining interest in clinical trials. For the comprehensive assessment of benefits and risks, generalized pairwise comparisons are increasingly used to estimate the net benefit based on multiple prioritized outcomes. Although previous research has demonstrated that the correlations between the outcomes impact the net benefit and its estimate, the direction and magnitude of this impact remain unclear. In this study, we investigated the impact of correlations between two binary or Gaussian variables on the true net benefit values via theoretical and numerical analyses. We also explored the impact of correlations between survival and categorical variables on the net benefit estimates based on four existing methods (Gehan, Péron, Gehan with correction, and Péron with correction) in the presence of right censoring via simulation and application to actual oncology clinical trial data. Our theoretical and numerical analyses revealed that the true net benefit values were impacted by the correlations in various directions depending on the outcome distributions. With binary endpoints, this direction was governed by a simple rule with a threshold of 50% for a favorable outcome. Our simulation showed that the net benefit estimates based on Gehan\'s or Péron\'s scoring rule could be substantially biased in the presence of right censoring, and that the direction and magnitude of this bias were associated with the outcome correlations. The recently proposed correction method greatly reduced this bias, even in the presence of strong outcome correlations. The impact of correlations should be carefully considered when interpreting the net benefit and its estimate.

The win odds and the net benefit are related directly to each other and indirectly, through ties, to the win ratio. These three win statistics test the same null hypothesis of equal win probabilities between two groups. They provide similar p-values and powers, because the Z-values of their statistical tests are approximately equal. Thus, they can complement one another to show the strength of a treatment effect. In this article, we show that the estimated variances of the win statistics are also directly related regardless of ties or indirectly related through ties. Since its introduction in 2018, the stratified win ratio has been applied in designs and analyses of clinical trials, including Phase III and Phase IV studies. This article generalizes the stratified method to the win odds and the net benefit. As a result, the relations of the three win statistics and the approximate equivalence of their statistical tests also hold for the stratified win statistics.

BACKGROUND: The Net Benefit (Δ) is a measure of the benefit-risk balance in clinical trials, based on generalized pairwise comparisons (GPC) using several prioritized outcomes and thresholds of clinical relevance. We extended Δ to N-of-1 trials, with a focus on patient-level and population-level Δ.
METHODS: We developed a Δ estimator at the individual level as an extension of the stratum-specific Δ, and at the population-level as an extension of the stratified Δ. We performed a simulation study mimicking PROFIL, a series of 38 N-of-1 trials testing sildenafil in Raynaud\'s phenomenon, to assess the power for such an analysis with realistic data. We then reanalyzed PROFIL using GPC. This reanalysis was finally interpreted in the context of the main analysis of PROFIL which used Bayesian individual probabilities of efficacy.
RESULTS: Simulations under the null showed good size of the test for both individual and population levels. The test lacked power when being simulated from the true PROFIL data, even when increasing the number of repetitions up to 140 days per patient. PROFIL individual-level estimated Δ were well correlated with the probabilities of efficacy from the Bayesian analysis while showing similarly wide confidence intervals. Population-level estimated Δ was not significantly different from zero, consistently with the previous Bayesian analysis.
CONCLUSIONS: GPC can be used to estimate individual Δ which can then be aggregated in a meta-analytic way in N-of-1 trials. GPC ability to easily incorporate patient preferences allow for more personalized treatment evaluation, while needing much less computing time than Bayesian modeling.