BACKGROUND: Clinical trials often involve some form of interim monitoring to determine futility before planned trial completion. While many options for interim monitoring exist (e.g., alpha-spending, conditional power), nonparametric based interim monitoring methods are also needed to account for more complex trial designs and analyses. The upstrap is one recently proposed nonparametric method that may be applied for interim monitoring.
METHODS: Upstrapping is motivated by the case resampling bootstrap and involves repeatedly sampling with replacement from the interim data to simulate thousands of fully enrolled trials. The p-value is calculated for each upstrapped trial and the proportion of upstrapped trials for which the p-value criteria are met is compared with a pre-specified decision threshold. To evaluate the potential utility for upstrapping as a form of interim futility monitoring, we conducted a simulation study considering different sample sizes with several different proposed calibration strategies for the upstrap. We first compared trial rejection rates across a selection of threshold combinations to validate the upstrapping method. Then, we applied upstrapping methods to simulated clinical trial data, directly comparing their performance with more traditional alpha-spending and conditional power interim monitoring methods for futility.
RESULTS: The method validation demonstrated that upstrapping is much more likely to find evidence of futility in the null scenario than the alternative across a variety of simulations settings. Our three proposed approaches for calibration of the upstrap had different strengths depending on the stopping rules used. Compared to O\'Brien-Fleming group sequential methods, upstrapped approaches had type I error rates that differed by at most 1.7% and expected sample size was 2-22% lower in the null scenario, while in the alternative scenario power fluctuated between 15.7% lower and 0.2% higher and expected sample size was 0-15% lower.
CONCLUSIONS: In this proof-of-concept simulation study, we evaluated the potential for upstrapping as a resampling-based method for futility monitoring in clinical trials. The trade-offs in expected sample size, power, and type I error rate control indicate that the upstrap can be calibrated to implement futility monitoring with varying degrees of aggressiveness and that performance similarities can be identified relative to considered alpha-spending and conditional power futility monitoring methods.

Adaptive clinical trials are growing in popularity as they are more flexible, efficient and ethical than traditional fixed designs. However, notwithstanding their increased use in assessing treatments for COVID-19, their use in critical care trials remains limited. A better understanding of the relative benefits of various adaptive designs may increase their use and interpretation.
Using two large critical care trials (ADRENAL.
gov number, NCT01448109. Updated 12-12-2017; NICE-SUGAR.
gov number, NCT00220987. Updated 01-29-2009), we assessed the performance of three frequentist and two bayesian adaptive approaches. We retrospectively re-analysed the trials with one, two, four, and nine equally spaced interims. Using the original hypotheses, we conducted 10,000 simulations to derive error rates, probabilities of making an early correct and incorrect decision, expected sample size and treatment effect estimates under the null scenario (no treatment effect) and alternative scenario (a positive treatment effect). We used a logistic regression model with 90-day mortality as the outcome and the treatment arm as the covariate. The null hypothesis was tested using a two-sided significance level (α) at 0.05.
Across all approaches, increasing the number of interims led to a decreased expected sample size. Under the null scenario, group sequential approaches provided good control of the type-I error rate; however, the type I error rate inflation was an issue for the Bayesian approaches. The Bayesian Predictive Probability and O\'Brien-Fleming approaches showed the highest probability of correctly stopping the trials (around 95%). Under the alternative scenario, the Bayesian approaches showed the highest overall probability of correctly stopping the ADRENAL trial for efficacy (around 91%), whereas the Haybittle-Peto approach achieved the greatest power for the NICE-SUGAR trial. Treatment effect estimates became increasingly underestimated as the number of interims increased.
This study confirms the right adaptive design can reach the same conclusion as a fixed design with a much-reduced sample size. The efficiency gain associated with an increased number of interims is highly relevant to late-phase critical care trials with large sample sizes and short follow-up times. Systematically exploring adaptive methods at the trial design stage will aid the choice of the most appropriate method.

An industry-academic collaboration was established to evaluate the choice of statistical test and study design for A/B testing in larger-scale industry experiments. Specifically, the standard approach at the industry partner was to apply a t-test for all outcomes, both continuous and binary, and to apply naïve interim monitoring strategies that had not evaluated the potential implications on operating characteristics such as power and type I error rates. Although many papers have summarized the robustness of the t-test, its performance for the A/B testing context of large-scale proportion data, with or without interim analyses, is needed. Investigating the effect of interim analyses on the robustness of the t-test is important, because interim analyses rely on a fraction of the total sample size and one should ensure that desired properties are maintained when a t-test is implemented not just at the end of the study, but for making interim decisions. Through simulation studies, the performance of the t-test, Chi-squared test, and Chi-squared test with Yate\'s correction when applied to binary outcomes data is evaluated. Further, interim monitoring through a naïve approach with no correction for multiple testing versus the O\'Brien-Fleming boundary are considered in designs that allow early termination for futility, difference, or both. Results indicate that the t-test achieves similar power and type I error rates for binary outcomes data with the large sample sizes used in industrial A/B tests with and without interim monitoring, and naïve interim monitoring without corrections leads to poorly performing studies.

A sequential multiple assignment randomized trial (SMART) facilitates the comparison of multiple adaptive treatment strategies (ATSs) simultaneously. Previous studies have established a framework to test the homogeneity of multiple ATSs by a global Wald test through inverse probability weighting. SMARTs are generally lengthier than classical clinical trials due to the sequential nature of treatment randomization in multiple stages. Thus, it would be beneficial to add interim analyses allowing for an early stop if overwhelming efficacy is observed. We introduce group sequential methods to SMARTs to facilitate interim monitoring based on the multivariate chi-square distribution. Simulation studies demonstrate that the proposed interim monitoring in SMART (IM-SMART) maintains the desired type I error and power with reduced expected sample size compared to the classical SMART. Finally, we illustrate our method by reanalyzing a SMART assessing the effects of cognitive behavioral and physical therapies in patients with knee osteoarthritis and comorbid subsyndromal depressive symptoms.

Cluster-randomized trials (CRTs) of infectious disease preventions often yield correlated, interval-censored data: dependencies may exist between observations from the same cluster, and event occurrence may be assessed only at intermittent study visits. This data structure must be accounted for when conducting interim monitoring and futility assessment for CRTs. In this article, we propose a flexible framework for conditional power estimation when outcomes are correlated and interval-censored. Under the assumption that the survival times follow a shared frailty model, we first characterize the correspondence between the marginal and cluster-conditional survival functions, and then use this relationship to semiparametrically estimate the cluster-specific survival distributions from the available interim data. We incorporate assumptions about changes to the event process over the remainder of the trial-as well as estimates of the dependency among observations in the same cluster-to extend these survival curves through the end of the study. Based on these projected survival functions, we generate correlated interval-censored observations, and then calculate the conditional power as the proportion of times (across multiple full-data generation steps) that the null hypothesis of no treatment effect is rejected. We evaluate the performance of the proposed method through extensive simulation studies, and illustrate its use on a large cluster-randomized HIV prevention trial.

When a clinical trial has a composite endpoint and a comparison of treatment strategies with multiple intervention components, interim data reviews by a data safety and monitoring board (DSMB) can be challenging as the data evolve on multiple fronts. We illustrate with a study in the treatment of Kaposi sarcoma (KS), an HIV-associated cancer with a multi-faceted disease presentation. The study, ACTG-A5264/AMC-067, was a 1:1 randomized trial to compare two strategies: immediate initiation of etoposide with antiretroviral therapy (ART), or ART with delayed etoposide upon disease progression. The outcome was a composite endpoint that included the following events, ordered from worst to best in the following three categories: (1) KS progression at 48 weeks, death, initiation of alternate KS treatment, loss to study follow-up; (2) stable KS; and (3) partial or complete KS response at 48 weeks. We present the interim results on the composite endpoint and the individual components, where components favored different study arms at an interim review. To facilitate interim data monitoring for complex trials, we recommend clear communications between the study team and the DSMB prior to the initiation of the trial on the need for a composite endpoint, the intentions behind the defined strategies, and relative importance of individual components of the composite endpoint. We also recommend flexibility in the timing of data reviews by the DSMB to interpret emerging data in multiple dimensions. Clinicaltrials.govNCT01352117.

In an ongoing clinical trial, there will always be a risk for unanticipated critical safety problems, such as excessive occurrence of serious adverse events. When such a problem arises, the trial administrators must conduct an immediate evaluation to determine whether the trial should be terminated to protect patients. This decision is complicated but may be aided by statistical stopping rules. Sequential stopping rules are appropriate for immediate decisions, but frequentist approaches may not be useful because the unknown truncated end of the trial makes it impossible to define type I errors. Thus, a Bayesian stopping rule is proposed that is based on the posterior distribution with an informative prior distribution, and a guideline to construct this stopping rule is presented. Some operating characteristics are evaluated and compared with those of the modified sequential probability ratio test (SPRT), the maximized SPRT, and Pocock\'s test. The proposed method has flexibility for construction and could provide a more desirable performance than the other compared methods.

Recent guidelines have de-emphasized the role of routine surveillance computed tomography (CT) scans for diffuse large B-cell lymphoma (DLBCL) patients who achieve a complete response to front-line therapy. This shift in practice recommendations was prompted by retrospective studies that failed to demonstrate clear clinical utility for surveillance CT in unselected DLBCL patients. Controversy remains, however, over the role of routine surveillance CT in the highest risk patients for treatment failure who would remain candidates for aggressive salvage therapies. Novel high-throughput sequencing methods can non-invasively monitor tumor-specific DNA in the blood and offers clear advantages designed to overcome fundamental limitations of CT scans. This review will discuss the current controversies surrounding monitoring clinical outcomes in aggressive B-cell lymphomas, with a specific emphasis on DLBCL. Fundamental limitations of imaging scans will be addressed and the potential of monitoring circulating tumor DNA as an adjunct or replacement for CT scans will be discussed.

In clinical research and development, interim monitoring is critical for better decision-making and minimizing the risk of exposing patients to possible ineffective therapies. For interim futility or efficacy monitoring, predictive probability methods are widely adopted in practice. Those methods have been well studied for univariate variables. However, for longitudinal studies, predictive probability methods using univariate information from only completers may not be most efficient, and data from on-going subjects can be utilized to improve efficiency. On the other hand, leveraging information from on-going subjects could allow an interim analysis to be potentially conducted once a sufficient number of subjects reach an earlier time point. For longitudinal outcomes, we derive closed-form formulas for predictive probabilities, including Bayesian predictive probability, predictive power, and conditional power and also give closed-form solutions for predictive probability of success in a future trial and the predictive probability of success of the best dose. When predictive probabilities are used for interim monitoring, we study their distributions and discuss their analytical cutoff values or stopping boundaries that have desired operating characteristics. We show that predictive probabilities utilizing all longitudinal information are more efficient for interim monitoring than that using information from completers only. To illustrate their practical application for longitudinal data, we analyze 2 real data examples from clinical trials.

Interim monitoring is a key component of randomized clinical trial design from both ethical and efficiency perspectives. In studies with time-to-event endpoints, timing of interim analyses is typically based on observing a pre-specified proportion of the total number of events required for the final analysis. While most randomized clinical trial designs pool events over the experimental and control arms in determining the analysis times, some designs use only the control-arm events for scheduling interim looks.
To evaluate the performance of the pooled and control-arm-based interim monitoring approaches and to propose a new procedure, the earliest information time procedure, that combines the benefits of the two approaches.
The analytical and logistical considerations for the procedures are presented. The methodology is illustrated on data from three published randomized clinical trials. The procedures are compared in a simulation study.
The control-arm approach results in a slight inflation of the study type I error in one-sided randomized clinical trial designs. When the new treatment is no better than the control treatment, the pooled-arm approach results in, on average, earlier stopping times than the control-arm approach. When the new treatment works exceptionally well, the average stopping times under the control-arm approach are earlier than those under the pooled approach. The proposed earliest information time procedure is shown to result in stopping times corresponding to the best (earliest) of the two approaches over the entire range of alternatives.
The earliest information time procedure may result in a slight inflation of the type I error (especially in small trials); when exact control of the type I error is required, it is necessary to use a simulation-based method to correct the inflation.
In time-to-event settings, the earliest information time procedure is an attractive alternative to the pooled and control-arm approaches. Improving the timing of interim analyses helps to minimize patient exposure to inferior treatments and to accelerate dissemination of the study results.