PDF(Pubmed)
Abstract：
Interrupted time series (ITS) designs are increasingly used for estimating the effect of shocks in natural experiments. Currently, ITS designs are often used in scenarios with many time points and simple data structures. This research investigates the performance of ITS designs when the number of time points is limited and with complex data structures. Using a Monte Carlo simulation study, we empirically derive the performance-in terms of power, bias and precision- of the ITS design. Scenarios are considered with multiple interventions, a low number of time points and different effect sizes based on a motivating example of the learning loss due to COVID school closures. The results of the simulation study show the power of the step change depends mostly on the sample size, while the power of the slope change depends on the number of time points. In the basic scenario, with both a step and a slope change and an effect size of 30% of the pre-intervention slope, the required sample size for detecting a step change is 1,100 with a minimum of twelve time points. For detecting a slope change the required sample size decreases to 500 with eight time points. To decide if there is enough power researchers should inspect their data, hypothesize about effect sizes and consider an appropriate model before applying an ITS design to their research. This paper contributes to the field of methodology in two ways. Firstly, the motivation example showcases the difficulty of employing ITS designs in cases which do not adhere to a single intervention. Secondly, models are proposed for more difficult ITS designs and their performance is tested.
摘要：
中断时间序列（ITS）设计越来越多地用于估计自然实验中冲击的影响。目前,ITS设计通常用于具有许多时间点和简单数据结构的场景。本研究调查了在时间点数量有限且数据结构复杂的情况下ITS设计的性能。使用蒙特卡罗模拟研究，我们从经验上推导了性能-在权力方面，ITS设计的偏见和精确性。考虑多种干预措施的情况，基于COVID学校停课导致学习损失的一个激励性例子，时间点数量少，影响大小不同。模拟研究的结果表明，阶跃变化的功率主要取决于样本大小，而斜率变化的功率取决于时间点的数量。在基本场景中，具有步长和坡度变化，并且效果大小为干预前坡度的30％，检测阶跃变化所需的样本量为1100，最少为十二个时间点。为了检测斜率变化,所需的样本大小在八个时间点下减少到500。为了确定是否有足够的电力研究人员应该检查他们的数据，假设效应大小，并在将ITS设计应用于他们的研究之前考虑一个合适的模型。本文从两个方面为方法论领域做出了贡献。首先,动机示例展示了在不坚持单一干预的情况下采用ITS设计的困难。其次,提出了更困难的ITS设计模型，并对其性能进行了测试。
