没有考虑与信度和效度相关的心理测量学问题,差异赤字,统计能力可能会破坏研究的结论。在使用事件相关脑电位(ERPs)的研究中,众多环境因素(人口抽样,任务,数据记录,分析管道,等。)会影响ERP评分的可靠性。本综述考虑了影响ERP评分可靠性的环境因素以及可靠性对统计分析的下游影响。鉴于ERP的上下文依赖性,建议在逐个研究的基础上正式评估ERP评分的可靠性.ERP研究的推荐指南包括1)报告可接受的可靠性阈值和观察分数的可靠性估计,2)指定用于估计可靠性的方法,和3)证明如何选择最小试验计数。建议内部一致性的可靠性阈值至少为0.70,和0.80的阈值是优选的。该评论还主张使用泛化理论来估计分数可靠性(泛化理论模拟可靠性),作为对经典测试理论可靠性估计的改进,这表明后者不太适合ERP研究。为了便于可靠性估计的计算和报告,一个开源的Matlab程序,ERP可靠性分析工具箱,是presented。
Failing to consider psychometric issues related to reliability and validity, differential deficits, and statistical power potentially undermines the conclusions of a study. In research using event-related brain potentials (ERPs), numerous contextual factors (population sampled, task, data recording, analysis pipeline, etc.) can impact the reliability of ERP scores. The present review considers the contextual factors that influence ERP score reliability and the downstream effects that reliability has on statistical analyses. Given the context-dependent nature of ERPs, it is recommended that ERP score reliability be formally assessed on a study-by-study basis. Recommended
guidelines for ERP studies include 1) reporting the threshold of acceptable reliability and reliability estimates for observed scores, 2) specifying the approach used to estimate reliability, and 3) justifying how trial-count minima were chosen. A reliability threshold for internal consistency of at least 0.70 is recommended, and a threshold of 0.80 is preferred. The review also advocates the use of generalizability theory for estimating score dependability (the generalizability theory analog to reliability) as an improvement on classical test theory reliability estimates, suggesting that the latter is less well suited to ERP research. To facilitate the calculation and reporting of dependability estimates, an open-source Matlab program, the ERP Reliability Analysis Toolbox, is presented.