PDF(Pubmed)
Abstract：
Psychometrics is historically grounded in the study of individual differences. Consequently, common metrics such as quantitative validity and reliability require between-person variance in a psychological variable to be meaningful. Experimental psychology, in contrast, deals with variance between treatments, and experiments often strive to minimise within-group person variance. In this article, I ask whether and how psychometric evaluation can be performed in experimental psychology. A commonly used strategy is to harness between-person variance in the treatment effect. Using simulated data, I show that this approach can be misleading when between-person variance is low, and in the face of methods variance. I argue that this situation is common in experimental psychology, because low between-person variance is desirable, and because methods variance is no more problematic in experimental settings than any other source of between-person variance. By relating validity and reliability with the corresponding concepts in measurement science outside psychology, I show how experiment-based calibration can serve to compare the psychometric quality of different measurement methods in experimental psychology.
摘要：
心理测量学在历史上是建立在个体差异的研究。因此,定量效度和可靠性等常见指标要求心理变量中的人与人之间的差异有意义。实验心理学,相比之下,处理处理之间的差异，和实验往往努力使组内的人的差异最小化。在这篇文章中,我问是否以及如何在实验心理学中进行心理测量评估。一种常用的策略是在治疗效果中利用人与人之间的差异。使用模拟数据，我表明，当人与人之间的方差较低时，这种方法可能会产生误导，面对方法的差异。我认为这种情况在实验心理学中很常见，因为较低的人与人之间的差异是可取的，并且因为方法差异在实验设置中并不比人与人之间差异的任何其他来源更有问题。通过将效度和信度与心理学之外的测量科学中的相应概念相关联，我展示了基于实验的校准如何用于比较实验心理学中不同测量方法的心理测量质量。
