PDF(Pubmed)
Abstract：
Digital interventions often suffer from low usage, which may reflect insufficient attention to user experience. Moreover, the existing evaluation methods have limited applicability in the remote study of user experience of complex interventions that have expansive content and that are used over an extensive period of time. To alleviate these challenges, we describe here a novel qualitative Ecological Momentary Assessment (EMA) method: the CORTO method (Contextual, One-item, Repeated, Timely, Open-ended). We used it to gather digital intervention user experience data from Finnish adults (n = 184) who lived with interview-confirmed major depressive disorder (MDD) and took part in a randomized controlled trial (RCT) that studied the efficacy of a novel 12-week game-based digital intervention for depression. A second dataset on user experience was gathered with retrospective interviews (n = 22). We inductively coded the CORTO method and retrospective interview data, which led to four user experience categories: (1) contextual use, (2) interaction-elicited emotional experience, (3) usability, and (4) technical issues. Then, we used the created user experience categories and Template Analysis to analyze both datasets together, and reported the results qualitatively. Finally, we compared the two datasets with each other. We found that the data generated with the CORTO method offered more insights into usability and technical categories than the interview data that particularly illustrated the contextual use. The emotional valence of the interview data was more positive compared with the CORTO data. Both the CORTO and interview data detected 55 % of the micro-level categories; 20 % of micro-level categories were only detected by the CORTO data and 25 % only by the interview data. We found that the during-intervention user experience measurement with the CORTO method can provide intervention-specific insights, and thereby further the iterative user-centered intervention development. Overall, these findings highlight the impact of evaluation methods on the categories and qualities of insights acquired in intervention research.
摘要：
数字干预通常使用率低，这可能反映出对用户体验的关注不足。此外，现有的评估方法在复杂干预措施的用户体验的远程研究中适用性有限，这些干预措施具有广泛的内容，并且在很长一段时间内使用。为了缓解这些挑战，我们在这里描述了一种新的定性生态瞬时评估(EMA)方法：CRTO方法(上下文，一项,重复,及时,开放式)。我们使用它来收集来自芬兰成年人（n=184）的数字干预用户体验数据，这些成年人患有经访谈确认的重度抑郁症（MDD），并参加了一项随机对照试验（RCT），该试验研究了一种新颖的12周基于游戏的数字干预抑郁症的疗效。通过回顾性访谈收集了关于用户体验的第二个数据集(n=22)。我们对CRTO方法和回顾性访谈数据进行了归纳编码，这导致了四个用户体验类别：(1)上下文使用，(2)互动引发的情感体验，（3）可用性，（四）技术问题。然后，我们使用创建的用户体验类别和模板分析一起分析这两个数据集，并定性报告结果。最后,我们将这两个数据集进行了比较。我们发现，与特别说明上下文使用的访谈数据相比，使用CRTO方法生成的数据提供了对可用性和技术类别的更多见解。与CRTO数据相比，访谈数据的情绪效价更为积极。CRTO和访谈数据均检测到55％的微观类别；20％的微观类别仅由CRTO数据检测到，25％仅由访谈数据检测到。我们发现，使用CORTO方法进行干预期间用户体验测量可以提供特定于干预的见解，从而进一步进行以用户为中心的迭代干预开发。总的来说,这些发现强调了评估方法对干预研究中获得的见解的类别和质量的影响.
