背景:对话聊天机器人是一种新兴的数字戒烟干预措施。没有研究报告停止聊天机器人的整个开发过程。
目的:描述一种名为“QuitBot”的新颖而全面的戒烟对话聊天机器人的以用户为中心的设计开发过程。\"
方法:开发QuitBot的四年形成性研究遵循了一个11步的过程:(1)指定一个概念模型,(2)对现有干预措施进行内容分析(63小时的干预记录),(3)评估用户需求,(4)培养聊天的个性(“个性”),(5)原型制作内容和角色,(6)开发全部功能,(7)编程的QuitBot,(8)进行日记学习,(9)进行试点随机试验,(10)审查试验结果,(11)添加自由形式问答(QnA)函数,基于用户反馈的试点试验结果。添加QnA函数本身的过程涉及一个三步过程:(a)生成QnA对,(B)对QnA对的大型语言模型(LLM)进行微调,和(C)评估LLM模型输出。
结果:一项戒烟计划,为期42天,进行2至3分钟的对话,涵盖从动机到戒烟的主题,设置退出日期,选择FDA批准的戒烟药物,应对触发器,并从失误/复发中恢复。在一项试点随机试验中,三个月的结果数据保留率为96%,与美国国家癌症研究所的SmokereeTXT(SFT)短信程序相比,QuitBot表现出较高的用户参与度和有希望的戒烟率-特别是在那些观看了所有42天节目内容的人中:30天完整案例,在三个月的随访中,QuitBot的点患病率禁欲(PPA)率为63%(39/62)。SFT为38%(45/117)(OR=2.58;95%CI:1.34,4.99;P=0.005)。然而,FacebookMessenger(FM)间歇性地阻止参与者访问QuitBot,因此我们从FM过渡到独立的智能手机应用程序作为通信渠道。参与者对QuitBot无法回答他们的开放式问题感到沮丧,这使我们开发了一个核心对话功能,使用户能够提出有关戒烟的开放式问题,并让QuitBot以准确和专业的答案做出回应。要支持此功能,我们开发了一个由11,000个QnA对组成的图书馆,内容涉及与戒烟相关的主题。模型测试结果表明,微软基于Azure的QnA制造商有效地处理了与我们的11,000个QnA对库相匹配的问题。一个微调,上下文化的GPT3.5回答了我们的QnA对库中不存在的问题。
结论:开发过程产生了第一个基于LLM的戒烟计划,作为对话聊天机器人交付。迭代测试带来了显著的增强,包括对交付渠道的改进。一个关键的补充是包含了LLM支持的核心会话功能,允许用户提出开放式问题。
背景:ClinicalTrials.gov标识符,NCT03585231。
BACKGROUND: Conversational chatbots are an emerging digital intervention for smoking
cessation. No studies have reported on the entire development process of a
cessation chatbot.
OBJECTIVE: We aim to report results of the user-centered design development process and randomized controlled trial for a novel and comprehensive quit smoking conversational chatbot called QuitBot.
METHODS: The 4 years of formative research for developing QuitBot followed an 11-step process: (1) specifying a conceptual model; (2) conducting content analysis of existing interventions (63 hours of intervention transcripts); (3) assessing user needs; (4) developing the chat\'s persona (\"personality\"); (5) prototyping content and persona; (6) developing full functionality; (7) programming the QuitBot; (8) conducting a diary study; (9) conducting a pilot randomized controlled trial (RCT); (10) reviewing results of the RCT; and (11) adding a free-form question and answer (QnA) function, based on user feedback from pilot RCT results. The process of adding a QnA function itself involved a three-step process: (1) generating QnA pairs, (2) fine-tuning large language models (LLMs) on QnA pairs, and (3) evaluating the LLM outputs.
RESULTS: We developed a quit smoking program spanning 42 days of 2- to 3-minute conversations covering topics ranging from motivations to quit, setting a quit date, choosing Food and Drug Administration-approved
cessation medications, coping with triggers, and recovering from lapses and relapses. In a pilot RCT with 96% three-month outcome data retention, QuitBot demonstrated high user engagement and promising
cessation rates compared to the National Cancer Institute\'s SmokefreeTXT text messaging program, particularly among those who viewed all 42 days of program content: 30-day, complete-case, point prevalence abstinence rates at 3-month follow-up were 63% (39/62) for QuitBot versus 38.5% (45/117) for SmokefreeTXT (odds ratio 2.58, 95% CI 1.34-4.99; P=.005). However, Facebook Messenger intermittently blocked participants\' access to QuitBot, so we transitioned from Facebook Messenger to a stand-alone smartphone app as the communication channel. Participants\' frustration with QuitBot\'s inability to answer their open-ended questions led to us develop a core conversational feature, enabling users to ask open-ended questions about quitting cigarette smoking and for the QuitBot to respond with accurate and professional answers. To support this functionality, we developed a library of 11,000 QnA pairs on topics associated with quitting cigarette smoking. Model testing results showed that Microsoft\'s Azure-based QnA maker effectively handled questions that matched our library of 11,000 QnA pairs. A fine-tuned, contextualized GPT-3.5 (OpenAI) responds to questions that are not within our library of QnA pairs.
CONCLUSIONS: The development process yielded the first LLM-based quit smoking program delivered as a conversational chatbot. Iterative testing led to significant enhancements, including improvements to the delivery channel. A pivotal addition was the inclusion of a core LLM-supported conversational feature allowing users to ask open-ended questions.
BACKGROUND: ClinicalTrials.gov NCT03585231; https://clinicaltrials.gov/study/NCT03585231.