背景:机构审查委员会(IRB)因IRB工作人员不足或缺乏经验而延误批准研究提案而受到批评。人工智能(AI)特别是大型语言模型(LLM),在协助IRB成员进行迅速有效的审查过程方面具有巨大的潜力。
方法:在七个经过验证的案例研究中,评估了四个LLM是否可以识别潜在的伦理问题。LLM被提示与研究参与者的拟议资格标准相关的查询,脆弱性问题,在知情同意书(ICD)中披露的信息,风险-效益评估和使用安慰剂的理由。向LLM发出了另一个查询,以针对这些情况生成ICD。
结果:所有四个LLM都能够为与所有七个案例相关的查询提供答案。总的来说,对于大多数元素,响应是均匀的。LLM在识别安慰剂组的适用性方面表现欠佳,风险缓解策略和潜在风险在某些案例研究中研究参与者,只有一个提示。然而,多个提示导致更好的输出在所有这些领域。每个LLM都包含所有情况下ICD的所有基本要素。使用行话,在AI生成的ICD中,低估收益和未能陈述潜在风险是关键的观察结果。
结论:LLM可能可以增强对临床研究中潜在伦理问题的识别,它们可以用作辅助工具,以预先筛选研究提案并提高IRB的效率。
BACKGROUND: Institutional review boards (IRBs) have been criticised for delays in approvals for research proposals due to inadequate or inexperienced IRB staff. Artificial intelligence (AI), particularly large language models (LLMs), has significant potential to assist IRB members in a prompt and efficient reviewing process.
METHODS: Four LLMs were evaluated on whether they could identify potential ethical issues in seven validated
case studies. The LLMs were prompted with queries related to the proposed eligibility criteria of the study participants, vulnerability issues, information to be disclosed in the informed consent document (ICD), risk-benefit assessment and justification of the use of a placebo. Another query was issued to the LLMs to generate ICDs for these
case scenarios.
RESULTS: All four LLMs were able to provide answers to the queries related to all seven cases. In general, the responses were homogeneous with respect to most elements. LLMs performed suboptimally in identifying the suitability of the placebo arm, risk mitigation strategies and potential risks to study participants in certain
case studies with a single prompt. However, multiple prompts led to better outputs in all of these domains. Each of the LLMs included all of the fundamental elements of the ICD for all
case scenarios. Use of jargon, understatement of benefits and failure to state potential risks were the key observations in the AI-generated ICD.
CONCLUSIONS: It is likely that LLMs can enhance the identification of potential ethical issues in clinical research, and they can be used as an adjunct tool to prescreen research proposals and enhance the efficiency of an IRB.