PDF(Pubmed)
Abstract：
BACKGROUND: Neonatal pain assessment (NPA) represents a huge global problem of essential importance, as a timely and accurate assessment of neonatal pain is indispensable for implementing pain management.
OBJECTIVE: To investigate the consistency of pain scores derived through video-based NPA (VB-NPA) and on-site NPA (OS-NPA), providing the scientific foundation and feasibility of adopting VB-NPA results in a real-world scenario as the gold standard for neonatal pain in clinical studies and labels for artificial intelligence (AI)-based NPA (AI-NPA) applications.
METHODS: A total of 598 neonates were recruited from a pediatric hospital in China.
METHODS: This observational study recorded 598 neonates who underwent one of 10 painful procedures, including arterial blood sampling, heel blood sampling, fingertip blood sampling, intravenous injection, subcutaneous injection, peripheral intravenous cannulation, nasopharyngeal suctioning, retention enema, adhesive removal, and wound dressing. Two experienced nurses performed OS-NPA and VB-NPA at a 10-day interval through double-blind scoring using the Neonatal Infant Pain Scale to evaluate the pain level of the neonates. Intra-rater and inter-rater reliability were calculated and analyzed, and a paired samples t-test was used to explore the bias and consistency of the assessors\' pain scores derived through OS-NPA and VB-NPA. The impact of different label sources was evaluated using three state-of-the-art AI methods trained with labels given by OS-NPA and VB-NPA, respectively.
RESULTS: The intra-rater reliability of the same assessor was 0.976-0.983 across different times, as measured by the intraclass correlation coefficient. The inter-rater reliability was 0.983 for single measures and 0.992 for average measures. No significant differences were observed between the OS-NPA scores and the assessment of an independent VB-NPA assessor. The different label sources only caused a limited accuracy loss of 0.022-0.044 for the three AI methods.
CONCLUSIONS: VB-NPA in a real-world scenario is an effective way to assess neonatal pain due to its high intra-rater and inter-rater reliability compared to OS-NPA and could be used for the labeling of large-scale NPA video databases for clinical studies and AI training.
OBJECTIVE: To investigate the consistency of pain scores derived through video-based NPA (VB-NPA) and on-site NPA (OS-NPA), providing the scientific foundation and feasibility of adopting VB-NPA results in a real-world scenario as the gold standard for neonatal pain in clinical studies and labels for artificial intelligence (AI)-based NPA (AI-NPA) applications.
METHODS: A total of 598 neonates were recruited from a pediatric hospital in China.
METHODS: This observational study recorded 598 neonates who underwent one of 10 painful procedures, including arterial blood sampling, heel blood sampling, fingertip blood sampling, intravenous injection, subcutaneous injection, peripheral intravenous cannulation, nasopharyngeal suctioning, retention enema, adhesive removal, and wound dressing. Two experienced nurses performed OS-NPA and VB-NPA at a 10-day interval through double-blind scoring using the Neonatal Infant Pain Scale to evaluate the pain level of the neonates. Intra-rater and inter-rater reliability were calculated and analyzed, and a paired samples t-test was used to explore the bias and consistency of the assessors\' pain scores derived through OS-NPA and VB-NPA. The impact of different label sources was evaluated using three state-of-the-art AI methods trained with labels given by OS-NPA and VB-NPA, respectively.
RESULTS: The intra-rater reliability of the same assessor was 0.976-0.983 across different times, as measured by the intraclass correlation coefficient. The inter-rater reliability was 0.983 for single measures and 0.992 for average measures. No significant differences were observed between the OS-NPA scores and the assessment of an independent VB-NPA assessor. The different label sources only caused a limited accuracy loss of 0.022-0.044 for the three AI methods.
CONCLUSIONS: VB-NPA in a real-world scenario is an effective way to assess neonatal pain due to its high intra-rater and inter-rater reliability compared to OS-NPA and could be used for the labeling of large-scale NPA video databases for clinical studies and AI training.
摘要：
背景：新生儿疼痛评估（NPA）代表了一个至关重要的巨大全球问题，及时准确的评估新生儿疼痛是实施疼痛管理不可缺少的。
目的：研究通过基于视频的NPA（VB-NPA）和现场NPA（OS-NPA）得出的疼痛评分的一致性，提供在现实世界中采用VB-NPA结果的科学依据和可行性，作为临床研究中新生儿疼痛的黄金标准和基于人工智能（AI）的NPA（AI-NPA）应用的标签。
方法：从中国某儿科医院招募598例新生儿。
方法：这项观察性研究记录了598例新生儿，这些新生儿经历了10例疼痛手术之一，包括动脉采血,脚跟采血,指尖采血,静脉注射,皮下注射,外周静脉插管,鼻咽吸引,保留灌肠,粘合剂去除,和伤口敷料。两名经验丰富的护士使用新生儿疼痛量表通过双盲评分以10天的间隔进行OS-NPA和VB-NPA，以评估新生儿的疼痛程度。计算和分析了评分者内部和评分者之间的可靠性，使用配对样本t检验来探索通过OS-NPA和VB-NPA得出的评估者疼痛评分的偏倚和一致性。使用三种最先进的AI方法评估了不同标签来源的影响，这些方法使用OS-NPA和VB-NPA给出的标签进行了训练，分别。
结果：同一评估者的内部可靠性在不同时间为0.976-0.983，由类内相关系数测量。单一措施的评估者间可靠性为0.983，平均措施为0.992。在OS-NPA评分和独立VB-NPA评估者的评估之间没有观察到显著差异。对于三种AI方法，不同的标签来源仅导致0.022-0.044的有限精度损失。
结论：与OS-NPA相比，VB-NPA在现实世界中是评估新生儿疼痛的有效方法，因为它具有较高的评分者内部和评分者间可靠性，可用于标记大规模NPA视频数据库以进行临床研究和AI培训。
目的：研究通过基于视频的NPA（VB-NPA）和现场NPA（OS-NPA）得出的疼痛评分的一致性，提供在现实世界中采用VB-NPA结果的科学依据和可行性，作为临床研究中新生儿疼痛的黄金标准和基于人工智能（AI）的NPA（AI-NPA）应用的标签。
方法：从中国某儿科医院招募598例新生儿。
方法：这项观察性研究记录了598例新生儿，这些新生儿经历了10例疼痛手术之一，包括动脉采血,脚跟采血,指尖采血,静脉注射,皮下注射,外周静脉插管,鼻咽吸引,保留灌肠,粘合剂去除,和伤口敷料。两名经验丰富的护士使用新生儿疼痛量表通过双盲评分以10天的间隔进行OS-NPA和VB-NPA，以评估新生儿的疼痛程度。计算和分析了评分者内部和评分者之间的可靠性，使用配对样本t检验来探索通过OS-NPA和VB-NPA得出的评估者疼痛评分的偏倚和一致性。使用三种最先进的AI方法评估了不同标签来源的影响，这些方法使用OS-NPA和VB-NPA给出的标签进行了训练，分别。
结果：同一评估者的内部可靠性在不同时间为0.976-0.983，由类内相关系数测量。单一措施的评估者间可靠性为0.983，平均措施为0.992。在OS-NPA评分和独立VB-NPA评估者的评估之间没有观察到显著差异。对于三种AI方法，不同的标签来源仅导致0.022-0.044的有限精度损失。
结论：与OS-NPA相比，VB-NPA在现实世界中是评估新生儿疼痛的有效方法，因为它具有较高的评分者内部和评分者间可靠性，可用于标记大规模NPA视频数据库以进行临床研究和AI培训。
