A team of four annotators did linguistic annotation of 30,000 clinical progress notes from 231 Veterans in VHA electronic health records who attempted suicide or who died by suicide for terms relating to the 3ST factors. Annotation involved manually assigning a label to words or phrases that indicated presence or absence of the factor (polarity). These words and phrases were entered into a controlled vocabulary that was then used by our computational system to tag 14 million clinical progress notes from Veterans who attempted or died by suicide after 2013. Tagged text was extracted and machine-labelled for presence or absence of the 3ST factors. Accuracy of these machine-labels was determined for 1000 randomly selected extractions for each factor against a ground truth created by our annotators.
Linguistic annotation identified 8486 terms that related to 33 subclasses across the four factors and polarities. Precision of machine-labeled extractions ranged from 0.73 to 1.00 for most factor-polarity combinations, whereas recall was somewhat lower 0.65-0.91.
The ontology that was developed consists of classes that represent each of the four 3ST factors, subclasses, relationships, and terms that map onto those classes which are stored in a controlled vocabulary (https://bioportal.bioontology.org/ontologies/THREE-ST). The use case that we present shows how scores based on clinical notes tagged for terms in the controlled vocabulary capture meaningful change in the 3ST factors during weeks preceding a suicidal event.
方法:一个由四个注释者组成的团队对VHA电子健康记录中231名退伍军人的30,000个临床进展记录进行了语言注释,这些退伍军人企图自杀或因自杀而死亡,涉及与3ST因素有关的术语。注释涉及手动将标签分配给指示存在或不存在因素(极性)的单词或短语。这些单词和短语被输入到一个受控的词汇中,然后被我们的计算系统用来标记2013年后企图自杀或死于自杀的退伍军人的1400万份临床进展记录。提取标记文本并机器标记是否存在3ST因子。根据我们的注释者创建的地面实况,针对每个因素随机选择的1000次提取,确定了这些机器标签的准确性。
结果:语言注释确定了8486个术语,涉及四个因素和极性的33个子类。对于大多数因子-极性组合,机器标记的提取精度范围为0.73至1.00,而召回率略低0.65-0.91。
结论:开发的本体由代表四个3ST因子的类组成,子类,关系,以及映射到存储在受控词汇表(https://bioportal)中的那些类的术语。bioontology.org/ontologies/THREE-ST)。我们提供的用例显示了基于临床注释标记的受控词汇中的术语的分数如何在自杀事件发生前的几周内捕获3ST因子的有意义的变化。