蛋白质语言模型(pLM)彻底改变了蛋白质系统的计算建模,建立以结构特征为中心的数值嵌入。为了提高蛋白质嵌入中可获得的生物化学相关特性的广度,我们设计了注释词汇,一种由结构化本体定义的蛋白质属性的可读语言。我们从头开始训练AnnotationTransformers(AT),以恢复掩蔽的蛋白质属性输入,而不参考氨基酸序列,仅在蛋白质描述上构建新的数字特征空间。我们在各种模型架构中利用AT表示,用于蛋白质表示和生成。为了展示注释词汇整合的优点,我们进行了515个不同的下游实验。使用一个新的损失函数,在商业计算中只有3美元,我们的主要表示模型CAMP为15个常见数据集中的5个生成了最先进的嵌入,其余数据具有竞争力;通过注释词汇突出了潜在空间策展的计算效率。为了标准化从头生成的蛋白质序列的比较,我们提出了一种新的基于序列比对的评分,该评分比传统的语言建模指标更加灵活和生物学相关.我们的生成模型,GSM,使用类似BERT的生成方案,从仅注释提示中产生高对齐分数。特别值得注意的是,许多GSM幻觉返回统计上显著的BLAST命中,其中,即使地面真值与整个训练集具有低序列同一性,富集分析也显示与注释提示匹配的属性。总的来说,注释词汇工具箱提供了一个有希望的途径,用本体和知识图的成员代替传统的标记,在特定领域增强变压器模型。简洁,准确,注释词汇对蛋白质的有效描述提供了一种新颖的方法来构建蛋白质的数字表示以进行蛋白质注释和设计。
Protein Language Models (pLMs) have revolutionized the computational modeling of protein systems, building numerical embeddings that are centered around structural features. To enhance the breadth of biochemically relevant properties available in protein embeddings, we engineered the Annotation Vocabulary, a transformer readable language of protein properties defined by structured ontologies. We trained Annotation Transformers (AT) from the ground up to recover masked protein property inputs without reference to amino acid sequences, building a new numerical feature space on protein descriptions alone. We leverage AT representations in various model architectures, for both protein representation and generation. To showcase the merit of Annotation Vocabulary integration, we performed 515 diverse downstream experiments. Using a novel loss function and only $3 in commercial compute, our premier representation model CAMP produces state-of-the-art embeddings for five out of 15 common datasets with competitive performance on the rest; highlighting the computational efficiency of latent space curation with Annotation Vocabulary. To standardize the comparison of de novo generated protein sequences, we suggest a new sequence alignment-based score that is more flexible and biologically relevant than traditional language modeling metrics. Our generative model, GSM, produces high alignment scores from annotation-only prompts with a BERT-like generation scheme. Of particular note, many GSM hallucinations return statistically significant BLAST hits, where enrichment analysis shows properties matching the annotation prompt - even when the ground truth has low sequence identity to the entire training set. Overall, the Annotation Vocabulary toolbox presents a promising pathway to replace traditional tokens with members of ontologies and knowledge graphs, enhancing transformer models in specific domains. The concise, accurate, and efficient descriptions of proteins by the Annotation Vocabulary offers a novel way to build numerical representations of proteins for protein annotation and design.