Multimodal neural network

  • 文章类型: Journal Article
    脑出血(ICH)患者偶尔会出现血肿扩张,与糟糕的结果联系在一起。结合图像的卷积神经网络(CNN)分析和表格数据的神经网络分析的多模态神经网络已知在预测和分类任务中显示有希望的结果。我们旨在开发一种可靠的多模式神经网络模型,该模型可全面分析CT图像和临床变量以预测血肿扩展。我们回顾性地纳入了2017年至2021年在四家医院的ICH患者,将三家医院的患者分配到培训和验证数据集,将一家医院的患者分配到测试数据集。收集入院CT图像和临床变量。CT检查结果由专家评估。开发和训练了三种类型的模型:(1)分析CT图像的CNN模型,(2)分析CT图像和临床变量的多模态CNN模型,和(3)非CNN模型,利用机器学习分析CT结果和临床变量。在测试数据集上对模型进行了评估,首先关注灵敏度,其次关注接收器工作曲线下面积(AUC)。两百七十三名患者(中位年龄,71岁[59-79];159名男性)在训练和验证数据集中,106名患者(中位年龄,70岁[62-82];63名男性)纳入测试数据集。CNN模型的灵敏度和AUC分别为1.000(95%置信区间[CI]0.768-1.000)和0.755(95%CI0.704-0.807);多模态CNN模型的灵敏度和AUC分别为1.000(95%CI0.768-1.000)和0.799(95%CI0.749-0.849);非CNN模型的灵敏度和AUC分别为0.857(95%CI0.572-0.我们开发了一种多模式神经网络模型,该模型结合了CT图像的CNN分析和临床变量的神经网络分析,以预测ICH中的血肿扩展。该模型经过外部验证,并显示出所有模型中的最佳性能。
    Hematoma expansion occasionally occurs in patients with intracerebral hemorrhage (ICH), associating with poor outcome. Multimodal neural networks incorporating convolutional neural network (CNN) analysis of images and neural network analysis of tabular data are known to show promising results in prediction and classification tasks. We aimed to develop a reliable multimodal neural network model that comprehensively analyzes CT images and clinical variables to predict hematoma expansion. We retrospectively enrolled ICH patients at four hospitals between 2017 and 2021, assigning patients from three hospitals to the training and validation dataset and patients from one hospital to the test dataset. Admission CT images and clinical variables were collected. CT findings were evaluated by experts. Three types of models were developed and trained: (1) a CNN model analyzing CT images, (2) a multimodal CNN model analyzing CT images and clinical variables, and (3) a non-CNN model analyzing CT findings and clinical variables with machine learning. The models were evaluated on the test dataset, focusing first on sensitivity and second on area under the receiver operating curve (AUC). Two hundred seventy-three patients (median age, 71 years [59-79]; 159 men) in the training and validation dataset and 106 patients (median age, 70 years [62-82]; 63 men) in the test dataset were included. Sensitivity and AUC of a CNN model were 1.000 (95% confidence interval [CI] 0.768-1.000) and 0.755 (95% CI 0.704-0.807); those of a multimodal CNN model were 1.000 (95% CI 0.768-1.000) and 0.799 (95% CI 0.749-0.849); and those of a non-CNN model were 0.857 (95% CI 0.572-0.982) and 0.733 (95% CI 0.625-0.840). We developed a multimodal neural network model incorporating CNN analysis of CT images and neural network analysis of clinical variables to predict hematoma expansion in ICH. The model was externally validated and showed the best performance of all the models.
    导出

    更多引用

    收藏

    翻译标题摘要

    我要上传

       PDF(Pubmed)

  • 文章类型: Journal Article
    近年来,使用人工智能算法对色素性皮肤病变进行分类的准确性有了显著提高。智能分析和分类系统明显优于皮肤科医生和肿瘤学家使用的视觉诊断方法。然而,由于缺乏通用性和潜在错误分类的风险,此类系统在临床实践中的应用受到严重限制。在临床病理实践中成功实施基于人工智能的工具需要对现有模型的有效性和性能进行全面研究,以及潜在研究发展的进一步有希望的领域。本系统综述的目的是调查和评估人工智能技术用于检测色素性皮肤病变的恶性形式的准确性。对于这项研究,从电子科学出版商中选择了10,589篇科学研究和评论文章,其中171篇文章被纳入本系统综述。所有选定的科学文章都根据所提出的神经网络算法从机器学习到多模态智能架构进行分发,并在手稿的相应部分进行了描述。这项研究旨在探索自动皮肤癌识别系统,从简单的机器学习算法到基于高级编码器-解码器模型的多模态集成系统,视觉变压器(ViT),以及生成和尖峰神经网络。此外,作为分析的结果,未来的研究方向,前景,并讨论了进一步开发用于对色素性皮肤病变进行分类的自动神经网络系统的潜力。
    In recent years, there has been a significant improvement in the accuracy of the classification of pigmented skin lesions using artificial intelligence algorithms. Intelligent analysis and classification systems are significantly superior to visual diagnostic methods used by dermatologists and oncologists. However, the application of such systems in clinical practice is severely limited due to a lack of generalizability and risks of potential misclassification. Successful implementation of artificial intelligence-based tools into clinicopathological practice requires a comprehensive study of the effectiveness and performance of existing models, as well as further promising areas for potential research development. The purpose of this systematic review is to investigate and evaluate the accuracy of artificial intelligence technologies for detecting malignant forms of pigmented skin lesions. For the study, 10,589 scientific research and review articles were selected from electronic scientific publishers, of which 171 articles were included in the presented systematic review. All selected scientific articles are distributed according to the proposed neural network algorithms from machine learning to multimodal intelligent architectures and are described in the corresponding sections of the manuscript. This research aims to explore automated skin cancer recognition systems, from simple machine learning algorithms to multimodal ensemble systems based on advanced encoder-decoder models, visual transformers (ViT), and generative and spiking neural networks. In addition, as a result of the analysis, future directions of research, prospects, and potential for further development of automated neural network systems for classifying pigmented skin lesions are discussed.
    导出

    更多引用

    收藏

    翻译标题摘要

    我要上传

    求助全文

  • 文章类型: Journal Article
    近几十年来,线虫秀丽隐杆线虫(C.线虫)使衰老研究取得了巨大进展。然而,手动执行这些测定是一项费力的任务。为了解决这个问题,许多秀丽隐杆线虫测定自动化技术正在被开发以增加通量和准确性。在本文中,提出并分析了利用双峰神经网络预测秀丽隐杆线虫寿命的方法。具体来说,该模型使用序列的图像和活的C.elegans到当前的一天的计数来预测寿命曲线终止。已使用模拟器对该网络进行了训练,以避免训练此类模型的标签成本。此外,提出了一种估计模型预测不确定性的方法。利用这种不确定性,已经分析了一个标准,以决定在什么时候可以停止分析,用户可以依靠模型的预测。该方法已通过实际实验进行了分析和验证。结果表明,不确定性从平均寿命降低,并且所获得的大多数预测与手动获得的曲线没有统计学上的显着差异。
    In recent decades, assays with the nematode Caenorhabditis elegans (C. elegans) have enabled great advances to be made in research on aging. However, performing these assays manually is a laborious task. To solve this problem, numerous C. elegans assay automation techniques are being developed to increase throughput and accuracy. In this paper, a method for predicting the lifespan of C. elegans nematodes using a bimodal neural network is proposed and analyzed. Specifically, the model uses the sequence of images and the count of live C. elegans up to the current day to predict the lifespan curve termination. This network has been trained using a simulator to avoid the labeling costs of training such a model. In addition, a method for estimating the uncertainty of the model predictions has been proposed. Using this uncertainty, a criterion has been analyzed to decide at what point the assay could be halted and the user could rely on the model\'s predictions. The method has been analyzed and validated using real experiments. The results show that uncertainty is reduced from the mean lifespan and that most of the predictions obtained do not present statistically significant differences with respect to the curves obtained manually.
    导出

    更多引用

    收藏

    翻译标题摘要

    我要上传

       PDF(Pubmed)

  • 文章类型: Journal Article
    通过分析传统的深度学习多模式检索方法,建立了基于卷积神经网络的多模式检索优化模型。本文基于用户画像模型,提出了一种创新的半监督社交网络用户画像分析模型(UPAM),它将用户的社交信息和一些已知的用户属性信息(如教育背景和居住地)集成到一个统一的主题模型框架中。最后,提出了一种基于用户社会信息和部分已知用户属性信息的半监督用户画像分析方法。根据用户属性的相关性,交叉验证方法用于训练模型预测任务,提高预测效果。在第一级模型中,使用不同的模型来提取用户查询中的特征,第二层次模型的基础,堆叠用于进一步整合特性,最终实现属性人口预测,和实验验证表明,所提出的模型在种群的各种属性中的有效性。
    By analyzing traditional deep learning multimode retrieval methods, an optimized multimode retrieval model based on convolutional neural network is established. This article proposes an innovative semi-supervised social network user portrait analysis model (UPAM) based on user portrait model, which integrates users\' social information and some known user attribute information (such as educational background and residence) into a unified topic model framework. Finally, a semi-supervised user portrait analysis method based on user social information and partial known user attribute information is proposed. According to the correlation of user attributes, the cross-validation method is used to train model prediction task and improve the prediction effect. In the first-level model, using a different model to extract the features in the user query, the basis of the second hierarchy model, Stacking is used to further integrate characteristics, finally realizing the attribute population forecast, and experimental verification showing the proposed model\'s effectiveness in various attributes of a population.
    导出

    更多引用

    收藏

    翻译标题摘要

    我要上传

       PDF(Pubmed)

公众号