curse of dimensionality

维度的诅咒
  • 文章类型: Journal Article
    马尔可夫法是一种常用的可靠性评估方法。它通常用于描述系统的动态特性,比如它的可修复性,故障序列和多个退化状态。然而,“维度的诅咒”,是指随着系统复杂度的增加,系统状态空间呈指数增长,对基于马尔可夫方法的复杂系统可靠性评估提出了挑战。为了应对这一挑战,提出了一种基于非齐次马尔可夫过程的复杂系统可靠性评估方法。这种方法需要将复杂的系统分解为多级子系统,每个都有相对较小的状态空间,根据系统功能。自下而上为每个子系统/系统建立齐次马尔可夫模型或非齐次马尔可夫模型。为了利用下层子系统模型的结果作为上层子系统模型的输入,提出了一种将子系统的不可用性曲线转换为其相应的2×2动态状态转移概率矩阵(STPM)的算法。然后将STPM用作上层系统的非齐次马尔可夫模型的输入。基于所提出的方法,使用反应堆保护系统(RPS)的可靠性评估进行了案例研究,然后将其与基于其他两种对比方法的模型进行比较。通过对比验证了该方法的有效性和准确性。
    The Markov method is a common reliability assessment method. It is often used to describe the dynamic characteristics of a system, such as its repairability, fault sequence and multiple degradation states. However, the \"curse of dimensionality\", which refers to the exponential growth of the system state space with the increase in system complexity, presents a challenge to reliability assessments for complex systems based on the Markov method. In response to this challenge, a novel reliability assessment method for complex systems based on non-homogeneous Markov processes is proposed. This method entails the decomposition of a complex system into multilevel subsystems, each with a relatively small state space, in accordance with the system function. The homogeneous Markov model or the non-homogeneous Markov model is established for each subsystem/system from bottom to top. In order to utilize the outcomes of the lower-level subsystem models as inputs to the upper-level subsystem model, an algorithm is proposed for converting the unavailability curve of a subsystem into its corresponding 2×2 dynamic state transition probability matrix (STPM). The STPM is then employed as an input to the upper-level system\'s non-homogeneous Markov model. A case study is presented using the reliability assessment of the Reactor Protection System (RPS) based on the proposed method, which is then compared with the models based on the other two contrast methods. This comparison verifies the effectiveness and accuracy of the proposed method.
    导出

    更多引用

    收藏

    翻译标题摘要

    我要上传

       PDF(Pubmed)

  • 文章类型: Journal Article
    许多复杂的人类疾病的原因在很大程度上仍然未知。遗传学在揭示复杂人类疾病的分子机制中起着重要作用。表征复杂人类疾病的遗传学的关键步骤是在全基因组尺度上无偏差地鉴定疾病相关基因转录本。混杂因素可能导致假阳性。配对设计,例如测量同一受试者治疗前后的基因表达,可以减少已知混杂因素的影响。然而,并非所有已知的混杂因素都可以在配对/匹配设计中进行控制。基于模型的聚类,例如分层模型的混合,已提出检测配对样品之间差异表达的基因转录本。据我们所知,没有基于模型的基因聚类方法有能力调整协变量的影响。在这篇文章中,在使用配对设计的高通量全基因组数据鉴定差异表达的转录本时,我们提出了一种新的分层模型与协变量调整的混合.仿真研究和实际数据分析都表明了该方法的良好性能。
    The causes of many complex human diseases are still largely unknown. Genetics plays an important role in uncovering the molecular mechanisms of complex human diseases. A key step to characterize the genetics of a complex human disease is to unbiasedly identify disease-associated gene transcripts on a whole-genome scale. Confounding factors could cause false positives. Paired design, such as measuring gene expression before and after treatment for the same subject, can reduce the effect of known confounding factors. However, not all known confounding factors can be controlled in a paired/match design. Model-based clustering, such as mixtures of hierarchical models, has been proposed to detect gene transcripts differentially expressed between paired samples. To the best of our knowledge, no model-based gene clustering methods have the capacity to adjust for the effects of covariates yet. In this article, we proposed a novel mixture of hierarchical models with covariate adjustment in identifying differentially expressed transcripts using high-throughput whole-genome data from paired design. Both simulation study and real data analysis show the good performance of the proposed method.
    导出

    更多引用

    收藏

    翻译标题摘要

    我要上传

       PDF(Pubmed)

  • 文章类型: Journal Article
    Despite the numerous band selection (BS) algorithms reported in the field, most if not all have exhibited maximal accuracy when more spectral bands are utilized for classification. This apparently disagrees with the theoretical model of the \'curse of dimensionality\' phenomenon, without apparent explanations. If it were true, then BS would be deemed as an academic piece of research without real benefits to practical applications. This paper presents a spatial spectral mutual information (SSMI) BS scheme that utilizes a spatial feature extraction technique as a preprocessing step, followed by the clustering of the mutual information (MI) of spectral bands for enhancing the efficiency of the BS. Through the SSMI BS scheme, a sharp \'bell\'-shaped accuracy-dimensionality characteristic that peaks at about 20 bands has been observed for the very first time. The performance of the proposed SSMI BS scheme has been validated through 6 hyperspectral imaging (HSI) datasets (Indian Pines, Botswana, Barrax, Pavia University, Salinas, and Kennedy Space Center (KSC)), and its classification accuracy is shown to be approximately 10% better than seven state-of-the-art BS schemes (Saliency, HyperBS, SLN, OCF, FDPC, ISSC, and Convolution Neural Network (CNN)). The present result confirms that the high efficiency of the BS scheme is essentially important to observe and validate the Hughes\' phenomenon in the analysis of HSI data. Experiments also show that the classification accuracy can be affected by as much as approximately 10% when a single \'crucial\' band is included or missed out for classification.
    导出

    更多引用

    收藏

    翻译标题摘要

    我要上传

       PDF(Sci-hub)

       PDF(Pubmed)

  • 文章类型: Journal Article
    We evaluate the validity of a projection-based test checking linear models when the number of covariates tends to infinity, and analyze two gene expression datasets. We show that the test is still consistent and derive the asymptotic distributions under the null and alternative hypotheses. The asymptotic properties are almost the same as those when the number of covariates is fixed as long as p/n → 0 with additional mild assumptions. The test dramatically gains dimension reduction, and its numerical performance is remarkable.
    导出

    更多引用

    收藏

    翻译标题摘要

    我要上传

       PDF(Sci-hub)

  • 文章类型: Journal Article
    The traditional predictive model for remaining useful life predictions cannot achieve adaptiveness, which is one of the main problems of said predictions. This paper proposes a LightGBM-based Remaining useful life (RUL) prediction method which considers the process and machining state. Firstly, a multi-information fusion strategy that can effectively reduce the model error and improve the generalization ability of the model is proposed. Secondly, a preprocessing method for improving the time precision and small-time granularity of feature extraction while avoiding dimensional explosion is proposed. Thirdly, an importance coefficient and a custom loss function related to the process and machining state are proposed. Finally, using the processing data of actual tool life cycle, through five evaluation indexes and 25 sets of contrast experiments, the superiority and effectiveness of the proposed method are verified.
    导出

    更多引用

    收藏

    翻译标题摘要

    我要上传

       PDF(Sci-hub)

       PDF(Pubmed)

  • 文章类型: Journal Article
    Due to the spectral complexity and high dimensionality of hyperspectral images (HSIs), the processing of HSIs is susceptible to the curse of dimensionality. In addition, the classification results of ground truth are not ideal. To overcome the problem of the curse of dimensionality and improve classification accuracy, an improved spatial-spectral weight manifold embedding (ISS-WME) algorithm, which is based on hyperspectral data with their own manifold structure and local neighbors, is proposed in this study. The manifold structure was constructed using the structural weight matrix and the distance weight matrix. The structural weight matrix was composed of within-class and between-class coefficient representation matrices. These matrices were obtained by using the collaborative representation method. Furthermore, the distance weight matrix integrated the spatial and spectral information of HSIs. The ISS-WME algorithm describes the whole structure of the data by the weight matrix constructed by combining the within-class and between-class matrices and the spatial-spectral information of HSIs, and the nearest neighbor samples of the data are retained without changing when embedding to the low-dimensional space. To verify the classification effect of the ISS-WME algorithm, three classical data sets, namely Indian Pines, Pavia University, and Salinas scene, were subjected to experiments for this paper. Six methods of dimensionality reduction (DR) were used for comparison experiments using different classifiers such as k-nearest neighbor (KNN) and support vector machine (SVM). The experimental results show that the ISS-WME algorithm can represent the HSI structure better than other methods, and effectively improves the classification accuracy of HSIs.
    导出

    更多引用

    收藏

    翻译标题摘要

    我要上传

       PDF(Sci-hub)

       PDF(Pubmed)

公众号