缺失数据是大规模数据集中的常见问题,其适当的处理对于数据分析至关重要。Missingness可以归类为(1)完全随机丢失(MCAR),(2)随机缺失(MAR),(3)非随机缺失(MNAR)。不同的错误机制需要不同的归责策略。多重归责,一种在多个估算数据中平均结果的方法,比单一填补更适合处理各种缺失机制。MissForest,使用随机森林的非参数缺失值填补策略,是缺失数据最普遍的多重填补方法之一,因为它可以应用于混合类型的数据,并且不需要分布假设。然而,最近的一项研究发现,missForest会对非正态数据产生有偏差的结果。此外,missForest在计算上是昂贵的。
因此,我们旨在通过结合基于二进制粒子群优化(BPSO)的特征选择策略来进一步开发missForest算法。
BPSO是一种进化算法,以全局优化和计算效率而闻名。通过在使用missForest估算缺失值之前使用基于BPSO的特征选择步骤,通过修剪冗余变量,可以提高连续变量的填补精度。
在这项研究中,通过在估算步骤之前进行特征选择,具有BPSO的missForest(BPSOmf)在连续变量方面比单独的missForest显示出更好的估算精度。
当归集目标数据主要由连续变量组成时,BPSOmf是一种适当且可靠的方法。
Missing data are a common problem in large-scale datasets and its appropriate handling is crucial for data analyses. Missingness can be categorized as (1) missing completely at random (MCAR), (2) missing at random (MAR), and (3) missing not at random (MNAR). Different missingness mechanisms require different imputation strategies. Multiple imputation, an approach for averaging outcomes across multiple imputed data, is more suitable than single imputation for dealing with various missing mechanisms. missForest, a nonparametric missing value imputation strategy using random forest, is one of the most prevalent multiple imputation methods for missing-data because it can be applied to mixed-type data and does not require distributional assumptions. However, a recent study found that
missForest can produce biased results for non-normal data. In addition,
missForest is computationally expensive.
Therefore, we aimed to further develop the
missForest algorithm by combining a binary particle swarm optimization (BPSO)-based feature-selection strategy.
The BPSO is an evolutionary algorithm that is well known for global optimization and computational efficiency. By using the BPSO-based feature selection step prior to imputing missing values with
missForest, the imputation accuracy for continuous variables could be increased by pruning redundant variables.
In this study,
missForest with BPSO (BPSOmf) showed better imputation accuracy than
missForest alone with respect to continuous variables by feature selection prior to the imputation step.
BPSOmf is an appropriate and robust method when the imputation target data consist mainly of continuous variables.