Abstract：
Environmental changes and human activities have caused serious degradation of murals around the world. Scratches are one of the most common issues in these damaged murals. We propose a new method for virtually enhancing and removing scratches from murals; which can provide an auxiliary reference and support for actual restoration. First, principal component analysis (PCA) was performed on the hyperspectral data of a mural after reflectance correction, and high-pass filtering was performed on the selected first principal component image. Principal component fusion was used to replace the original first principal component with a high-pass filtered first principal component image, which was then inverse PCA transformed with the other original principal component images to obtain an enhanced hyperspectral image. The linear information in the mural was therefore enhanced, and the differences between the scratches and background improved. Second, the enhanced hyperspectral image of the mural was synthesized as a true colour image and converted to the HSV colour space. The light brightness component of the image was estimated using the multi-scale Gaussian function and corrected with a 2D gamma function, thus solving the problem of localised darkness in the murals. Finally, the enhanced mural images were applied as input to the triplet domain translation network pretrained model. The local branches in the translation network perform overall noise smoothing and colour recovery of the mural, while the partial nonlocal block is used to extract the information from the scratches. The mapping process was learned in the hidden space for virtual removal of the scratches. In addition, we added a Butterworth high-pass filter at the end of the network to generate the final restoration result of the mural with a clearer visual effect and richer high-frequency information. We verified and validated these methods for murals in the Baoguang Hall of Qutan Temple. The results show that the proposed method outperforms the restoration results of the total variation (TV) model, curvature-driven diffusion (CDD) model, and Criminisi algorithm. Moreover, the proposed combined method produces better recovery results and improves the visual richness, readability, and artistic expression of the murals compared with direct recovery using a triple domain translation network.
摘要：
环境变化和人类活动造成了世界各地壁画的严重退化。划痕是这些受损壁画中最常见的问题之一。我们提出了一种虚拟增强和消除壁画划痕的新方法；这可以为实际修复提供辅助参考和支持。首先,主成分分析（PCA）对反射校正后的壁画的高光谱数据，对选择的第一主成分图像进行高通滤波。主成分融合用于用高通滤波的第一主成分图像代替原始的第一主成分。然后将其与其他原始主成分图像进行PCA逆变换，以获得增强的高光谱图像。因此，壁画中的线性信息得到了增强，划痕和背景之间的差异得到了改善。第二,将增强的高光谱壁画图像合成为真彩色图像，并转换为HSV颜色空间。使用多尺度高斯函数估计图像的光亮度分量，并使用2D伽马函数进行校正，从而解决了壁画中局部黑暗的问题。最后,将增强后的壁画图像作为输入应用于三元组域平移网络预训练模型。翻译网络中的局部分支执行壁画的整体噪声平滑和颜色恢复，而部分非局部块用于从划痕中提取信息。在隐藏空间中学习映射过程，以虚拟去除划痕。此外,我们在网络末端添加了Butterworth高通滤波器，以生成具有更清晰的视觉效果和更丰富的高频信息的壁画的最终还原结果。我们在曲谈寺宝光堂壁画中对这些方法进行了验证和验证。结果表明，该方法优于全变分(TV)模型的恢复结果，曲率驱动扩散(CDD)模型，和Criminisi算法。此外，所提出的组合方法产生更好的恢复结果，并提高了视觉丰富度，可读性,与使用三域翻译网络直接恢复壁画的艺术表现相比。
