背景:基于图像的作物生长建模可以通过揭示空间作物随时间的发展来为精准农业做出重大贡献,它允许对相关的未来植物性状进行早期和特定位置的估计,如叶面积或生物量。生成现实而清晰的作物图像的前提是将多个生长影响条件集成到一个模型中,例如初始生长阶段的图像,相关的生长时间,以及有关现场治疗的更多信息。虽然基于图像的模型比基于过程的模型为作物生长建模提供了更大的灵活性,在各种影响增长的条件的综合整合方面仍然存在很大的研究差距。需要进一步的探索和调查来解决这一差距。
方法:我们提出了一个两阶段框架,该框架由第一个图像生成模型和第二个增长估计模型组成,独立训练。图像生成模型是有条件的Wasserstein生成对抗网络(CWGAN)。在此模型的生成器中,条件批量归一化(CBN)用于集成不同类型的条件以及输入图像。这允许模型根据多个影响因素生成时变人工图像。框架的第二部分通过得出植物特定的性状并将其与非人工(真实)参考图像的性状进行比较来使用这些图像进行植物表型分析。此外,使用多尺度结构相似性(MS-SSIM)评估图像质量,学习感知图像块相似性(LPIPS),和Fréchet起始距离(FID)。在推理过程中,该框架允许为训练中使用的任何条件组合生成图像;我们称这种生成为数据驱动的作物生长模拟。
结果:实验是在三个不同复杂度的数据集上进行的。这些数据集包括实验室植物拟南芥(拟南芥)和在实际田间条件下生长的作物,即花椰菜(GrowliFlower)和由蚕豆和春小麦(MixedCrop)组成的作物混合物。在所有情况下,该框架允许现实的,清晰的图像世代,从短期到长期预测的质量略有下降。对于在不同处理下生长的混合作物(不同品种,播种密度),结果表明,添加这些处理信息增加了一代质量和表型的准确性测量的估计生物量。用受过训练的框架对不同的生长影响条件进行模拟,为这些因素如何与作物外观相关提供了有价值的见解。这在复杂的情况下特别有用,探索较少的作物混合物系统。进一步的结果表明,添加基于过程的模拟生物量作为条件增加了来自预测图像的衍生表型性状的准确性。这证明了我们的框架作为数据驱动和基于过程的作物生长模型之间的接口的潜力。
结论:通过多条件CWGAN,对未来植物外观的真实生成和模拟是充分可行的。提出的框架补充了基于流程的模型,克服了它们的局限性,例如对假设的依赖和低精确的现场定位特异性,通过对空间作物发育的逼真可视化,直接导致模型预测的高度可解释性。
BACKGROUND: Image-based crop growth modeling can substantially contribute to precision agriculture by revealing spatial crop development over time, which allows an early and location-specific estimation of relevant future plant traits, such as leaf area or biomass. A prerequisite for realistic and sharp crop image generation is the integration of multiple growth-influencing conditions in a model, such as an image of an initial growth stage, the associated growth time, and further information about the field treatment. While image-based models provide more flexibility for crop growth modeling than process-based models, there is still a significant research gap in the comprehensive integration of various growth-influencing conditions. Further exploration and investigation are needed to address this gap.
METHODS: We present a two-stage framework consisting first of an image generation model and second of a growth estimation model, independently trained. The image generation model is a conditional Wasserstein generative adversarial network (CWGAN). In the generator of this model, conditional batch normalization (CBN) is used to integrate conditions of different types along with the input image. This allows the model to generate time-varying artificial images dependent on multiple influencing factors. These images are used by the second part of the framework for plant phenotyping by deriving plant-specific traits and comparing them with those of non-artificial (real) reference images. In addition, image quality is evaluated using multi-scale structural similarity (MS-SSIM), learned perceptual image patch similarity (LPIPS), and Fréchet inception distance (FID). During inference, the framework allows image generation for any combination of conditions used in training; we call this generation data-driven crop growth simulation.
RESULTS: Experiments are performed on three datasets of different complexity. These datasets include the laboratory plant Arabidopsis thaliana (Arabidopsis) and crops grown under real field conditions, namely cauliflower (GrowliFlower) and crop mixtures consisting of faba bean and spring wheat (MixedCrop). In all cases, the framework allows realistic, sharp image generations with a slight loss of quality from short-term to long-term predictions. For MixedCrop grown under varying treatments (different cultivars, sowing densities), the results show that adding these treatment information increases the generation quality and phenotyping accuracy measured by the estimated biomass. Simulations of varying growth-influencing conditions performed with the trained framework provide valuable insights into how such factors relate to crop appearances, which is particularly useful in complex, less explored crop mixture systems. Further results show that adding process-based simulated biomass as a condition increases the accuracy of the derived phenotypic traits from the predicted images. This demonstrates the potential of our framework to serve as an interface between a data-driven and a process-based crop growth model.
CONCLUSIONS: The realistic generation and simulation of future plant appearances is adequately feasible by multi-conditional CWGAN. The presented framework complements process-based models and overcomes their limitations, such as the reliance on assumptions and the low exact field-localization specificity, by realistic visualizations of the spatial crop development that directly lead to a high explainability of the model predictions.