基于实例白化与特征恢复的草莓病害识别领域泛化方法

doi:10.12133/j.smartag.SA202411016

Smart Agriculture ›› 2025, Vol. 7 ›› Issue (1): 124-135.doi: 10.12133/j.smartag.SA202411016

• 专题--农业知识智能服务和智慧无人农场（下） • 上一篇下一篇

基于实例白化与特征恢复的草莓病害识别领域泛化方法

胡晓波¹, 许桃胜²(), 王成军¹, 朱洪波¹, 甘雷¹

^1. 安徽理工大学人工智能学院，安徽淮南 232001，中国
^2. 中国科学院合肥物质科学研究院，安徽合肥 230031，中国

收稿日期:2024-10-30 出版日期:2025-01-30
基金项目:
安徽理工大学高层次引进人才科研启动基金(2024yjrc05); 国家自然科学基金(62003001); 安徽理工大学校级重点项目(XCZX2021-01)
作者简介:
胡晓波，博士，讲师，研究方向为深度学习与农作物病虫害识别等。E-mail：huxiaobo@mail.ustc.edu.cn
通信作者:
许桃胜，博士，副研究员，研究方向为智慧农业、农业大数据和数据挖掘等。E-mail：xtsvm@ustc.edu.cn

Domain Generalization Method of Strawberry Disease Recognition Based on Instance Whitening and Restitution

HU Xiaobo¹, XU Taosheng²(), WANG Chengjun¹, ZHU Hongbo¹, GAN Lei¹

^1. School of Artificial Intelligence, Anhui University of Science and Technology, Huainan 232001, China
^2. Hefei Institutes of Physical Science, Chinese Academy of Science, Hefei 230031, China

Received:2024-10-30 Online:2025-01-30
Foundation items:Scientific Research Foundation for High-level Talents of Anhui University of Science and Technology(2024yjrc05); National Natural Science Foundation of China(62003001); University-Level Key Projects of Anhui University of Science and Technology(XCZX2021-01)
About author:
HU Xiaobo, E-mail: huxiaobo@mail.ustc.edu.cn
Corresponding author:
XU Taosheng, E-mail: xtsvm@ustc.edu.cn

摘要/Abstract

摘要：

【目的/意义】 基于深度神经网络的草莓病害识别模型通常假设训练集（源域）和测试集（目标域）满足独立同分布。然而，在实际应用中由于光照、背景环境和草莓品种等多种因素的影响，测试集与训练集存在领域差异，造成模型在应用过程中的识别精度出现明显下降。针对这一问题，本研究提出一种基于实例白化与特征恢复的领域泛化方法，用于提升草莓病害识别模型的泛化性能。 【方法】 该方法首先利用实例白化技术消除源域和目标域间的风格差异，再从滤除的风格特征中提取任务相关特征，最后将任务相关特征恢复到白化后的特征中，以减轻实例白化对特征类别区分度的影响。为增强从风格特征中分离任务相关特征的效果，设计了两个特征提取器分别提取任务相关和任务无关特征，并采用双段恢复损失约束两特征提取器所提取特征与任务的相关性，引入互信息损失确保特征的相互独立，进一步增强特征分类效果。 【结果和讨论】 该方法可以在不降低源域识别精度的前提下，有效提升各病害识别模型在目标域上的泛化性能，如AlexNet加入该算法后，其不同风格目标域上的识别精度可分别提升3.97个百分点和2.79个百分点。相较于IBN-Net（Instance Batch Normalization Net）、可切换白化（Switchable Whitening, SW）、样式归一化和恢复模块（Style Normalization and Restitution, SNR）等其他领域泛化方法，该算法在测试数据集上的泛化性能可分别提高2.63%、2.35%和1.14%。 【结论】 本方法可有效提升基于深度学习的草莓病害识别模型在目标域中的泛化性能，可为草莓病害精准识别提供可靠的技术支撑。

关键词: 深度神经网络, 草莓病害识别, 实例白化, 特征恢复, 领域泛化

Abstract:

[Objective] Strawberry disease recognition models based on deep neural networks generally assume that the training (source domain) and the test (target domain) datasets are identically and independently distributed. However, in practical applications, due to the influence of illumination, background and strawberry variety, the target domain often exhibits significant domain shift from the source domain. The domain shift result in accuracy decline of the models in target domain. To address this problem, a domain generalization method based on instant whitening and restitution (IWR) was proposed to improve the generalization performance of strawberry disease identification models in this research. [Methods] Samples from different source often exhibit great domain shift due to variations in strawberry varieties, regional climate, and photography methods. Therefore, a dataset was constructed for domain generalization research on strawberry disease using two distinct approaches. The first dataset was acquired using a Nikon D810 camera at multiple strawberry farms in Changfeng county, Anhui province, with a fixed sampling schedule and fixed camera distance. In contrast, the second dataset was an open-source collection, primarily comprising images captured using smartphones in multiple strawberry greenhouses in Korea, with varied and random shooting distances and angles. The IWR module mitigated style variations (e.g., illumination, color) through instance whitening, where features were normalized to reduce domain discrepancies between the datasets. However, such operation was task-ignorant and inevitable removed some task-relevant information, which may be harmful to classification performance of the models. To remedy this, the removed task-relevant features were attempted to recover. Specifically, two modules were designed to extract task-relevant and task-irrelevant feature from the filtered style features, respectively. A dual restitution loss was utilized to constraint the modules' feature correlation between the task and a mutual loss was used to ensure the independence of the features. In addition, a separation optimization strategy was adopted to further enhance the feature separation effect of the two modules. [Results and Discussions] The F₁-Score was adopted as evaluation metrics. A series of ablations studies and comparative experiments were conducted to demonstrate the effectiveness of the proposed IWR. The ablation experiments proved that the IWR could effectively eliminate the style variations between different datasets and separate task-relevant feature from the filtered style features, which could simultaneously enhance model generalization and discrimination capabilities. The recognition accuracy increased when IWR pluged to AlexNet, GoogLeNet, ResNet-18, ResNet-50, MobileNetV2 and MobileNetV3. It demonstrated that the proposed IWR was an effective way to improve the generalization of the models. Compared with other domain generalization methods such as IBNNet, SW and SNR, the generalization performance of the proposed algorithm on test datasets could be improved by 2.63%, 2.35% and 1.14%, respectively. To better understand how IWR works, the intermediate feature maps of ResNet-50 without and with IWR were compared. The visualization result showed that the model with IWR was more robust when the image style changed. These results indicated that the proposed IWR achieves high classification accuracy and boosts the generalization performance of the models. [Conclusions] An instance whitening and restitution module was presented, which aimed to learn generalizable and discriminative feature representations for effective domain generalization. The IWR was a plug-and-play module, it could be inserted into existing convolutional networks for strawberry disease recognition. Style normalization and restitution (SNR) reduced the style information through instance whitening operation and then restitutes the task-relevant discriminative features caused by instance whitening. The introduced dual restitution loss and mutual loss further facilitate the separation of task-relevant and task-irrelevant feature. The schemes powered by IWR achieves the state-of-the-art performance on strawberry disease identification.

Key words: deep neural networks, strawberry disease recognition, instant whitening, feature restitution, domain generalization

中图分类号:

胡晓波, 许桃胜, 王成军, 朱洪波, 甘雷. 基于实例白化与特征恢复的草莓病害识别领域泛化方法[J]. 智慧农业(中英文), 2025, 7(1): 124-135.

HU Xiaobo, XU Taosheng, WANG Chengjun, ZHU Hongbo, GAN Lei. Domain Generalization Method of Strawberry Disease Recognition Based on Instance Whitening and Restitution[J]. Smart Agriculture, 2025, 7(1): 124-135.

图/表 16

表1

图 1

图2

图3

图 4

表2

任务相关特征提取器 E T R F和任务无关特征提取器 E T I F详细信息

$E T R F$ 提取任务相关特征		$E T I F$ 提取任务无关特征
Conv 5×5×128， stride=1， padding 2		Conv 3×3×128， stride=1， padding 1
Instance Normalization， ReLU		Instance Normalization， ReLU
Conv 5×5×128， stride=1， padding 2		Conv 3×3×128， stride=1， padding 1
Instance Normalization， ReLU		Instance Normalization， ReLU
Conv 5×5×128， stride=1， padding 2		Conv 3×3×128， stride=1， padding 1
Instance Normalization， Sigmoid		Instance Normalization， Sigmoid

表2

表3

互信息估计器 Φ的详细信息

互信息估计器 $Φ$
FC1 128×128， FC2 128×128
FC3 128×1

表3

表4

IWR模型训练流程

算法 1 分离优化算法
	Input：输入样本集（X，Y）；骨干网络 $E b$ ；任务相关特征提取器 $E T R F$ ；任务相关特征提取器 $E T I F$ ；互信息估计器 $Φ$
	Output：骨干网络 $E ̂ b$ ；任务相关特征提取器 $E ̂ T R F$ ；任务相关特征提取器 $E ̂ T I F$ ；互信息估计器 $Φ ̂$
1	While 模型不收敛 do
2		从（X，Y）选取一批样本；
3		特征类别区分度优化：
4		固定互信息估计器 $Φ$ 参数；
5		计算损失 $L f d = L c l s + L D R$ ；
6		通过 $L f d$ 更新 $E b$ 、 $E T R F$ 、 $E T I F$ ；
7		特征独立性优化：
8		固定骨干网络 $E b$ 参数；
9		计算损失 $L f i = L M I + L D R$ ；
10		通过 $L f i$ 更新 $Φ$ 、 $E T R F$ 、 $E T I F$ ；
11	End
12	return $E ̂ b = E b$ ； $E T R F = E ̂ T R F$ ； $E T I F = E ̂ T I F$ ；

表4

表5

不同特征分离方法领域泛化性能比较

方法	F ₁-Score/%
方法	DGSR1 $→$ DGSR2	DGSR2 $→$ DGSR1
$I W R T R$	67.75	66.62
$I W R R - T R$	68.32	67.89
$I W R - M I$	68.01	66.63
$I W R a t t e n$	68.64	68.12
$I W R$	69.46	68.97

表5

表 6

双段恢复损失的消融实验结果

方法	F ₁-Score/%
方法	DGSR1 $→$ DGSR2	DGSR2 $→$ DGSR1
ResNet-50	66.65	65.15
IWR w/o $L D R$	66.81	65.53
IWR w/o $L D R -$	68.63	68.27
IWR w/o $L D R +$	67.96	66.92
IWR	69.46	68.97

表 6

表7

不同Stage加入IWR对Resnet泛化性能影响

方法	F ₁-Score/%
方法	DGSR1 $→$ DGSR2	DGSR2 $→$ DGSR1
Stage 1	67.54	65.81
Stage 2	67.75	66.35
Stage 3	67.91	66.76
Stage 4	68.19	67.48
Stage 5	68.72	68.39
IWR （All Stage）	69.46	68.97

表7

表8

IWR对不同网络领域泛化性能提升效果对比

方法	F ₁-Score/%
	DGSR1 $→$ DGSR2		DGSR2 $→$ DGSR1
	DGSR1	DGSR2	DGSR2	DGSR1
AlexNet	90.90	62.12	95.39	62.25
AlexNet-IWR	91.96	66.09	95.48	65.04
GoogLeNet	92.44	62.87	95.62	63.44
GoogLeNet-IWR	92.86	67.45	95.69	65.81
ResNet-18	92.91	64.80	96.85	64.06
ResNet-18-IWR	92.99	67.99	96.82	66.79
ResNet-50	93.90	66.65	97.33	65.15
ResNet-50-IWR	94.21	69.46	97.30	68.97
MobileNetV2	93.47	66.76	97.29	62.55
MobileNetV2-IWR	93.74	68.35	97.38	65.31
MobileNetV3	91.84	62.92	96.01	60.67
MobileNetV3-IWR	92.65	66.43	96.15	63.89

表8

表9

不同风格归一化方法领域泛化性能比较

骨干网络	方法	DGSR1 $→$ DGSR2			DGSR2 $→$ DGSR1
骨干网络	方法	平均值/%	标准差/%	95%置信区间/%	平均值/%	标准差/%	95%置信区间/%
ResNet-50	原模型	66.65	1.74	65.12~68.18	65.15	1.98	63.41~66.89
	IBNNet	67.72	1.69	66.24~69.20	66.34	1.83	64.74~67.94
	SW	67.41	1.65	65.96~68.85	66.62	1.87	64.98~68.26
	SNR	68.62	1.54	67.27~69.97	67.83	1.72	66.32~69.34
	IWR	69.46	1.50	68.15~70.77	68.97	1.66	67.51~70.43

表9

图5

图 6

表 10

参考文献 27

1	王辉, 陈睿鹏, 余志雪, 等. 基于卟啉和半导体单壁碳纳米管的场效应气体传感器检测草莓恶疫霉[J]. 智慧农业(中英文), 2022, 4(3): 143-151.
	WANG H, CHEN R P, YU Z X, et al. Porphyrin and semiconducting single wall carbon nanotubes based semiconductor field effect gas sensor for determination of phytophthora strawberries[J]. Smart agriculture, 2022, 4(3): 143-151.
2	LI Y, WANG J C, WU H R, et al. Detection of powdery mildew on strawberry leaves based on DAC-YOLOv4 model[J]. Computers and electronics in agriculture, 2022, 202: ID 107418.
3	LI G Q, JIAO L, CHEN P, et al. Spatial convolutional self-attention-based transformer module for strawberry disease identification under complex background[J]. Computers and electronics in agriculture, 2023, 212: ID 108121.
4	XIE C Q, HE Y. Spectrum and image texture features analysis for early blight disease detection on eggplant leaves[J]. Sensors, 2016, 16(5): ID 676.
5	DWIVEDI P, KUMAR S, VIJH S, et al. Study of machine learning techniques for plant disease recognition in agriculture[C]// 2021 11th International Conference on Cloud Computing, Data Science & Amp; Engineering (Confluence). Piscataway, New Jersey, USA: IEEE, 2021: 752-756.
6	KHAN M A, AKRAM T, SHARIF M, et al. An automated system for cucumber leaf diseased spot detection and classification using improved saliency method and deep features selection[J]. Multimedia tools and applications, 2020, 79(25): 18627-18656.
7	杜甜甜, 南新元, 黄家興, 等. 改进RegNet识别多种农作物病害受害程度[J]. 农业工程学报, 2022, 38(15): 150-158.
	DU T T, NAN X Y, HUANG J X, et al. Identifying the damage degree of various crop diseases using an improved RegNet[J]. Transactions of the Chinese society of agricultural engineering, 2022, 38(15): 150-158.
8	胡晓波, 许桃胜, 黄伟, 等. 交互式双分支特征融合的草莓病害程度快速诊断方法[J]. 农业机械学报, 2023, 54(11): 225-235.
	HU X B, XU T S, HUANG W, et al. Interactive bilateral feature fusion network for real-time strawberry disease diagnosis[J]. Transactions of the Chinese society for agricultural machinery, 2023, 54(11): 225-235.
9	马盼, 杨子恒, 万虎, 等. 基于YOLOv8网络的棉蚜图像识别算法及软件系统设计[J]. 智能化农业装备学报(中英文), 2023(3): 42-49.
	MA P, YANG Z H, WAN H, et al. A new cotton aphid image recognition algorithm and software based on YOLOv8[J]. Journal of intelligent agricultural mechanization, 2023(3): 42-49.
10	NGUYEN H T, TRAN T D, NGUYEN T T, et al. Strawberry disease identification with vision transformer-based models[J]. Multimedia tools and applications, 2024, 83(29): 73101-73126.
11	WANG J P, LI Z Y, GAO G H, et al. BerryNet-lite: A lightweight convolutional neural network for strawberry disease identification[J]. Agriculture, 2024, 14(5): ID 665.
12	MOHANTY S P, HUGHES D P, SALATHÉ M. Using deep learning for image-based plant disease detection[J]. Frontiers in plant science, 2016, 7: ID 1419.
13	WANG J D, LAN C L, LIU C, et al. Generalizing to unseen domains: A survey on domain generalization[J]. IEEE transactions on knowledge and data engineering, 2023, 35(8): 8052-8072.
14	GILLES BLANCHARD G L, CLAYTON SCOTT. Generalizing from several related classification tasks to a new unlabeled sample[C]// Advances in neural information processing systems 24: 25th Annual Conference on Neural Information Processing Systems 2011. San Francisco, USA: Curran Associates Inc, 2011.
15	THOMAS GRUBINGER A B, HOLGER SCHONER, THOMAS NATSCHLAGER, HESKESTOM. Domain generalization based on transfer component analysis[C]// International Work-Conference on Artificial Neural Networks. Cham, Germany: Springer, 2015: 325-334.
16	TZENG E, HOFFMAN J, ZHANG N, et al. Deep domain confusion: Maximizing for domain invariance[EB/OL]. arXiv: 1412.3474, 2014.
17	GANG HUA H J G. Deep CORAL: Correlation Alignment forDeep Domain Adaptation[C]// European Conference on Computer Vision. Cham, Germany: Springer, 2016: 443-450.
18	ULYANOV D, VEDALDI A, LEMPITSKY V S. Instance normalization: The missing ingredient for fast stylization[EB/OL]. arXiv: 1607.08022, 2017.
19	PAN X G, LUO P, SHI J P, et al. Two at once: Enhancing learning and generalization capacities via IBN-net[C]// Computer Vision-ECCV 2018. Cham, Germany: Springer, 2018: 484-500.
20	PAN X G, ZHAN X H, SHI J P, et al. Switchable whitening for deep representation learning[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, New Jersey, USA: IEEE, 2019: 1863-1871.
21	CHOI S, JUNG S, YUN H, et al. RobustNet: Improving domain generalization in urban-scene segmentation via instance selective whitening[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New Jersey, USA: IEEE, 2021: 11575-11585.
22	JIN X, LAN C L, ZENG W J, et al. Style normalization and restitution for domain generalization and adaptation[J]. IEEE transactions on multimedia, 2022, 24: 3636-3651.
23	YIJUN LI C F, JIMEI YANG, ZHAOWEN WANG, XIN LU, MING HSUAN YANG. Universal style transfer via feature transforms[C]// Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017. San Francisco, USA: Curran Associates Inc, 2017.
24	HUANG L, ZHOU Y, ZHU F, et al. Iterative normalization: Beyond standardization towards efficient whitening[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New Jersey, USA: IEEE, 2019: 4869-4878.
25	AFZAAL U, BHATTARAI B, PANDEYA Y R, et al. An instance segmentation model for strawberry diseases based on mask R-CNN[J]. Sensors, 2021, 21(19): ID 6565.
26	BELGHAZI M I, BARATIN A, RAJESHWAR S, et al. Mutual information neural estimation[C]// Proceedings of the 35th International Conference on Machine Learning. New York, USA: PMLR. 2018: 531-540.
27	DENG J, DONG W, SOCHER R, et al. ImageNet: A large-scale hierarchical image database[C]// 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New Jersey, USA: IEEE, 2009: 248-255.

类别标签	病害类别	DGSR1数量/幅	DGSR2数量/幅	合计/幅
0	细菌性叶斑病	509	435	944
1	炭疽病（果实）	121	97	218
2	灰霉病（果实）	396	477	873
3	蛇眼病	312	615	927
4	白粉病（果实）	145	135	280
5	白粉病（叶子）	407	533	940
共计	6类	1 890	2 292	4 182

模型	参数量/ M	计算量（FLOPs）/G
ResNet-18	11.18	1.82
ResNet-18-IWR	14.04	7.93
ResNet-50	23.52	4.13
ResNet-50-IWR	30.07	16.02

基于实例白化与特征恢复的草莓病害识别领域泛化方法

Domain Generalization Method of Strawberry Disease Recognition Based on Instance Whitening and Restitution

在线阅读

知网下载

本地下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 16

参考文献 27

相关文章 1

编辑推荐

Metrics

本文评价