基于深度学习与特征可视化方法的草地贪夜蛾及其近缘种成虫识别

doi:10.12133/j.smartag.2020.2.3.202008-SA001

摘要/Abstract

摘要：

草地贪夜蛾是对粮食安全具有巨大威胁的害虫，早发现、早防治对虫情控制具有重要意义。目前，利用深度学习方法进行草地贪夜蛾及其近缘种成虫识别的相关研究存在数据量严重偏小的情况，有可能造成模型未能真正学习到草地贪夜蛾及其近缘种成虫的环形纹、肾形纹等关键视觉特征。针对上述问题，本研究在建立包含草地贪夜蛾在内的7种夜蛾科成虫，10,177幅图像组成的数据库基础上，采用迁移学习方式建立了VGG-16、ResNet-50和DenseNet-121，3种夜蛾成虫识别深度学习模型，并用相同的测试集测试了所有模型。结果表明，构建的模型识别准确率均超过了98%。此外，本研究用特征可视化技术展现了模型习得的特征，并验证了这些特征和专家进行人工识别的关键视觉特征的一致性——ResNet-50和DenseNet-121的平均特征识别率在85%左右，进一步支持了用深度学习进行草地贪夜蛾成虫实时识别的可行性。研究发现，不同模型对夜蛾科成虫视觉特征的学习能力不一样，在评价模型时不能仅看识别率，还需要加入视觉特征识别率指标对模型的学习内容进行评价。本研究通过试验证明可视化分析可以直观认识模型的特征学习情况，可为行业内或其他领域的研究人员提供参考。

关键词: 草地贪夜蛾, 夜蛾, 成虫识别, 深度学习, 视觉特征, 特征可视化, 迁移学习

Abstract:

Invasive pest fall armyworm (FAW) Spodoptera frugiperda is one of the serious threats to the food safety. Early warning and control plays a key role in FAW management. Nowadays, deep learning technology has been applied to recognize the image of FAW. However, there is a serious lack of training dataset in the current researches, which may mislead the model to learn features unrelated to the key visual characteristics (ring pattern, reniform pattern, etc.) of FAW adults and its close related species. Therefore, this research established a database of 10,177 images belonging to 7 species of noctuid adults, including FAW and 6 FAW close related species. Based on the small-scale dataset, transfer learning was used to build the recognition model of FAW adults by employing three deep learning models (VGG-16, ResNet-50 and DenseNet-121) pretrained on ImageNet. All of the models got more than 98% recognition accuracy on the same testing dataset. Moreover, by using feature visualization techniques, this research visualized the features learned by deep learning models and compared them to the related key visual characteristics recognized by human experts. The results showed that there was a high consistency between the two counterparts, i.e., the average feature recognition rate of ResNet-50 and DenseNet-121 was around 85%, which further demonstrated that it was possible to use the deep learning technology for the real-time monitoring of FAW adults. In addition, this study also found that the learning abilities of key visual characteristics among different models were different even though they have similar recognition accuracy. Herein, we suggest that when evaluating the model capacity, we should not only focus on the recognition rate, the ability of learning individual visual characteristics should be allocated importance for evaluating the model performance. For those important taxonomical traits, if the visualization results indicated that the model didn't learnt them, we should then modify our datasets or adjusting the training strategies to increase the learning ability. In conclusion, this study verified that visualizing the features learnt by the model is a good way to evaluate the learning ability of deep learning models, and to provide a possible way for other researchers in the field who want to understand the features learnt by deep learning models.

Key words: Spodoptera frugiperda, noctuid, adult moth recognition, deep learning, visual characteristics, feature visualization, transfer learning

中图分类号:

S431.9

魏靖, 王玉亭, 袁会珠, 张梦蕾, 王振营. 基于深度学习与特征可视化方法的草地贪夜蛾及其近缘种成虫识别[J]. 智慧农业（中英文）, 2020, 2(3): 75-85.

WEI Jing, WANG Yuting, YUAN Huizhu, ZHANG Menglei, WANG Zhenying. Identification and Morphological Analysis of Adult Spodoptera Frugiperda and Its Close Related Species Using Deep Learning[J]. Smart Agriculture, 2020, 2(3): 75-85.

参考文献

1	吴孔明. 中国草地贪夜蛾的防控策略[J]. 植物保护, 2020, 46(2): 1-5.
1	WU K. Management strategies of fall armyworm (Spodoptera frugiperda) in China[J]. Plant Protection, 2020, 46(2): 1-5.
2	张磊, 柳贝, 姜玉英, 等. 中国不同地区草地贪夜蛾种群生物型分子特征分析[J]. 植物保护, 2019, 45(4): 20-27.
2	ZHANG L, LIU B, JIANG Y, et al. Molecular characterization analysis of fall armyworm populations in China[J]. Plant Protection, 2019, 45(4): 20-27.
3	赵胜园, 罗倩明, 孙小旭, 等. 草地贪夜蛾与斜纹夜蛾的形态特征和生物学习性比较[J]. 中国植保导刊, 2019, 39(5): 26-35.
3	ZHAO S, LUO Q, SUN X, et al. Comparison of morphological and biological characteristics between Spodoptera frugiperda and Spodoptera litura[J]. China Plant Protection, 2019, 39(5): 26-35.
4	MOHANTY S P, HUGHES D P, SALATHé M. Using deep learning for image-based plant disease detection[J]. Frontiers in Plant Science, 2016, 7(1419): 1-10.
5	CHIWAMBA S H, PHIRI J, NKUNIKA P O Y, et al. An application of machine learning algorithms in automated identification and capturing of fall armyworm (FAW) moths in the field[C]// 2018 Ictsz International Conference In Icts (ICICT). Lusaka, Zambia: ICICT. 2019: 119-124.
6	CHULU F, PHIRI J, NKUNIKA P, et al. A convolutional neural network for automatic identification and classification of fall army worm moth[J]. International Journal of Advanced Computer Science and Applications, 2019, 10(7): 112-118.
7	于业达, 顾偌铖, 唐运林, 等. 基于深度学习的草地贪夜蛾自动识别 [J]. 西南大学学报(自然科学版), 2019, 41(9): 24-31.
7	YU Y, GU R, TANG Y, et al. A CNN-based automatic identification system for Spodoptera frugiperda[J]. Journal of Southwest University (Natural Science), 2019, 41(9): 24-31.
8	ESTEVA A, KUPREL B, NOVOA R A, et al. Dermatologist-level classification of skin cancer with deep neural networks [J]. Nature, 2017, 542(7639): 115-118.
9	SPRINGENBERG J T, DOSOVITSKIY A, BROX T, et al. Striving for simplicity: The all convolutional net[C]// 2015 International Conference on Learning Representations (ICLR). San Diego, CA, USA: ICLR. 2015: 1-14.
10	SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization [C]// 2017 IEEE International Conference on Computer Vision (ICCV). Piscataway, New York, USA: IEEE. 2017: 618-626.
11	CHATTOPADHYAY A, SARKAR A, HOWLADER P, et al. Grad-CAM++: Generalized gradient-based visual explanations for deep convolutional networks[C]// 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway, New York, USA: IEEE. 2018: 839-847,
12	朱弘复, 陈一心. 中国经济昆虫志[M]. 北京: 科学出版社, 1963: 23-23.
12	ZHU H, CHEN Y. Economic insects of China[M]. Beijing: China Science Publishing & Media Ltd., 1963: 23-23.
13	洪晓月, 丁锦华. 农业昆虫学(第二版)[M]. 北京: 中国农业出版社, 2007: 46-235.
13	HONG X, DING J. Agricultural entomology (2nd ed)[M]. Beijing: China Agriculture Press Co., Ltd., 2007: 46-235.
14	孔德英, 孙涛, 滕少娜, 等. 草地贪夜蛾及其近似种的鉴定[J]. 植物检疫, 2019, 33(4): 37-40.
14	KONG D, SUN T, TENG S, et al. Identification of fall armyworm, Spodoptera frugiperda, and its similar species in morphology [J]. Plant Quarantine, 2019, 33(4): 37-40.
15	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]// 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), Piscataway. New York, USA: IEEE. 2015, 730-734.
16	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2016: 770-778.
17	HUANG G, LIU Z, MAATEN L V D, et al. Densely connected convolutional networks[C]// 2017 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2017: 2261-2269.
18	DENG J, DONG W, SOCHER R, et al. Imagenet: A large-scale hierarchical image database[C]// 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2009: 248-255.
19	LOSHCHILOV I, HUTTER F. SGDR: Stochastic gradient descent with warm restarts[C]// 2017 International Conference on Learning Representations (ICLR). Toulon, France: ICLR. 2017: 1-16.
20	GOTMARE A, KESKAR N S, XIONG C, et al. A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation[C]// 2019 International Conference on Learning Representations (ICLR). New Orleans, LA, USA: ICLR. 2019: 1-16.
21	SMITH L N. Cyclical learning rates for training neural networks[C]// 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway, New York, USA: IEEE. 2015: 464-472.
22	ZHOU B, KHOSLA A, LAPEDRIZA, et al. Learning deep features for discriminative localization[C]// 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2016: 2921-2929.
23	PARK J, KIM D I, CHOI B, et al. Classification and morphological analysis of vector mosquitoes using deep convolutional neural networks[J]. Scientific Reports, 2020, 10(1): 1-12.
24	LI K, WU Z, PENG K, et al. Tell me where to look: guided attention inference network[C]// 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2018:9215-9223.

[1]	毛克彪, 张晨阳, 施建成, 王旭明, 郭中华, 李春树, 董立新, 吴门新, 孙瑞静, 武胜利, 姬大彬, 蒋玲梅, 赵天杰, 邱玉宝, 杜永明, 徐同仁. 基于人工智能的地球物理参数反演范式理论及判定条件[J]. 智慧农业(中英文), 2023, 5(2): 161-171.
[2]	潘晨露, 张正华, 桂文豪, 马家俊, 严晨曦, 张晓敏. 融合ECA机制与DenseNet201的水稻病虫害识别方法[J]. 智慧农业(中英文), 2023, 5(2): 45-55.
[3]	朱海鹏, 张玉安, 李欢欢, 王建文, 杨英魁, 宋仁德. 基于改进残差网络模型的不同部位牦牛肉分类识别方法[J]. 智慧农业(中英文), 2023, 5(2): 115-125.
[4]	夏雪, 柴秀娟, 张凝, 周硕, 孙琦鑫, 孙坦. 用于边缘计算设备的果树挂果量轻量化估测模型[J]. 智慧农业(中英文), 2023, 5(2): 1-12.
[5]	王亚鹏, 曹姗姗, 李全胜, 孙伟. 融合迁移学习和集成学习的自然背景下荒漠植物识别方法[J]. 智慧农业(中英文), 2023, 5(2): 93-103.
[6]	郭阳阳, 杜书增, 乔永亮, 梁栋. 深度学习在家畜智慧养殖中研究应用进展[J]. 智慧农业(中英文), 2023, 5(1): 52-65.
[7]	左敏, 胡天宇, 董微, 张可心, 张青川. 基于Informer神经网络的农产品物流需求预测分析——以华中地区为例[J]. 智慧农业(中英文), 2023, 5(1): 34-43.
[8]	张文景, 蒋泽中, 秦立峰. 基于弱监督下改进的CBAM-ResNet18模型识别苹果多种叶部病害[J]. 智慧农业(中英文), 2023, 5(1): 111-121.
[9]	计洁, 金洲, 王儒敬, 刘海燕, 李志远. 基于递进式卷积网络的农业命名实体识别方法[J]. 智慧农业(中英文), 2023, 5(1): 122-131.
[10]	胡松涛, 翟瑞芳, 王应华, 刘志, 朱剑忠, 任荷, 杨万能, 宋鹏. 基于多源数据的马铃薯植株表型参数提取[J]. 智慧农业(中英文), 2023, 5(1): 132-145.
[11]	刘晓航, 张昭, 刘嘉滢, 张漫, 李寒, FLORES Paulo, 韩雄哲. 基于多种深度学习算法的田间玉米籽粒检测与计数[J]. 智慧农业(中英文), 2022, 4(4): 49-60.
[12]	许钰林, 康孟珍, 王秀娟, 华净, 王浩宇, 沈震. 基于深度学习的玉米和大豆期货价格智能预测[J]. 智慧农业(中英文), 2022, 4(4): 156-163.
[13]	罗庆, 饶元, 金秀, 江朝晖, 王坦, 王丰仪, 张武. 基于改进YOLOv5s和多模态图像的树上毛桃检测[J]. 智慧农业(中英文), 2022, 4(4): 84-104.
[14]	张志博, 赵西宁, 高晓东, 张利, 杨孟豪. 基于改进Linknet网络的黄土高原苹果园精准提取[J]. 智慧农业(中英文), 2022, 4(3): 95-107.
[15]	商枫楠, 周学成, 梁英凯, 肖明玮, 陈桥, 罗陈迪. 基于改进YOLOX的自然环境中火龙果检测方法[J]. 智慧农业(中英文), 2022, 4(3): 120-131.