Identification and Morphological Analysis of Adult Spodoptera Frugiperda and Its Close Related Species Using Deep Learning

doi:10.12133/j.smartag.2020.2.3.202008-SA001

Abstract

Abstract:

Invasive pest fall armyworm (FAW) Spodoptera frugiperda is one of the serious threats to the food safety. Early warning and control plays a key role in FAW management. Nowadays, deep learning technology has been applied to recognize the image of FAW. However, there is a serious lack of training dataset in the current researches, which may mislead the model to learn features unrelated to the key visual characteristics (ring pattern, reniform pattern, etc.) of FAW adults and its close related species. Therefore, this research established a database of 10,177 images belonging to 7 species of noctuid adults, including FAW and 6 FAW close related species. Based on the small-scale dataset, transfer learning was used to build the recognition model of FAW adults by employing three deep learning models (VGG-16, ResNet-50 and DenseNet-121) pretrained on ImageNet. All of the models got more than 98% recognition accuracy on the same testing dataset. Moreover, by using feature visualization techniques, this research visualized the features learned by deep learning models and compared them to the related key visual characteristics recognized by human experts. The results showed that there was a high consistency between the two counterparts, i.e., the average feature recognition rate of ResNet-50 and DenseNet-121 was around 85%, which further demonstrated that it was possible to use the deep learning technology for the real-time monitoring of FAW adults. In addition, this study also found that the learning abilities of key visual characteristics among different models were different even though they have similar recognition accuracy. Herein, we suggest that when evaluating the model capacity, we should not only focus on the recognition rate, the ability of learning individual visual characteristics should be allocated importance for evaluating the model performance. For those important taxonomical traits, if the visualization results indicated that the model didn't learnt them, we should then modify our datasets or adjusting the training strategies to increase the learning ability. In conclusion, this study verified that visualizing the features learnt by the model is a good way to evaluate the learning ability of deep learning models, and to provide a possible way for other researchers in the field who want to understand the features learnt by deep learning models.

Key words: Spodoptera frugiperda, noctuid, adult moth recognition, deep learning, visual characteristics, feature visualization, transfer learning

CLC Number:

S431.9

WEI Jing, WANG Yuting, YUAN Huizhu, ZHANG Menglei, WANG Zhenying. Identification and Morphological Analysis of Adult Spodoptera Frugiperda and Its Close Related Species Using Deep Learning[J]. Smart Agriculture, 2020, 2(3): 75-85.

References

1	吴孔明. 中国草地贪夜蛾的防控策略[J]. 植物保护, 2020, 46(2): 1-5.
1	WU K. Management strategies of fall armyworm (Spodoptera frugiperda) in China[J]. Plant Protection, 2020, 46(2): 1-5.
2	张磊, 柳贝, 姜玉英, 等. 中国不同地区草地贪夜蛾种群生物型分子特征分析[J]. 植物保护, 2019, 45(4): 20-27.
2	ZHANG L, LIU B, JIANG Y, et al. Molecular characterization analysis of fall armyworm populations in China[J]. Plant Protection, 2019, 45(4): 20-27.
3	赵胜园, 罗倩明, 孙小旭, 等. 草地贪夜蛾与斜纹夜蛾的形态特征和生物学习性比较[J]. 中国植保导刊, 2019, 39(5): 26-35.
3	ZHAO S, LUO Q, SUN X, et al. Comparison of morphological and biological characteristics between Spodoptera frugiperda and Spodoptera litura[J]. China Plant Protection, 2019, 39(5): 26-35.
4	MOHANTY S P, HUGHES D P, SALATHé M. Using deep learning for image-based plant disease detection[J]. Frontiers in Plant Science, 2016, 7(1419): 1-10.
5	CHIWAMBA S H, PHIRI J, NKUNIKA P O Y, et al. An application of machine learning algorithms in automated identification and capturing of fall armyworm (FAW) moths in the field[C]// 2018 Ictsz International Conference In Icts (ICICT). Lusaka, Zambia: ICICT. 2019: 119-124.
6	CHULU F, PHIRI J, NKUNIKA P, et al. A convolutional neural network for automatic identification and classification of fall army worm moth[J]. International Journal of Advanced Computer Science and Applications, 2019, 10(7): 112-118.
7	于业达, 顾偌铖, 唐运林, 等. 基于深度学习的草地贪夜蛾自动识别 [J]. 西南大学学报(自然科学版), 2019, 41(9): 24-31.
7	YU Y, GU R, TANG Y, et al. A CNN-based automatic identification system for Spodoptera frugiperda[J]. Journal of Southwest University (Natural Science), 2019, 41(9): 24-31.
8	ESTEVA A, KUPREL B, NOVOA R A, et al. Dermatologist-level classification of skin cancer with deep neural networks [J]. Nature, 2017, 542(7639): 115-118.
9	SPRINGENBERG J T, DOSOVITSKIY A, BROX T, et al. Striving for simplicity: The all convolutional net[C]// 2015 International Conference on Learning Representations (ICLR). San Diego, CA, USA: ICLR. 2015: 1-14.
10	SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization [C]// 2017 IEEE International Conference on Computer Vision (ICCV). Piscataway, New York, USA: IEEE. 2017: 618-626.
11	CHATTOPADHYAY A, SARKAR A, HOWLADER P, et al. Grad-CAM++: Generalized gradient-based visual explanations for deep convolutional networks[C]// 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway, New York, USA: IEEE. 2018: 839-847,
12	朱弘复, 陈一心. 中国经济昆虫志[M]. 北京: 科学出版社, 1963: 23-23.
12	ZHU H, CHEN Y. Economic insects of China[M]. Beijing: China Science Publishing & Media Ltd., 1963: 23-23.
13	洪晓月, 丁锦华. 农业昆虫学(第二版)[M]. 北京: 中国农业出版社, 2007: 46-235.
13	HONG X, DING J. Agricultural entomology (2nd ed)[M]. Beijing: China Agriculture Press Co., Ltd., 2007: 46-235.
14	孔德英, 孙涛, 滕少娜, 等. 草地贪夜蛾及其近似种的鉴定[J]. 植物检疫, 2019, 33(4): 37-40.
14	KONG D, SUN T, TENG S, et al. Identification of fall armyworm, Spodoptera frugiperda, and its similar species in morphology [J]. Plant Quarantine, 2019, 33(4): 37-40.
15	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]// 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), Piscataway. New York, USA: IEEE. 2015, 730-734.
16	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2016: 770-778.
17	HUANG G, LIU Z, MAATEN L V D, et al. Densely connected convolutional networks[C]// 2017 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2017: 2261-2269.
18	DENG J, DONG W, SOCHER R, et al. Imagenet: A large-scale hierarchical image database[C]// 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2009: 248-255.
19	LOSHCHILOV I, HUTTER F. SGDR: Stochastic gradient descent with warm restarts[C]// 2017 International Conference on Learning Representations (ICLR). Toulon, France: ICLR. 2017: 1-16.
20	GOTMARE A, KESKAR N S, XIONG C, et al. A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation[C]// 2019 International Conference on Learning Representations (ICLR). New Orleans, LA, USA: ICLR. 2019: 1-16.
21	SMITH L N. Cyclical learning rates for training neural networks[C]// 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway, New York, USA: IEEE. 2015: 464-472.
22	ZHOU B, KHOSLA A, LAPEDRIZA, et al. Learning deep features for discriminative localization[C]// 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2016: 2921-2929.
23	PARK J, KIM D I, CHOI B, et al. Classification and morphological analysis of vector mosquitoes using deep convolutional neural networks[J]. Scientific Reports, 2020, 10(1): 1-12.
24	LI K, WU Z, PENG K, et al. Tell me where to look: guided attention inference network[C]// 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2018:9215-9223.

[1]	HU Lingyan, GUO Ruiya, GUO Zhanjun, XU Guohui, GAI Rongli, WANG Zumin, ZHANG Yumeng, JU Bowen, NIE Xiaoyu. U-Net Greenhouse Sweet Cherry Image Segmentation Method Integrating PDE Plant Temporal Image Contrastive Learning and GCN Skip Connections [J]. Smart Agriculture, 2025, 7(3): 131-142.
[2]	LI Ruijie, WANG Aidong, WU Huaxing, LI Ziqiu, FENG Xiangqian, HONG Weiyuan, TANG Xuejun, QIN Jinhua, WANG Danying, CHU Guang, ZHANG Yunbo, CHEN Song. Remote Sensing for Rice Growth Stages Monitoring: Research Progress, Bottleneck Problems and Technical Optimization Paths [J]. Smart Agriculture, 2025, 7(3): 89-107.
[3]	HAN Yu, QI Kangkang, ZHENG Jiye, LI Jinai, JIANG Fugui, ZHANG Xianglun, YOU Wei, ZHANG Xia. Lightweight Cattle Facial Recognition Method Based on Improved YOLOv11 [J]. Smart Agriculture, 2025, 7(3): 173-184.
[4]	MA Liu, MAO Kebiao, GUO Zhonghua. Defogging Remote Sensing Images Method Based on a Hybrid Attention-Based Generative Adversarial Network [J]. Smart Agriculture, 2025, 7(2): 172-182.
[5]	XU Shiwei, LI Qianchuan, LUAN Rupeng, ZHUANG Jiayu, LIU Jiajia, XIONG Lu. Agricultural Market Monitoring and Early Warning: An Integrated Forecasting Approach Based on Deep Learning [J]. Smart Agriculture, 2025, 7(1): 57-69.
[6]	JIN Ning, GUO Yufeng, HAN Xiaodong, MIAO Yisheng, WU Huarui. Method for Calculating Semantic Similarity of Short Agricultural Texts Based on Transfer Learning [J]. Smart Agriculture, 2025, 7(1): 33-43.
[7]	GONG Yu, WANG Ling, ZHAO Rongqiang, YOU Haibo, ZHOU Mo, LIU Jie. Tomato Growth Height Prediction Method by Phenotypic Feature Extraction Using Multi-modal Data [J]. Smart Agriculture, 2025, 7(1): 97-110.
[8]	QI Zijun, NIU Dangdang, WU Huarui, ZHANG Lilin, WANG Lunfeng, ZHANG Hongming. Chinese Kiwifruit Text Named Entity Recognition Method Based on Dual-Dimensional Information and Pruning [J]. Smart Agriculture, 2025, 7(1): 44-56.
[9]	ZHANG Hui, HU Jun, SHI Hang, LIU Changxi, WU Miao. Precision Target Spraying System Integrated with Remote Deep Learning Recognition Model for Cabbage Plant Centers [J]. Smart Agriculture, 2024, 6(6): 85-95.
[10]	LU Bibo, LIANG Di, YANG Jie, SONG Aiqing, HUANGFU Shangwei. Image Segmentation Method of Chinese Yam Leaves in Complex Background Based on Improved ENet [J]. Smart Agriculture, 2024, 6(6): 109-120.
[11]	YAN Congkuan, ZHU Dequan, MENG Fankai, YANG Yuqing, TANG Qixing, ZHANG Aifang, LIAO Juan. Rice Leaf Disease Image Enhancement Based on Improved CycleGAN [J]. Smart Agriculture, 2024, 6(6): 96-108.
[12]	LUO Youlu, PAN Yonghao, XIA Shunxing, TAO Youzhi. Lightweight Apple Leaf Disease Detection Algorithm Based on Improved YOLOv8 [J]. Smart Agriculture, 2024, 6(5): 128-138.
[13]	LIU Yi, ZHANG Yanjun. ReluformerN: Lightweight High-Low Frequency Enhanced for Hyperspectral Agricultural Lancover Classification [J]. Smart Agriculture, 2024, 6(5): 74-87.
[14]	NIAN Yue, ZHAO Kaixuan, JI Jiangtao. Cow Hoof Slippage Detecting Method Based on Enhanced DeepLabCut Model [J]. Smart Agriculture, 2024, 6(5): 153-163.
[15]	ZHANG Yanqi, ZHOU Shuo, ZHANG Ning, CHAI Xiujuan, SUN Tan. A Regional Farming Pig Counting System Based on Improved Instance Segmentation Algorithm [J]. Smart Agriculture, 2024, 6(4): 53-63.