Identification and Morphological Analysis of Adult Spodoptera Frugiperda and Its Close Related Species Using Deep Learning

doi:10.12133/j.smartag.2020.2.3.202008-SA001

Abstract

Abstract:

Invasive pest fall armyworm (FAW) Spodoptera frugiperda is one of the serious threats to the food safety. Early warning and control plays a key role in FAW management. Nowadays, deep learning technology has been applied to recognize the image of FAW. However, there is a serious lack of training dataset in the current researches, which may mislead the model to learn features unrelated to the key visual characteristics (ring pattern, reniform pattern, etc.) of FAW adults and its close related species. Therefore, this research established a database of 10,177 images belonging to 7 species of noctuid adults, including FAW and 6 FAW close related species. Based on the small-scale dataset, transfer learning was used to build the recognition model of FAW adults by employing three deep learning models (VGG-16, ResNet-50 and DenseNet-121) pretrained on ImageNet. All of the models got more than 98% recognition accuracy on the same testing dataset. Moreover, by using feature visualization techniques, this research visualized the features learned by deep learning models and compared them to the related key visual characteristics recognized by human experts. The results showed that there was a high consistency between the two counterparts, i.e., the average feature recognition rate of ResNet-50 and DenseNet-121 was around 85%, which further demonstrated that it was possible to use the deep learning technology for the real-time monitoring of FAW adults. In addition, this study also found that the learning abilities of key visual characteristics among different models were different even though they have similar recognition accuracy. Herein, we suggest that when evaluating the model capacity, we should not only focus on the recognition rate, the ability of learning individual visual characteristics should be allocated importance for evaluating the model performance. For those important taxonomical traits, if the visualization results indicated that the model didn't learnt them, we should then modify our datasets or adjusting the training strategies to increase the learning ability. In conclusion, this study verified that visualizing the features learnt by the model is a good way to evaluate the learning ability of deep learning models, and to provide a possible way for other researchers in the field who want to understand the features learnt by deep learning models.

Key words: Spodoptera frugiperda, noctuid, adult moth recognition, deep learning, visual characteristics, feature visualization, transfer learning

CLC Number:

S431.9

WEI Jing, WANG Yuting, YUAN Huizhu, ZHANG Menglei, WANG Zhenying. Identification and Morphological Analysis of Adult Spodoptera Frugiperda and Its Close Related Species Using Deep Learning[J]. Smart Agriculture, 2020, 2(3): 75-85.

References

1	吴孔明. 中国草地贪夜蛾的防控策略[J]. 植物保护, 2020, 46(2): 1-5.
1	WU K. Management strategies of fall armyworm (Spodoptera frugiperda) in China[J]. Plant Protection, 2020, 46(2): 1-5.
2	张磊, 柳贝, 姜玉英, 等. 中国不同地区草地贪夜蛾种群生物型分子特征分析[J]. 植物保护, 2019, 45(4): 20-27.
2	ZHANG L, LIU B, JIANG Y, et al. Molecular characterization analysis of fall armyworm populations in China[J]. Plant Protection, 2019, 45(4): 20-27.
3	赵胜园, 罗倩明, 孙小旭, 等. 草地贪夜蛾与斜纹夜蛾的形态特征和生物学习性比较[J]. 中国植保导刊, 2019, 39(5): 26-35.
3	ZHAO S, LUO Q, SUN X, et al. Comparison of morphological and biological characteristics between Spodoptera frugiperda and Spodoptera litura[J]. China Plant Protection, 2019, 39(5): 26-35.
4	MOHANTY S P, HUGHES D P, SALATHé M. Using deep learning for image-based plant disease detection[J]. Frontiers in Plant Science, 2016, 7(1419): 1-10.
5	CHIWAMBA S H, PHIRI J, NKUNIKA P O Y, et al. An application of machine learning algorithms in automated identification and capturing of fall armyworm (FAW) moths in the field[C]// 2018 Ictsz International Conference In Icts (ICICT). Lusaka, Zambia: ICICT. 2019: 119-124.
6	CHULU F, PHIRI J, NKUNIKA P, et al. A convolutional neural network for automatic identification and classification of fall army worm moth[J]. International Journal of Advanced Computer Science and Applications, 2019, 10(7): 112-118.
7	于业达, 顾偌铖, 唐运林, 等. 基于深度学习的草地贪夜蛾自动识别 [J]. 西南大学学报(自然科学版), 2019, 41(9): 24-31.
7	YU Y, GU R, TANG Y, et al. A CNN-based automatic identification system for Spodoptera frugiperda[J]. Journal of Southwest University (Natural Science), 2019, 41(9): 24-31.
8	ESTEVA A, KUPREL B, NOVOA R A, et al. Dermatologist-level classification of skin cancer with deep neural networks [J]. Nature, 2017, 542(7639): 115-118.
9	SPRINGENBERG J T, DOSOVITSKIY A, BROX T, et al. Striving for simplicity: The all convolutional net[C]// 2015 International Conference on Learning Representations (ICLR). San Diego, CA, USA: ICLR. 2015: 1-14.
10	SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization [C]// 2017 IEEE International Conference on Computer Vision (ICCV). Piscataway, New York, USA: IEEE. 2017: 618-626.
11	CHATTOPADHYAY A, SARKAR A, HOWLADER P, et al. Grad-CAM++: Generalized gradient-based visual explanations for deep convolutional networks[C]// 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway, New York, USA: IEEE. 2018: 839-847,
12	朱弘复, 陈一心. 中国经济昆虫志[M]. 北京: 科学出版社, 1963: 23-23.
12	ZHU H, CHEN Y. Economic insects of China[M]. Beijing: China Science Publishing & Media Ltd., 1963: 23-23.
13	洪晓月, 丁锦华. 农业昆虫学(第二版)[M]. 北京: 中国农业出版社, 2007: 46-235.
13	HONG X, DING J. Agricultural entomology (2nd ed)[M]. Beijing: China Agriculture Press Co., Ltd., 2007: 46-235.
14	孔德英, 孙涛, 滕少娜, 等. 草地贪夜蛾及其近似种的鉴定[J]. 植物检疫, 2019, 33(4): 37-40.
14	KONG D, SUN T, TENG S, et al. Identification of fall armyworm, Spodoptera frugiperda, and its similar species in morphology [J]. Plant Quarantine, 2019, 33(4): 37-40.
15	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]// 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), Piscataway. New York, USA: IEEE. 2015, 730-734.
16	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2016: 770-778.
17	HUANG G, LIU Z, MAATEN L V D, et al. Densely connected convolutional networks[C]// 2017 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2017: 2261-2269.
18	DENG J, DONG W, SOCHER R, et al. Imagenet: A large-scale hierarchical image database[C]// 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2009: 248-255.
19	LOSHCHILOV I, HUTTER F. SGDR: Stochastic gradient descent with warm restarts[C]// 2017 International Conference on Learning Representations (ICLR). Toulon, France: ICLR. 2017: 1-16.
20	GOTMARE A, KESKAR N S, XIONG C, et al. A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation[C]// 2019 International Conference on Learning Representations (ICLR). New Orleans, LA, USA: ICLR. 2019: 1-16.
21	SMITH L N. Cyclical learning rates for training neural networks[C]// 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway, New York, USA: IEEE. 2015: 464-472.
22	ZHOU B, KHOSLA A, LAPEDRIZA, et al. Learning deep features for discriminative localization[C]// 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2016: 2921-2929.
23	PARK J, KIM D I, CHOI B, et al. Classification and morphological analysis of vector mosquitoes using deep convolutional neural networks[J]. Scientific Reports, 2020, 10(1): 1-12.
24	LI K, WU Z, PENG K, et al. Tell me where to look: guided attention inference network[C]// 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2018:9215-9223.

[1]	LI Hao, DU Yuqiu, XIAO Xingzhu, CHEN Yanxi. Remote Sensing Identification Method of Cultivated Land at Hill County of Sichuan Basin Based on Deep Learning [J]. Smart Agriculture, 2024, 6(3): 34-45.
[2]	NIE Ganggang, RAO Honghui, LI Zefeng, LIU Muhua. Severity Grading Model for Camellia Oleifera Anthracnose Infection Based on Improved YOLACT [J]. Smart Agriculture, 2024, 6(3): 138-147.
[3]	ZHANG Jing, ZHAO Zexuan, ZHAO Yanru, BU Hongchao, WU Xingyu. Oilseed Rape Sclerotinia in Hyperspectral Images Segmentation Method Based on Bi-GRU and Spatial-Spectral Information Fusion [J]. Smart Agriculture, 2024, 6(2): 40-48.
[4]	PANG Chunhui, CHEN Peng, XIA Yi, ZHANG Jun, WANG Bing, ZOU Yan, CHEN Tianjiao, KANG Chenrui, LIANG Dong. HI-FPN: A Hierarchical Interactive Feature Pyramid Network for Accurate Wheat Lodging Localization Across Multiple Growth Periods [J]. Smart Agriculture, 2024, 6(2): 128-139.
[5]	ZHANG Yuyu, BING Shuying, JI Yuanhao, YAN Beibei, XU Jinpu. Grading Method of Fresh Cut Rose Flowers Based on Improved YOLOv8s [J]. Smart Agriculture, 2024, 6(2): 118-127.
[6]	ZHANG Jianhua, YAO Qiong, ZHOU Guomin, WU Wendi, XIU Xiaojie, WANG Jian. Intelligent Identification of Crop Agronomic Traits and Morphological Structure Phenotypes: A Review [J]. Smart Agriculture, 2024, 6(2): 14-27.
[7]	GUO Wang, YANG Yusen, WU Huarui, ZHU Huaji, MIAO Yisheng, GU Jingqiu. Big Models in Agriculture: Key Technologies, Application and Future Directions [J]. Smart Agriculture, 2024, 6(2): 1-13.
[8]	WANG Herong, CHEN Yingyi, CHAI Yingqian, XU Ling, YU Huihui. Image Segmentation Method Combined with VoVNetv2 and Shuffle Attention Mechanism for Fish Feeding in Aquaculture [J]. Smart Agriculture, 2023, 5(4): 137-149.
[9]	LI Zhengkai, YU Jiahui, PAN Shijia, JIA Zefeng, NIU Zijie. Individual Tree Skeleton Extraction and Crown Prediction Method of Winter Kiwifruit Trees [J]. Smart Agriculture, 2023, 5(4): 92-104.
[10]	TANG Hui, WANG Ming, YU Qiushi, ZHANG Jiaxi, LIU Liantao, WANG Nan. Root Image Segmentation Method Based on Improved UNet and Transfer Learning [J]. Smart Agriculture, 2023, 5(3): 96-109.
[11]	PAN Weiting, SUN Mengli, YUN Yan, LIU Ping. Identification Method of Wheat Grain Phenotype Based on Deep Learning of ImCascade R-CNN [J]. Smart Agriculture, 2023, 5(3): 110-120.
[12]	GUAN Bolun, ZHANG Liping, ZHU Jingbo, LI Runmei, KONG Juanjuan, WANG Yan, DONG Wei. The Key Issues and Evaluation Methods for Constructing Agricultural Pest and Disease Image Datasets: A Review [J]. Smart Agriculture, 2023, 5(3): 17-34.
[13]	LONG Jianing, ZHANG Zhao, LIU Xiaohang, LI Yunxia, RUI Zhaoyu, YU Jiangfan, ZHANG Man, FLORES Paulo, HAN Zhexiong, HU Can, WANG Xufeng. Wheat Lodging Types Detection Based on UAV Image Using Improved EfficientNetV2 [J]. Smart Agriculture, 2023, 5(3): 62-74.
[14]	ZHANG Gan, YAN Haifeng, HU Gensheng, ZHANG Dongyan, CHENG Tao, PAN Zhenggao, XU Haifeng, SHEN Shuhao, ZHU Keyu. Identification Method of Wheat Field Lodging Area Based on Deep Learning Semantic Segmentation and Transfer Learning [J]. Smart Agriculture, 2023, 5(3): 75-85.
[15]	LIU Yixue, SONG Yuyang, CUI Ping, FANG Yulin, SU Baofeng. Diagnosis of Grapevine Leafroll Disease Severity Infection via UAV Remote Sensing and Deep Learning [J]. Smart Agriculture, 2023, 5(3): 49-61.