Identification and Morphological Analysis of Adult Spodoptera Frugiperda and Its Close Related Species Using Deep Learning

doi:10.12133/j.smartag.2020.2.3.202008-SA001

Abstract

Abstract:

Invasive pest fall armyworm (FAW) Spodoptera frugiperda is one of the serious threats to the food safety. Early warning and control plays a key role in FAW management. Nowadays, deep learning technology has been applied to recognize the image of FAW. However, there is a serious lack of training dataset in the current researches, which may mislead the model to learn features unrelated to the key visual characteristics (ring pattern, reniform pattern, etc.) of FAW adults and its close related species. Therefore, this research established a database of 10,177 images belonging to 7 species of noctuid adults, including FAW and 6 FAW close related species. Based on the small-scale dataset, transfer learning was used to build the recognition model of FAW adults by employing three deep learning models (VGG-16, ResNet-50 and DenseNet-121) pretrained on ImageNet. All of the models got more than 98% recognition accuracy on the same testing dataset. Moreover, by using feature visualization techniques, this research visualized the features learned by deep learning models and compared them to the related key visual characteristics recognized by human experts. The results showed that there was a high consistency between the two counterparts, i.e., the average feature recognition rate of ResNet-50 and DenseNet-121 was around 85%, which further demonstrated that it was possible to use the deep learning technology for the real-time monitoring of FAW adults. In addition, this study also found that the learning abilities of key visual characteristics among different models were different even though they have similar recognition accuracy. Herein, we suggest that when evaluating the model capacity, we should not only focus on the recognition rate, the ability of learning individual visual characteristics should be allocated importance for evaluating the model performance. For those important taxonomical traits, if the visualization results indicated that the model didn't learnt them, we should then modify our datasets or adjusting the training strategies to increase the learning ability. In conclusion, this study verified that visualizing the features learnt by the model is a good way to evaluate the learning ability of deep learning models, and to provide a possible way for other researchers in the field who want to understand the features learnt by deep learning models.

Key words: Spodoptera frugiperda, noctuid, adult moth recognition, deep learning, visual characteristics, feature visualization, transfer learning

CLC Number:

S431.9

WEI Jing, WANG Yuting, YUAN Huizhu, ZHANG Menglei, WANG Zhenying. Identification and Morphological Analysis of Adult Spodoptera Frugiperda and Its Close Related Species Using Deep Learning[J]. Smart Agriculture, 2020, 2(3): 75-85.

Figures/Tables 7

Table 1

Table 2

Fig. 1

Table 3

Fig. 2

Table 4

Table 5

References 24

1	吴孔明. 中国草地贪夜蛾的防控策略[J]. 植物保护, 2020, 46(2): 1-5.
	WU K. Management strategies of fall armyworm (Spodoptera frugiperda) in China[J]. Plant Protection, 2020, 46(2): 1-5.
2	张磊, 柳贝, 姜玉英, 等. 中国不同地区草地贪夜蛾种群生物型分子特征分析[J]. 植物保护, 2019, 45(4): 20-27.
	ZHANG L, LIU B, JIANG Y, et al. Molecular characterization analysis of fall armyworm populations in China[J]. Plant Protection, 2019, 45(4): 20-27.
3	赵胜园, 罗倩明, 孙小旭, 等. 草地贪夜蛾与斜纹夜蛾的形态特征和生物学习性比较[J]. 中国植保导刊, 2019, 39(5): 26-35.
	ZHAO S, LUO Q, SUN X, et al. Comparison of morphological and biological characteristics between Spodoptera frugiperda and Spodoptera litura[J]. China Plant Protection, 2019, 39(5): 26-35.
4	MOHANTY S P, HUGHES D P, SALATHÉ M. Using deep learning for image-based plant disease detection[J]. Frontiers in Plant Science, 2016, 7(1419): 1-10.
5	CHIWAMBA S H, PHIRI J, NKUNIKA P O Y, et al. An application of machine learning algorithms in automated identification and capturing of fall armyworm (FAW) moths in the field[C]// 2018 Ictsz International Conference In Icts (ICICT). Lusaka, Zambia: ICICT. 2019: 119-124.
6	CHULU F, PHIRI J, NKUNIKA P, et al. A convolutional neural network for automatic identification and classification of fall army worm moth[J]. International Journal of Advanced Computer Science and Applications, 2019, 10(7): 112-118.
7	于业达, 顾偌铖, 唐运林, 等. 基于深度学习的草地贪夜蛾自动识别 [J]. 西南大学学报(自然科学版), 2019, 41(9): 24-31.
	YU Y, GU R, TANG Y, et al. A CNN-based automatic identification system for Spodoptera frugiperda[J]. Journal of Southwest University (Natural Science), 2019, 41(9): 24-31.
8	ESTEVA A, KUPREL B, NOVOA R A, et al. Dermatologist-level classification of skin cancer with deep neural networks [J]. Nature, 2017, 542(7639): 115-118.
9	SPRINGENBERG J T, DOSOVITSKIY A, BROX T, et al. Striving for simplicity: The all convolutional net[C]// 2015 International Conference on Learning Representations (ICLR). San Diego, CA, USA: ICLR. 2015: 1-14.
10	SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization [C]// 2017 IEEE International Conference on Computer Vision (ICCV). Piscataway, New York, USA: IEEE. 2017: 618-626.
11	CHATTOPADHYAY A, SARKAR A, HOWLADER P, et al. Grad-CAM++: Generalized gradient-based visual explanations for deep convolutional networks[C]// 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway, New York, USA: IEEE. 2018: 839-847,
12	朱弘复, 陈一心. 中国经济昆虫志[M]. 北京: 科学出版社, 1963: 23-23.
	ZHU H, CHEN Y. Economic insects of China[M]. Beijing: China Science Publishing & Media Ltd., 1963: 23-23.
13	洪晓月, 丁锦华. 农业昆虫学(第二版)[M]. 北京: 中国农业出版社, 2007: 46-235.
	HONG X, DING J. Agricultural entomology (2nd ed)[M]. Beijing: China Agriculture Press Co., Ltd., 2007: 46-235.
14	孔德英, 孙涛, 滕少娜, 等. 草地贪夜蛾及其近似种的鉴定[J]. 植物检疫, 2019, 33(4): 37-40.
	KONG D, SUN T, TENG S, et al. Identification of fall armyworm, Spodoptera frugiperda, and its similar species in morphology [J]. Plant Quarantine, 2019, 33(4): 37-40.
15	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]// 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), Piscataway. New York, USA: IEEE. 2015, 730-734.
16	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2016: 770-778.
17	HUANG G, LIU Z, MAATEN L V D, et al. Densely connected convolutional networks[C]// 2017 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2017: 2261-2269.
18	DENG J, DONG W, SOCHER R, et al. Imagenet: A large-scale hierarchical image database[C]// 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2009: 248-255.
19	LOSHCHILOV I, HUTTER F. SGDR: Stochastic gradient descent with warm restarts[C]// 2017 International Conference on Learning Representations (ICLR). Toulon, France: ICLR. 2017: 1-16.
20	GOTMARE A, KESKAR N S, XIONG C, et al. A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation[C]// 2019 International Conference on Learning Representations (ICLR). New Orleans, LA, USA: ICLR. 2019: 1-16.
21	SMITH L N. Cyclical learning rates for training neural networks[C]// 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway, New York, USA: IEEE. 2015: 464-472.
22	ZHOU B, KHOSLA A, LAPEDRIZA, et al. Learning deep features for discriminative localization[C]// 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2016: 2921-2929.
23	PARK J, KIM D I, CHOI B, et al. Classification and morphological analysis of vector mosquitoes using deep convolutional neural networks[J]. Scientific Reports, 2020, 10(1): 1-12.
24	LI K, WU Z, PENG K, et al. Tell me where to look: guided attention inference network[C]// 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New York, USA: IEEE. 2018:9215-9223.

物种名（均为成虫）	图片总数目/幅	训练集数目（80%）/幅	训练集平衡采样策略
草地贪夜蛾（雄虫）	850	680	680 + 120的过采样
棉铃虫	1438	1150	800的欠采样
甜菜夜蛾	731	585	585 + 215的过采样
斜纹夜蛾	2282	1826	800的欠采样
粘虫	2523	2019	800的欠采样
小地老虎	1976	1581	800的欠采样
黄地老虎	377	302	302 + 498的过采样

物种	视觉特征
草地贪夜蛾（雄虫）	特征1：环形纹黄褐色，边缘内侧较浅，外侧为黑色至黑褐色；特征2：肾形纹明显，颜色灰褐色；特征3：环形纹上方有一个黑褐色至黑色斑纹；特征4：前翅顶角处有一个较大白色斑纹；特征5：肾形纹靠前缘脉一侧有一个白斑；特征6：亚端线白色；特征7：肾形纹靠臀脉侧白斑可与环形纹相连，渐变为黄褐色
棉铃虫	特征1：前翅中横线有肾形纹，边褐色，中央深褐色斑；特征2：环形纹褐边，中央有一个褐色点；特征3：中横线由肾形纹内侧斜至后缘，末端达环形纹的正下方；特征4：亚端线的锯齿纹较均匀，距外缘的宽度大致相等。
甜菜夜蛾	特征1：前翅中央近前缘外方有一个肾形斑；特征2：内方有一个环形斑
斜纹夜蛾	特征1：前翅翅基部前半部有白线条数条，内、外横线之间有灰白色宽带，自内横线前缘斜伸至外横线近内缘1/3处；特征2：灰白色宽带中有2条褐色线纹（雄蛾不显著）；特征3：复杂的黑褐色斑纹；特征4：翅基部前半部有白线数条
粘虫	特征1：前翅中央近前缘处有2个淡黄色圆斑；特征2：外方圆斑下有一个小白点；特征3：外方圆斑两侧各有一小黑点；特征4：前翅顶角有一条向后缘的黑色斜纹
小地老虎	特征1：前翅环形纹黑色，有一个圆灰环；特征2：肾形纹黑色黑边；特征3：肾形纹外方有一个黑色楔形黑斑；特征4：黑边楔状纹；特征5：亚外缘线上有2个尖端向内的楔形黑斑
黄地老虎	特征1：前翅环形纹暗褐色黑边，圆形或微呈椭圆形；特征2：肾形纹棕褐色黑边，较大；特征3：清晰黑边楔状纹

序号	训练策略或超参数	设置
1	优化器类型	随机梯度下降（Stochastic Gradient Descent）
2	学习率调整策略	带热重启的随机梯度下降（Stochastic Gradient Descent with Warm Restart）^[19,20]，基本周期为1，倍率为2，每个周期开始时学习率为基础学习率，结束后学习率降为0
3	学习率退火方式	余弦退火（Cosine Annealing）^[21]
4	基础学习率	0.01
5	动量因子	0.9
6	权重衰减	0.0005
7	批大小	32
8	训练集数据增强策略	维持图片宽高比，最短边调整为224像素；中心裁剪224×224像素图像块；随机水平翻转；随机垂直翻转；随机平移；随机旋转；像素值按照ImageNet数据集进行归一化
9	总训练周期数	200

模型结构	平衡采样	数据增强	每类测试样本数（共7类）	准确率/%	epoch^th （总数200）
VGG-16			38	98.46	108
		√		99.22	104
	√			98.07	116
	√	√		99.61	99
ResNet-50				99.22	50
		√		99.61	103
	√			98.45	91
	√	√		99.61	116
DenseNet-121				99.22	116
		√		99.22	110
	√			98.45	96
	√	√		99.22	64

物种	特征索引	有效样本数/总样本数	特征识别率/%
			DenseNet-121		ResNet-50		VGG-16
			平衡采样	随机洗牌	平衡采样	随机洗牌	平衡采样	随机洗牌
草地贪夜蛾（雄虫）	1	30/30	95.0	96.7	91.7	95.0	93.3	86.7
	2	16/30	87.5	87.5	84.4	90.6	37.5	40.6
	3	20/30	77.5	75.0	67.5	60.0	10.0	10.0
	4	26/30	30.8	23.1	40.4	42.3	3.9	0.0
	5	27/30	81.5	85.2	85.2	90.7	44.4	59.3
	6	24/30	52.1	64.6	75.0	75.0	8.3	8.3
	7	29/30	96.6	96.6	100.0	98.3	65.5	63.8
棉铃虫	1	27/30	98.2	100.0	100.0	100.0	77.8	83.3
	2	24/30	97.9	100.0	95.8	93.8	68.8	87.5
	3	16/30	87.5	84.4	75.0	81.3	15.6	34.4
	4	20/30	22.5	52.5	47.5	80.0	12.5	32.5
甜菜夜蛾	1	29/30	89.7	86.2	94.8	87.9	13.8	25.9
甜菜夜蛾	2	30/30	91.7	93.3	93.3	100.0	68.3	86.7
斜纹夜蛾	1	30/30	100.0	100.0	100.0	100.0	96.7	98.3
	2	18/30	100.0	100.0	100.0	100.0	86.1	83.3
	3	30/30	100.0	100.0	98.3	100.0	26.7	41.7
	4	23/30	67.4	73.9	67.4	76.1	32.6	28.3
粘虫	1	4/30	75.0	75.0	50.0	75.0	50.0	0.0
	2	30/30	95.0	98.3	95.0	95.0	86.7	96.7
	3	28/30	98.2	100.0	96.4	96.4	83.9	100.0
	4	17/30	73.5	55.9	58.8	55.9	38.2	23.5
小地老虎	1	15/30	90.0	80.0	86.7	56.7	43.3	46.7
	2	27/30	100.0	96.3	96.3	85.2	90.7	85.2
	3	30/30	100.0	100.0	100.0	95.0	98.3	100.0
	4	27/30	90.7	88.9	87.0	74.1	59.3	64.8
	5	24/30	95.8	95.8	93.8	95.8	31.3	54.2
黄地老虎	1	25/28	80.0	74.0	74.0	68.0	64.0	58.0
	2	26/28	92.3	96.2	92.3	94.2	80.8	84.6
	3	27/28	98.2	100.0	96.3	98.2	72.2	85.2
特征识别率平均值/%			85.0	85.5	84.2	84.8	53.8	57.6