Distilled-MobileNet Model of Convolutional Neural Network Simplified Structure for Plant Disease Recognition

doi:10.12133/j.smartag.2021.3.1.202009-SA004

Abstract

Abstract:

The development of convolutional neural networks(CNN) has brought a large number of network parameters and huge model volumes, which greatly limites the application on devices with small computing resources, such as single-chip microcomputers and mobile devices. In order to solve the problem, a structured model compression method was studied in this research. Its core idea was using knowledge distillation to transfer the knowledge from the complex integrated model to a lightweight small-scale neural network. Firstly, VGG16 was used to train a teacher model with a higher recognition rate, whose volume was much larger than the student model. Then the knowledge in the model was transfered to MobileNet by using distillation. The parameters number of the VGG16 model was greatly reduced. The knowledge-distilled model was named Distilled-MobileNet, and was applied to the classification task of 38 common diseases (powdery mildew, Huanglong disease, etc.) of 14 crops (soybean, cucumber, tomato, etc.). The performance test of knowledge distillation on four different network structures of VGG16, AlexNet, GoogleNet, and ResNet showed that when VGG16 was used as a teacher model, the accuracy of the model was improved to 97.54%. Using single disease recognition rate, average accuracy rate, model memory and average recognition time as 4 indicators to evaluate the accuracy of the trained Distilled-MobileNet model in a real environment, the results showed that, the average accuracy of the model reached 97.62%, and the average recognition time was shortened to 0.218 s, only accounts for 13.20% of the VGG16 model, and the model size was reduced to only 19.83 MB, which was 93.60% smaller than VGG16. Compared with traditional neural networks, distilled-mobile model has a significant improvement in reducing size and shorting recognition time, and can provide a new idea for disease recognition on devices with limited memory and computing resources.

Key words: disease identification, deep learning, model compression, knowledge distillation, convolutional neural network

CLC Number:

TP3-0

QIU Wenjie, YE Jin, HU Liangqing, YANG Juan, LI Qili, MO Jianyou, YI Wanmao. Distilled-MobileNet Model of Convolutional Neural Network Simplified Structure for Plant Disease Recognition[J]. Smart Agriculture, 2021, 3(1): 109-117.

References

1	KAMILARIS A, PRENAFETA-BOLDú F X. A review of the use of convolutional neural networks in agriculture[J]. The Journal of Agricultural Science, 2018, 156(3): 312-322.
2	KARLEKAR A, SEAL A. SoyNet: Soybean leaf diseases classification[J]. Computers and Electronics in Agriculture, 2020, 172: ID 105342.
3	TETILA E C, MACHADO B B, MENEZES G K, et al. Automatic recognition of soybean leaf diseases Using UAV images and deep convolutional neural networks[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 17(5): 903-907.
4	ADEEL A, KHAN M A, AKRAM T, et al. Entropy controlled deep features selection framework for grape leaf diseases recognition[J]. Expert Systems, 2020(1): DOI: 10.1111/exsy.12569.
5	ZHANG X, QIAO Y, MENG F, et al. Identification of maize leaf diseases using improved deep convolutional neural networks[J]. IEEE Access, 2018(6): 30370-30377.
6	陈桂芬, 赵姗, 曹丽英, 等.基于迁移学习与卷积神经网络的玉米植株病害识别[J]. 智慧农业, 2019, 1(2): 34-44.
6	CHEN G, ZHAO S, CAO L, et al. Corn plant disease recognition based on migration learning and convolutional neural network[J]. Smart Agriculture, 2019, 1(2): 34-44.
7	吴华瑞. 基于深度残差网络的番茄叶片病害识别方法[J]. 智慧农业, 2019, 1(4): 42-49.
7	WU H. Method of tomato leaf diseases recognition method based on deep residual network[J]. Smart Agriculture, 2019, 1(4): 42-49.
8	雷小康, 尹志刚, 赵瑞莲. 基于FPGA的卷积神经网络定点加速[J]. 计算机应用, 2020, 40(10): 2811-2816.
8	LEI X, YIN Z, ZHAO R. FPGA-based convolutional neural network fixed-point acceleration[J]. Computer Applications, 2020, 40(10): 2811-2816.
9	HAN R, LIU C, LI S, et al. Accelerating deep learning systems via critical set identification and model compression[J]. IEEE Transactions on Computers, 2020, 69(7): 1059-1070.
10	RAHMAN C R, ARKO P S, ALI M E, et al. Identification and recognition of rice diseases and pests using convolutional neural networks[EB/OL]. arXiv:1812.01
10	043v3 [cs.CV] 4 Mar2020.
11	SHIH K H, CHIU C, LIN J A, et al. Real-time object detection with reduced region proposal network via multi-feature concatenation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31: 2164-2173.
12	HC A, YL A, KCY B, et al. Block change learning for knowledge distillation[J]. Information Sciences, 2020, 513: 360-371.
13	WU M, CHIU C, WU K. Multi-teacher knowledge distillation for compressed video action recognition based on deep learning[C]// ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, New York, USA: IEEE, 2019.
14	HUGHES D P, SALATHE M. An open access repository of images on plant health to enable the development of mobile disease diagnostics[EB/OL]. 2015. arXiv:1511.
14	08060 [cs.CY].
15	HINTON G, VINYALS O, DEAN J. Distilling the knowledge in a neural network[EB/OL]. Computer Science, 2015. arXiv:1503.
15	02531 [stat.ML].
16	SHORTEN C, KHOSHGOFTAAR T M. A survey on image data augmentation for deep learning[J]. Journal of Big Data, 2019, 6(1): ID 60.
17	RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[EB/OL]. 2015. arXiv:1511.06434v
18	cs.LG] 19 Nov 2015.
19	WENG L, PRENEEL B. A secure perceptual hash algorithm for image content authentication[C]// 12th IFIP TC 6/TC 11 International Conference on Communications and Multimedia Security. Berlin Heidelberg, German: Springer Verlag, 2011: 108-121.
20	PAN S J, YANG Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2009, 22(10): 1345-1359.
21	HOWARD A G, ZHU M, CHEN B, et al. MobileNets: Efficient convolutional neural networks for mobile vision applications[EB/OL]. 2017. arXiv:1704.
21	04861 [cs.CV].
22	KAILATH T. The divergence and Bhattacharyya distance measures in signal selection[J]. IEEE Transactions on Communication Technology, 1967, 15(1): 52-60.
23	ASHOK A, RHINEHART N, BEAINY F, et al. N2N learning: Network to network compression via policy Gradient Reinforcement Learning[C]// 6th International Conference on Learning Representations, ICLR 2018. Vancouver, BC, Canada: International Conference on Learning Representations, ICLR, 2018.
24	CHEBOTAR Y, WATERS A. Distilling knowledge from ensembles of neural networks for speech recognition[C]// Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. San Francisco, CA, United States: International Speech Communication Association, 2016: 3439-3433.

[1]	MA Liu, MAO Kebiao, GUO Zhonghua. Defogging Remote Sensing Images Method Based on a Hybrid Attention-Based Generative Adversarial Network [J]. Smart Agriculture, 2025, 7(2): 172-182.
[2]	XU Shiwei, LI Qianchuan, LUAN Rupeng, ZHUANG Jiayu, LIU Jiajia, XIONG Lu. Agricultural Market Monitoring and Early Warning: An Integrated Forecasting Approach Based on Deep Learning [J]. Smart Agriculture, 2025, 7(1): 57-69.
[3]	GONG Yu, WANG Ling, ZHAO Rongqiang, YOU Haibo, ZHOU Mo, LIU Jie. Tomato Growth Height Prediction Method by Phenotypic Feature Extraction Using Multi-modal Data [J]. Smart Agriculture, 2025, 7(1): 97-110.
[4]	QI Zijun, NIU Dangdang, WU Huarui, ZHANG Lilin, WANG Lunfeng, ZHANG Hongming. Chinese Kiwifruit Text Named Entity Recognition Method Based on Dual-Dimensional Information and Pruning [J]. Smart Agriculture, 2025, 7(1): 44-56.
[5]	ZHANG Hui, HU Jun, SHI Hang, LIU Changxi, WU Miao. Precision Target Spraying System Integrated with Remote Deep Learning Recognition Model for Cabbage Plant Centers [J]. Smart Agriculture, 2024, 6(6): 85-95.
[6]	LU Bibo, LIANG Di, YANG Jie, SONG Aiqing, HUANGFU Shangwei. Image Segmentation Method of Chinese Yam Leaves in Complex Background Based on Improved ENet [J]. Smart Agriculture, 2024, 6(6): 109-120.
[7]	LUO Youlu, PAN Yonghao, XIA Shunxing, TAO Youzhi. Lightweight Apple Leaf Disease Detection Algorithm Based on Improved YOLOv8 [J]. Smart Agriculture, 2024, 6(5): 128-138.
[8]	LIU Yi, ZHANG Yanjun. ReluformerN: Lightweight High-Low Frequency Enhanced for Hyperspectral Agricultural Lancover Classification [J]. Smart Agriculture, 2024, 6(5): 74-87.
[9]	NIAN Yue, ZHAO Kaixuan, JI Jiangtao. Cow Hoof Slippage Detecting Method Based on Enhanced DeepLabCut Model [J]. Smart Agriculture, 2024, 6(5): 153-163.
[10]	ZHANG Yanqi, ZHOU Shuo, ZHANG Ning, CHAI Xiujuan, SUN Tan. A Regional Farming Pig Counting System Based on Improved Instance Segmentation Algorithm [J]. Smart Agriculture, 2024, 6(4): 53-63.
[11]	LI Minghuang, SU Lide, ZHANG Yong, ZONG Zheying, ZHANG Shun. Automatic Measurement of Mongolian Horse Body Based on Improved YOLOv8n-pose and 3D Point Cloud Analysis [J]. Smart Agriculture, 2024, 6(4): 91-102.
[12]	WENG Zhi, FAN Qi, ZHENG Zhiqiang. Automatic Measurement Method of Beef Cattle Body Size Based on Multimodal Image Information and Improved Instance Segmentation Network [J]. Smart Agriculture, 2024, 6(4): 64-75.
[13]	HOU Yiting, RAO Yuan, SONG He, NIE Zhenjun, WANG Tan, HE Haoxu. A Rapid Detection Method for Wheat Seedling Leaf Number in Complex Field Scenarios Based on Improved YOLOv8 [J]. Smart Agriculture, 2024, 6(4): 128-137.
[14]	LI Hao, DU Yuqiu, XIAO Xingzhu, CHEN Yanxi. Remote Sensing Identification Method of Cultivated Land at Hill County of Sichuan Basin Based on Deep Learning [J]. Smart Agriculture, 2024, 6(3): 34-45.
[15]	ZHANG Jun, CHEN Yuyan, QIN Zhenyu, ZHANG Mengyao, ZHANG Jun. Remote Sensing Extraction Method of Terraced Fields Based on Improved DeepLab v3+ [J]. Smart Agriculture, 2024, 6(3): 46-57.