Distilled-MobileNet Model of Convolutional Neural Network Simplified Structure for Plant Disease Recognition

doi:10.12133/j.smartag.2021.3.1.202009-SA004

Abstract

Abstract:

The development of convolutional neural networks(CNN) has brought a large number of network parameters and huge model volumes, which greatly limites the application on devices with small computing resources, such as single-chip microcomputers and mobile devices. In order to solve the problem, a structured model compression method was studied in this research. Its core idea was using knowledge distillation to transfer the knowledge from the complex integrated model to a lightweight small-scale neural network. Firstly, VGG16 was used to train a teacher model with a higher recognition rate, whose volume was much larger than the student model. Then the knowledge in the model was transfered to MobileNet by using distillation. The parameters number of the VGG16 model was greatly reduced. The knowledge-distilled model was named Distilled-MobileNet, and was applied to the classification task of 38 common diseases (powdery mildew, Huanglong disease, etc.) of 14 crops (soybean, cucumber, tomato, etc.). The performance test of knowledge distillation on four different network structures of VGG16, AlexNet, GoogleNet, and ResNet showed that when VGG16 was used as a teacher model, the accuracy of the model was improved to 97.54%. Using single disease recognition rate, average accuracy rate, model memory and average recognition time as 4 indicators to evaluate the accuracy of the trained Distilled-MobileNet model in a real environment, the results showed that, the average accuracy of the model reached 97.62%, and the average recognition time was shortened to 0.218 s, only accounts for 13.20% of the VGG16 model, and the model size was reduced to only 19.83 MB, which was 93.60% smaller than VGG16. Compared with traditional neural networks, distilled-mobile model has a significant improvement in reducing size and shorting recognition time, and can provide a new idea for disease recognition on devices with limited memory and computing resources.

Key words: disease identification, deep learning, model compression, knowledge distillation, convolutional neural network

CLC Number:

TP3-0

QIU Wenjie, YE Jin, HU Liangqing, YANG Juan, LI Qili, MO Jianyou, YI Wanmao. Distilled-MobileNet Model of Convolutional Neural Network Simplified Structure for Plant Disease Recognition[J]. Smart Agriculture, 2021, 3(1): 109-117.

References

1	KAMILARIS A, PRENAFETA-BOLDú F X. A review of the use of convolutional neural networks in agriculture[J]. The Journal of Agricultural Science, 2018, 156(3): 312-322.
2	KARLEKAR A, SEAL A. SoyNet: Soybean leaf diseases classification[J]. Computers and Electronics in Agriculture, 2020, 172: ID 105342.
3	TETILA E C, MACHADO B B, MENEZES G K, et al. Automatic recognition of soybean leaf diseases Using UAV images and deep convolutional neural networks[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 17(5): 903-907.
4	ADEEL A, KHAN M A, AKRAM T, et al. Entropy controlled deep features selection framework for grape leaf diseases recognition[J]. Expert Systems, 2020(1): DOI: 10.1111/exsy.12569.
5	ZHANG X, QIAO Y, MENG F, et al. Identification of maize leaf diseases using improved deep convolutional neural networks[J]. IEEE Access, 2018(6): 30370-30377.
6	陈桂芬, 赵姗, 曹丽英, 等.基于迁移学习与卷积神经网络的玉米植株病害识别[J]. 智慧农业, 2019, 1(2): 34-44.
6	CHEN G, ZHAO S, CAO L, et al. Corn plant disease recognition based on migration learning and convolutional neural network[J]. Smart Agriculture, 2019, 1(2): 34-44.
7	吴华瑞. 基于深度残差网络的番茄叶片病害识别方法[J]. 智慧农业, 2019, 1(4): 42-49.
7	WU H. Method of tomato leaf diseases recognition method based on deep residual network[J]. Smart Agriculture, 2019, 1(4): 42-49.
8	雷小康, 尹志刚, 赵瑞莲. 基于FPGA的卷积神经网络定点加速[J]. 计算机应用, 2020, 40(10): 2811-2816.
8	LEI X, YIN Z, ZHAO R. FPGA-based convolutional neural network fixed-point acceleration[J]. Computer Applications, 2020, 40(10): 2811-2816.
9	HAN R, LIU C, LI S, et al. Accelerating deep learning systems via critical set identification and model compression[J]. IEEE Transactions on Computers, 2020, 69(7): 1059-1070.
10	RAHMAN C R, ARKO P S, ALI M E, et al. Identification and recognition of rice diseases and pests using convolutional neural networks[EB/OL]. arXiv:1812.01
10	043v3 [cs.CV] 4 Mar2020.
11	SHIH K H, CHIU C, LIN J A, et al. Real-time object detection with reduced region proposal network via multi-feature concatenation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31: 2164-2173.
12	HC A, YL A, KCY B, et al. Block change learning for knowledge distillation[J]. Information Sciences, 2020, 513: 360-371.
13	WU M, CHIU C, WU K. Multi-teacher knowledge distillation for compressed video action recognition based on deep learning[C]// ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, New York, USA: IEEE, 2019.
14	HUGHES D P, SALATHE M. An open access repository of images on plant health to enable the development of mobile disease diagnostics[EB/OL]. 2015. arXiv:1511.
14	08060 [cs.CY].
15	HINTON G, VINYALS O, DEAN J. Distilling the knowledge in a neural network[EB/OL]. Computer Science, 2015. arXiv:1503.
15	02531 [stat.ML].
16	SHORTEN C, KHOSHGOFTAAR T M. A survey on image data augmentation for deep learning[J]. Journal of Big Data, 2019, 6(1): ID 60.
17	RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[EB/OL]. 2015. arXiv:1511.06434v
18	cs.LG] 19 Nov 2015.
19	WENG L, PRENEEL B. A secure perceptual hash algorithm for image content authentication[C]// 12th IFIP TC 6/TC 11 International Conference on Communications and Multimedia Security. Berlin Heidelberg, German: Springer Verlag, 2011: 108-121.
20	PAN S J, YANG Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2009, 22(10): 1345-1359.
21	HOWARD A G, ZHU M, CHEN B, et al. MobileNets: Efficient convolutional neural networks for mobile vision applications[EB/OL]. 2017. arXiv:1704.
21	04861 [cs.CV].
22	KAILATH T. The divergence and Bhattacharyya distance measures in signal selection[J]. IEEE Transactions on Communication Technology, 1967, 15(1): 52-60.
23	ASHOK A, RHINEHART N, BEAINY F, et al. N2N learning: Network to network compression via policy Gradient Reinforcement Learning[C]// 6th International Conference on Learning Representations, ICLR 2018. Vancouver, BC, Canada: International Conference on Learning Representations, ICLR, 2018.
24	CHEBOTAR Y, WATERS A. Distilling knowledge from ensembles of neural networks for speech recognition[C]// Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. San Francisco, CA, United States: International Speech Communication Association, 2016: 3439-3433.

[1]	MAO Kebiao, ZHANG Chenyang, SHI Jiancheng, WANG Xuming, GUO Zhonghua, LI Chunshu, DONG Lixin, WU Menxin, SUN Ruijing, WU Shengli, JI Dabin, JIANG Lingmei, ZHAO Tianjie, QIU Yubao, DU Yongming, XU Tongren. The Paradigm Theory and Judgment Conditions of Geophysical Parameter Retrieval Based on Artificial Intelligence [J]. Smart Agriculture, 2023, 5(2): 161-171.
[2]	PAN Chenlu, ZHANG Zhenghua, GUI Wenhao, MA Jiajun, YAN Chenxi, ZHANG Xiaomin. Rice Disease and Pest Recognition Method Integrating ECA Mechanism and DenseNet201 [J]. Smart Agriculture, 2023, 5(2): 45-55.
[3]	XIA Xue, CHAI Xiujuan, ZHANG Ning, ZHOU Shuo, SUN Qixin, SUN Tan. A Lightweight Fruit Load Estimation Model for Edge Computing Equipment [J]. Smart Agriculture, 2023, 5(2): 1-12.
[4]	GUO Yangyang, DU Shuzeng, QIAO Yongliang, LIANG Dong. Advances in the Applications of Deep Learning Technology for Livestock Smart Farming [J]. Smart Agriculture, 2023, 5(1): 52-65.
[5]	ZUO Min, HU Tianyu, DONG Wei, ZHANG Kexin, ZHANG Qingchuan. Forecast and Analysis of Agricultural Products Logistics Demand Based on Informer Neural Network: Take the Central China Aera as An Example [J]. Smart Agriculture, 2023, 5(1): 34-43.
[6]	ZHANG Wenjing, JIANG Zezhong, QIN Lifeng. Identifying Multiple Apple Leaf Diseases Based on the Improved CBAM-ResNet18 Model Under Weak Supervision [J]. Smart Agriculture, 2023, 5(1): 111-121.
[7]	JI Jie, JIN Zhou, WANG Rujing, LIU Haiyan, LI Zhiyuan. Progressive Convolutional Net Based Method for Agricultural Named Entity Recognition [J]. Smart Agriculture, 2023, 5(1): 122-131.
[8]	HU Songtao, ZHAI Ruifang, WANG Yinghua, LIU Zhi, ZHU Jianzhong, REN He, YANG Wanneng, SONG Peng. Extraction of Potato Plant Phenotypic Parameters Based on Multi-Source Data [J]. Smart Agriculture, 2023, 5(1): 132-145.
[9]	LIU Xiaohang, ZHANG Zhao, LIU Jiaying, ZHANG Man, LI Han, FLORES Paulo, HAN Xiongzhe. Infield Corn Kernel Detection and Counting Based on Multiple Deep Learning Networks [J]. Smart Agriculture, 2022, 4(4): 49-60.
[10]	XU Yulin, KANG Mengzhen, WANG Xiujuan, HUA Jing, WANG Haoyu, SHEN Zhen. Corn and Soybean Futures Price Intelligent Forecasting Based on Deep Learning [J]. Smart Agriculture, 2022, 4(4): 156-163.
[11]	LUO Qing, RAO Yuan, JIN Xiu, JIANG Zhaohui, WANG Tan, WANG Fengyi, ZHANG Wu. Multi-Class on-Tree Peach Detection Using Improved YOLOv5s and Multi-Modal Images [J]. Smart Agriculture, 2022, 4(4): 84-104.
[12]	ZHANG Zhibo, ZHAO Xining, GAO Xiaodong, ZHANG Li, YANG Menghao. Accurate Extraction of Apple Orchard on the Loess Plateau Based on Improved Linknet Network [J]. Smart Agriculture, 2022, 4(3): 95-107.
[13]	SHANG Fengnan, ZHOU Xuecheng, LIANG Yingkai, XIAO Mingwei, CHEN Qiao, LUO Chendi. Detection Method for Dragon Fruit in Natural Environment Based on Improved YOLOX [J]. Smart Agriculture, 2022, 4(3): 120-131.
[14]	HE Ruimin, ZHENG Kefeng, WEI Qinyang, ZHANG Xiaobin, ZHANG Jun, ZHU Yihang, ZHAO Yiying, GU Qing. Identification and Counting of Silkworms in Factory Farm Using Improved Mask R-CNN Model [J]. Smart Agriculture, 2022, 4(2): 163-173.
[15]	ZHUANG Jiayu, XU Shiwei, LI Yang, XIONG Lu, LIU Kebao, ZHONG Zhiping. Supply and Demand Forecasting Model of Multi-Agricultural Products Based on Deep Learning [J]. Smart Agriculture, 2022, 4(2): 174-182.