基于轻量级无锚点深度卷积神经网络的树上苹果检测模型

doi:10.12133/j.smartag.2020.2.1.202001-SA004

Smart Agriculture ›› 2020, Vol. 2 ›› Issue (1): 99-110.doi: 10.12133/j.smartag.2020.2.1.202001-SA004

• 专题--农业遥感与表型信息获取分析 • 上一篇下一篇

基于轻量级无锚点深度卷积神经网络的树上苹果检测模型

夏雪^1,², 孙琦鑫^1,², 侍啸^1,², 柴秀娟^1,²()

^1.中国农业科学院农业信息研究所，北京 100081
^2.农业农村部农业大数据重点实验室，北京 100081

收稿日期:2020-01-21 修回日期:2020-02-19 出版日期:2020-03-30 发布日期:2020-04-17
基金资助:
国家自然科学基金面上项目(61976219);中国农业科学院农业信息研究所基本科研业务费项目(JBYW-AII-2019-18);中国农业科学院科技创新工程项目(CAAS-ASTIP-2016-AII)
作者简介:夏雪（1983－），男，博士，助理研究员，研究方向：果树表型研究与应用,Email:xiaxue@caas.cn
通讯作者: 柴秀娟（1978－），女，博士，研究员，研究方向：机器视觉、智能感知、农业机器人，电话：13911671540， E-mail:chaixiujuan@caas.cn

Apple detection model based on lightweight anchor-free deep convolutional neural network

Xia Xue^1,², Sun Qixin^1,², Shi Xiao^1,², Chai Xiujuan^1,²()

^1.Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing 100081, China
^2.Key Laboratory of Agricultural Big Data, Ministry of Agriculture and Rural Affairs, Beijing 100081, China

Received:2020-01-21 Revised:2020-02-19 Online:2020-03-30 Published:2020-04-17
corresponding author: Xiujuan Chai E-mail:chaixiujuan@caas.cn

摘要/Abstract

摘要：

为提高现有苹果目标检测模型在硬件资源受限制条件下的性能和适应性，实现在保持较高检测精度的同时，减轻模型计算量，降低检测耗时，减少模型计算和存储资源占用的目的，本研究通过改进轻量级的MobileNetV3网络，结合关键点预测的目标检测网络（CenterNet），构建了用于苹果检测的轻量级无锚点深度学习网络模型（M-CenterNet），并通过与CenterNet和单次多重检测器（Single Shot Multibox Detector，SSD）网络比较了模型的检测精度、模型容量和运行速度等方面的综合性能。对模型的测试结果表明，本研究模型的平均精度、误检率和漏检率分别为88.9%、10.9%和5.8%；模型体积和帧率分别为14.2MB和8.1fps；在不同光照方向、不同远近距离、不同受遮挡程度和不同果实数量等条件下有较好的果实检测效果和适应能力。在检测精度相当的情况下，所提网络模型体积仅为CenterNet网络的1/4；相比于SSD网络，所提网络模型的AP提升了3.9%，模型体积降低了84.3%；本网络模型在CPU环境中的运行速度比CenterNet和SSD网络提高了近1倍。研究结果可为非结构环境下果园作业平台的轻量化果实目标检测模型研究提供新的思路。

关键词: 机器视觉, 深度学习, 轻量级网络, 无锚点, 苹果检测

Abstract:

Intelligent production and robotic oporation are the efficient and sustainable agronomic route to cut down economic and environmental costs and boosting orchard productivity. In the actual scene of the orchard, high performance visual perception system is the premise and key for accurate and reliable operation of the automatic cultivation platform. Most of the existing apple detection models, however, are difficult to be used on the platforms with limited hardware resources in terms of computing power and storage capacity due to too many parameters and large model volume. In order to improve the performance and adaptability of the existing apple detection model under the condition of limited hardware resources, while maintaining detection accuracy, reducing the calculation of the model and the model computing and storage footprint, shorten detection time, this method improved the lightweight MobileNetV3 and combined the object detection network which was based on keypoint prediction (CenterNet) to build a lightweight anchor-free model (M-CenterNet) for apple detection. The proposed model used heatmap to search the center point (keypotint) of the object, and predict whether each pixel was the center point of the apple, and the local offset of the keypoint and object size of the apple were estimated based on the extracted center point without the need for grouping or Non-Maximum Suppression (NMS). In view of its advantages in model volume and speed, improved MobileNetV3 which was equipped with transposed convolutional layers for the better semantic information and location information was used as the backbone of the network. Compared with CenterNet and SSD (Single Shot Multibox Detector), the comprehensive performance, detection accuracy, model capacity and running speed of the model were compared. The results showed that the average precision, error rate and miss rate of the proposed model were 88.9%, 10.9% and 5.8%, respectively, and its model volume and frame rate were 14.2MB and 8.1fps. The proposed model is of strong environmental adaptability and has a good detection effect under the circumstance of various light, different occlusion, different fruits’ distance and number. By comparing the performance of the accuracy with the CenterNet and the SSD models, the results showed that the proposed model was only 1/4 of the size of CenterNet model while has comparable detection accuracy. Compared with the SSD model, the average precision of the proposed model increased by 3.9%, and the model volume decreased by 84.3%. The proposed model runs almost twice as fast using CPU than the CenterNet and SSD models. This study provided a new approach for the research of lightweight model in fruit detection with orchard mobile platform under unstructured environment.

Key words: machine vision, deep learning, lightweight network, anchor-free, apple detection

中图分类号:

TP183

夏雪, 孙琦鑫, 侍啸, 柴秀娟. 基于轻量级无锚点深度卷积神经网络的树上苹果检测模型[J]. 智慧农业(中英文), 2020, 2(1): 99-110.

Xia Xue, Sun Qixin, Shi Xiao, Chai Xiujuan. Apple detection model based on lightweight anchor-free deep convolutional neural network[J]. Smart Agriculture, 2020, 2(1): 99-110.

图/表 14

图1

图2

图3

图4

图5

图6

图7

图 8

图9

图10

图11

图12

表 1

表2

参考文献 38

1	Kang H, Chen C. Fruit detection and segmentation for apple harvesting using visual sensor in orchards[J]. Sensors, 2019, 19(20): 4599-4614.
2	王丹丹, 何东健. 基于R-FCN深度卷积神经网络的机器人疏果前苹果目标的识别[J]. 农业工程学报, 2019, 35(3): 156-163.
	Wang D, He D. Recognition of apple targets before fruits thinning by robot based on R-FCN deep convolution neural network[J]. Transactions of the CSAE, 2019, 35(3): 156-163.
3	赵德安，吴任迪，刘晓洋，等. 基于YOLO深度卷积神经网络的复杂背景下机器人采摘苹果定位[J]. 农业工程学报, 2019, 35(3): 164-173.
	Zhao D, Wu R, Liu X, et al. Apple positioning based on YOLO deep convolutional neural network for picking robot in complex background[J]. Transactions of the CSAE, 2019, 35(3): 164-173.
4	Gené-Mola J, Vilaplana V, Rosell-Polo J R, et al. Multi-modal deep learning for Fuji apple detection using RGB-D cameras and their radiometric capabilities[J]. Computers and Electronics in Agriculture, 2019, 162: 689-698.
5	Underwood J P, Hung C, Whelan B, et al. Mapping almond orchard canopy volume, flowers, fruit and yield using LiDAR and vision sensors[J]. Computers and Electronics in Agriculture, 2016, 130: 83-96.
6	Bargoti S, Underwood J P. Image segmentation for fruit detection and yield estimation in apple orchards[J]. Journal of Field Robotics, 2017, 34(6): 1039-1060.
7	Silwal A, Gongal A, Karkee M. Apple identification in field environment with over the row machine vision system[J]. Agricultural Engineering International: CIGR Journal, 2014, 16(4): 66-75.
8	Wachs J P, Stern H I, Burks T, et al. Low and high-level visual feature-based apple detection from multi-modal images[J]. Precision Agriculture, 2010, 11(6): 717-735.
9	Qureshi W S, Payne A, Walsh K B, et al. Machine vision for counting fruit on mango tree canopies[J]. Precision Agriculture, 2017, 18(2): 224-244.
10	Zhou R, Damerow L, Sun Y, et al. Using colour features of cv.‘Gala’ apple fruits in an orchard in image processing to predict yield[J]. Precision Agriculture, 2012, 13(5): 568-580.
11	Wang Q, Nuske S, Bergerman M, et al. Automated crop yield estimation for apple orchards[C]// Experimental Robotics. Heidelberg: Springer, 2013: 745-758.
12	Xiong J, Liu Z, Lin R, et al. Green grape detection and picking-point calculation in a night-time natural environment using a charge-coupled device (ccd) vision sensor with artificial illumination[J]. Sensors, 2018, 18(4): 969-986.
13	Wang D, He D, Song H, et al. Combining SUN-based visual attention model and saliency contour detection algorithm for apple image segmentation[J]. Multimedia Tools and Applications, 2019, (78): 17391-17411.
14	Gongal A, Amatya S, Karkee M, et al. Sensors and systems for fruit detection and localization: A review[J]. Computers and Electronics in Agriculture, 2015, 116: 8-19.
15	Song Y, Glasbey C A, Horgan G W, et al. Automatic fruit recognition and counting from multiple images[J]. Biosystems Engineering, 2014, (118): 203-215.
16	Luo L, Tang Y, Zou X, et al. Robust grape cluster detection in a vineyard by combining the AdaBoost framework and multiple color components[J]. Sensors, 2016, 16(12): 2098-2118.
17	Wang C, Lee W S, Zou X, et al. Detection and counting of immature green citrus fruit based on the local binary patterns (lbp) feature using illumination-normalized images[J]. Precision Agriculture, 2018, 19(6): 1062-1083.
18	Guo Q, Chen Y, Tang Y, et al. Lychee fruit detection based on monocular machine vision in orchard environment[J]. Sensors, 2019, 19: no.4091.
19	Kestur R, Meduri A, Narasipura O. MangoNet: A deep semantic segmentation architecture for a method to detect and count mangoes in an open orchard[J]. Engineering Applications of Artificial Intelligence, 2019, 77: 59-69.
20	Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, inception-resnet and the impact of residual connections on learning[C]// Thirty-First AAAI Conference on Artificial Intelligence, 2017: 1-12.
21	Liu W, Wang Z, Liu X, et al. A survey of deep neural network architectures and their applications[J]. Neurocomputing, 2017, 234: 11-26.
22	Sa I, Ge Z, Dayoub F, et al. Deepfruits: a fruit detection system using deep neural networks[J]. Sensors, 2016, 16(8): 1222-1245.
23	Yu Y, Zhang K, Yang L, et al. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN[J]. Computers and Electronics in Agriculture, 2019, (163): 104846-104855.
24	陈桂芬, 赵姗, 曹丽英, 等. 基于迁移学习与卷积神经网络的玉米植株病害识别[J]. 智慧农业, 2019, 1(2): 34-44.
	Chen G, Zhao S, Cao L, et al. Corn plant disease recognition based on migration learning and convolutional neural network[J]. Smart Agriculture, 2019, 1(2): 34-44.
25	Tian Y, Yang G, Wang Z, et al. Apple detection during different growth stages in orchards using the improved YOLO-V3 model[J]. Computers and Electronics in Agriculture, 2019, (157): 417-426.
26	Koirala A, Walsh K B, Wang Z, et al. Deep learning for real-time fruit detection and orchard fruit load estimation: Benchmarking of ‘MangoYOLO’[J]. Precision Agriculture, 2019, (20): 1107-1135.
27	Williams H A M, Jones M H, Nejati M, et al. Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms[J]. Biosystems Engineering, 2019, (181): 140-156.
28	Wang D, Zhang N, Sun X, et al. AFP-Net: Realtime Anchor-Free Polyp Detection in Colonoscopy[C]// 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI). IEEE, 2019.
29	Law H, Deng J. Cornernet: Detecting objects as paired keypoints[C]// Proceedings of the European Conference on Computer Vision (ECCV), 2018: 734-750.
30	Duan K, Bai S, Xie L, et al. Centernet: Keypoint triplets for object detection[C]// Proceedings of the IEEE International Conference on Computer Vision. 2019: 6569-6578.
31	Zhou X, Zhuo J, Krahenbuhl P. Bottom-up object detection by grouping extreme and center points[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019: 850-859.
32	Zhou X, Wang D, Krähenbühl P. Objects as Points[J]. Cornell University, 2019, arXiv: 1904. 07850.
33	Howard A, Sandler M, Chu G, et al. Searching for mobilenetv3[J]. Cornell University, 2019, arXiv: 1905. 02244.
34	郑冬, 李向群, 许新征. 基于轻量化 SSD 的车辆及行人检测网络[J]. 南京师大学报 (自然科学版), 2019, 42(1): 73-81.
	Zheng D, Li X, Xu X. Vehicle and pedestrian detection model based on lightweight SSD[J]. Journal of Nanjing Normal University (Natural Science Edition). 2019, 42(1): 73-81.
35	白傑, 郝培涵, 陈思汉. 用轻量化卷积神经网络图像语义分割的交通场景理解[J]. 汽车安全与节能学报, 2018, 9(4): 433-440.
	Bai J, Hao P, Chen S. Traffic scene understanding using image semantic segmentation with an improved lightweight convolutional-neural-network[J]. Journal of Automotive Safety and Energy. 2018, 9(4): 433-440.
36	毕鹏程, 罗健欣, 陈卫卫. 轻量化卷积神经网络技术研究[J]. 计算机工程与应用, 2019, 55(16): 25-35.
	Bi P, Luo J, Chen W. Research on lightweight convolutional neural network technology[J]. Computer Engineering and Applications. 2019, 55(16): 25-35.
37	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 7132-7141.
38	Lin T Y, Maire M, Belongie S, et al. Microsoft coco: common objects in context[C]// European Conference on Computer Vision. Springer, Cham, 2014: 740-755.

模型名称	检测指标
	检测精度			模型容量
	平均精度（%）	误检率（%）	漏检率（%）	模型体积（MB）	参数量（个）
SSD	85.0	14.9	12.4	90.6	23.7×10⁶
CenterNet	88.3	11.9	7.3	60.3	15.8×10⁶
M-CenterNet	88.9	10.9	5.8	14.2	33.5×10⁵

网络模型	总耗时（s）	单帧耗时（ms）	帧率（fps）
SSD	35.0	241.5	4.1
CenterNet	30.9	212.8	4.7
M-CenterNet	17.8	123.0	8.1

[1]	李志军, 杨圣慧, 史德帅, 刘星星, 郑永军. 基于轻量化改进YOLOv5的苹果树产量测定方法[J]. 智慧农业（中英文）, 2021, 3(2): 100-114.
[2]	. 基于无人机图像以及不同机器学习和深度学习模型的小麦倒伏率检测[J]. 智慧农业（中英文）, 2021, 3(2): 23-34.
[3]	邱文杰, 叶进, 胡亮青, 杨娟, 李其利, 莫贱友, 易万茂. 面向植物病害识别的卷积神经网络精简结构Distilled-MobileNet模型[J]. 智慧农业（中英文）, 2021, 3(1): 109-117.
[4]	李亮德, 王秀娟, 康孟珍, 华净, 樊梦涵. 基于语义融合与模型蒸馏的农业实体识别[J]. 智慧农业（中英文）, 2021, 3(1): 118-128.
[5]	韩书庆, 张晶, 程国栋, 彭英琦, 张建华, 吴建寨. 奶牛跛行自动识别技术研究现状与挑战[J]. 智慧农业（中英文）, 2020, 2(3): 21-36.
[6]	郭威, 吴华瑞, 朱华吉. 设施温室影像采集与环境监测机器人系统设计及应用[J]. 智慧农业（中英文）, 2020, 2(3): 48-60.
[7]	李华龙, 李淼, 詹凯, 刘先旺, 杨选将, 胡泽林, 郭盼盼. 蛋鸡设施养殖环境质量评价预测模型构建方法及性能测试[J]. 智慧农业（中英文）, 2020, 2(3): 37-47.
[8]	魏靖, 王玉亭, 袁会珠, 张梦蕾, 王振营. 基于深度学习与特征可视化方法的草地贪夜蛾及其近缘种成虫识别[J]. 智慧农业（中英文）, 2020, 2(3): 75-85.
[9]	FLORESPaulo, MATHEWJithin, JAHANNusrat, STENGERJohn. 利用图像和机器学习检测大豆作物幼苗期玉米杂苗[J]. 智慧农业（中英文）, 2020, 2(3): 61-74.
[10]	周成全, 叶宏宝, 俞国红, 胡俊, 徐志福. 基于机器视觉与深度学习的西兰花表型快速提取方法研究[J]. 智慧农业（中英文）, 2020, 2(1): 121-132.
[11]	吴华瑞. 基于深度残差网络的番茄叶片病害识别方法[J]. 智慧农业, 2019, 1(4): 42-49.
[12]	陈桂芬, 赵姗, 曹丽英, 傅思维, 周佳鑫. 基于迁移学习与卷积神经网络的玉米植株病害识别[J]. 智慧农业, 2019, 1(2): 34-44.
[13]	周超, 徐大明, 吝凯, 陈澜, 张松, 孙传恒, 杨信廷. 基于近红外机器视觉的鱼类摄食强度评估方法研究[J]. 智慧农业, 2019, 1(1): 76-84.

基于轻量级无锚点深度卷积神经网络的树上苹果检测模型

Apple detection model based on lightweight anchor-free deep convolutional neural network

在线阅读

知网下载
（推荐）

本地下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 38

相关文章 13

编辑推荐

Metrics

本文评价

基于轻量级无锚点深度卷积神经网络的树上苹果检测模型

Apple detection model based on lightweight anchor-free deep convolutional neural network

在线阅读

知网下载（推荐）

本地下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 38

相关文章 13

编辑推荐

Metrics

本文评价

知网下载
（推荐）