Scale Adaptive Small Objects Detection Method in Complex Agricultural Environment: Taking Bees as Research Object

doi:10.12133/j.smartag.SA202203003

Abstract

Abstract:

Objects in farmlands often have characteristic of small volume and high density with variable light and complex background, and the available object detection models could not get satisfactory recognition results. Taking bees as research objects, a method that could overcome the influence from the complex backgrounds, the difficulty in small object feature extraction was proposed, and a detection algorithm was created for small objects irrelevant to image size. Firstly, the original image was split into some smaller sub-images to increase the object scale, and the marked objects were assigned to the sub-images to produce a new dataset. Then, the model was trained again using transfer learning to get a new object detection model. A certain overlap rate was set between two adjacent sub-images in order to restore the objects. The objects from each sub-image was collected and then non-maximum suppression (NMS) was performed to delete the redundant detection boxes caused by the network, an improved NMS named intersection over small NMS (IOS-NMS) was then proposed to delete the redundant boxes caused by the overlap between adjacent sub-images. Validation tests were performed when sub-image size was set was 300×300, 500×500 and 700×700, the overlap rate was set as 0.2 and 0.05 respectively, and the results showed that when using single shot multibox detector (SSD) as the object detection model, the recall rate and precision was generally higher than that of SSD with the maximum difference 3.8% and 2.6%, respectively. In order to further verify the algorithm in small target recognition with complex background, three bee images with different scales and different scenarios were obtained from internet and test experiments were conducted using the new proposed algorithm and SSD. The results showed that the proposed algorithm could improve the performance of target detection and had strong scale adaptability and generalization. Besides, the new algorithm required multiple forward reasoning for a single image, so it was not time-efficient and was not suitable for edge calculation.

Key words: object detection, machine vision, small object, farmland, bee, single shot multibox detector, YOLOv3

CLC Number:

GUO Xiuming, ZHU Yeping, LI Shijuan, ZHANG Jie, LYU Chunyang, LIU Shengping. Scale Adaptive Small Objects Detection Method in Complex Agricultural Environment: Taking Bees as Research Object[J]. Smart Agriculture, 2022, 4(1): 140-149.

Figures/Tables 12

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 1

Fig. 9

Table 2

Fig. 10

References 25

1	NAUATA N, HU H, ZHOU G T, et al. Structured label inference for visual understanding[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 42(5): 1257-1271.
2	黄凯奇, 任伟强, 谭铁牛. 图像物体分类与检测算法综述[J]. 计算机学报, 2014, 36(12):1-18.
	HUANG K, REN W, TAN T. A review on image object classification and detection[J]. Chinese Journal of Computers, 2014, 36(12):1-18.
3	ZOU Z, SHI Z, GUO Y, et al. Object detection in 20 years: A survey[J/OL]. arXiv: 1905.05055v2 [cs.CV], 2019.
4	李科岑, 王晓强, 林浩, 等. 深度学习中的单阶段小目标检测方法综述[J]. 计算机科学与探索, 2022, 16(1): 41-58.
	LI K, WANG X, LIN H, et al. Survey of one stage small object detection methods in deep learning[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(1): 41-58.
5	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: Optimal speed and accuracy of object detection[J/OL]. arXiv: , 2020.
6	REDMON J, FARHADI A. YOLOv3: An incremental improvement[J/OL]. arXiv:1804.02767 [cs.CV], 2018.
7	MAHTO P, GARG P, SETH P, et al. Refining YOLOv4 for vehicle detection[J]. International Journal of Advanced Research in Engineering and Technology, 2020, 11(5): 409-419.
8	ZHAI S, SHANG D, WANG S, et al. DF-SSD: An improved SSD object detection algorithm based on denseNet and feature fusion[J]. IEEE Access, 2020: 24344-24357.
9	奚琦, 张正道, 彭力. 基于改进密集网络与二次回归的小目标检测算法[J]. 计算机工程, 2021, 47(4): 241-247, 255.
	XI Q, ZHANG Z, PENG L. Small object detection algorithm based on improved dense network and quadratic regression[J]. Computer Engineering, 2021, 47(4): 241-247, 255.
10	SHENZ Q, LIU Z, LI J G, et al. DSOD: Learning deeply supervised object detectors from scratch[C]// The 2017 IEEE International Conference on Computer Vision. Washington D. C., USA: IEEE Computer Society, 2017: 1937-1945.
11	李航, 朱明. 基于深度卷积神经网络的小目标检测算法[J]. 计算机工程与科学, 2020, 42(4): 649-657.
	LI H, ZHU M. A small object detection algorithm based on deep convolutional neural network[J]. Computer Engineering & Science, 2020, 42(4): 649-657.
12	周慧, 严凤龙, 褚娜, 等. 一种改进复杂场景下小目标检测模型的方法[J/OL]. 计算机工程与应用: 1-8. [2021-10-04]. .
	ZHOU H, YAN F, ZHU N, et al. An approach to improve the detection model for small object in complex scenes[J/OL]. Computer Engineering and Applications:1-8. [2021-10-04]. .
13	ESTER M, KRIEGEL H P, SANDER J, et al. A density-based algorithm for discovering clusters in large spatial databases with noise[C]// The Second International Conference on Knowledge Discovery and Data Mining. Portland, Oregon, USA: AAAI, 1996: 226-231.
14	李云红, 张轩, 李传真, 等. 融合DBSCAN的改进YOLOv3目标检测算法[J/OL]. 计算机工程与应用: 1-12. [2021-10-04]. .
	LI Y, ZHANG X, LI C, et al. Improved YOLOv3 target detection algorithm combined with DBSCAN[J/OL]. Computer Engineering and Applications: 1-12. [2021-10-04]. .
15	REZATOFIGHI H, TSOI N, GWAK J Y, et al. Generalized intersection over union: A metric and a loss for bounding box regression[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2019: 658-666.
16	YANG Y, LIAO Y, CHENG L, et al. Remote sensing image aircraft target detection based on GIoU-YOLOv3[C]// 2021 6th International Conference on Intelligent Computing and Signal Processing. Piscataway, New York, USA: IEEE, 2021: 474-478.
17	ZHENG Z, ZHAO H, LIU W, et al. Distance-IoU loss: Faster and better learning for bounding box regression[C]// The 34th AAAI Conference on Artificial Intelligence, the 32nd Innovative Applications of Artificial Intelligence Conference, the 10th AAAI Symposium on Educational Advances in Artificial Intelligence. Piscataway, New York, USA: AAAI, 2020: 12993-13000.
18	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, (99): 2999-3007.
19	张炳力, 秦浩然, 江尚, 等. 基于RetinaNet 及优化损失函数的夜间车辆检测方法[J]. 汽车工程, 2021, 43(8): 1195-1202.
	ZHANG B, QIN H, JIANG S, et al. A method of vehicle detection at night based on RetinaNet and optimized loss functions[J]. Automotive Engineering, 2021, 43(8): 1195-1202.
20	郑秋梅, 王璐璐, 王风华. 基于改进卷积神经网络的交通场景小目标检测[J]. 计算机工程, 2020, 46(6): 26-33.
	ZHENG Q, WANG L, WANG F. Small object detection in traffic scene based on improved convolutional neural network[J]. Computer Engineering, 2020, 46(6): 26-33.
21	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
22	LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector[C]// In European Conference on Computer Vision. Cham, Switzerland: Springer: 2016.
23	Neubeck A, Gool L J V. Efficient non-maximum suppression[C]// International Conference on Pattern Recognition. Piscataway, New York, USA: IEEE Computer Society, 2006: 848-855.
24	李景琳, 姜晶菲, 窦勇, 等. 基于Soft-NMS的候选框去冗余加速器设计[J]. 计算机工程与科学, 2021, 43(4): 586-593.
	LI J, JIANG J, DOU Y, et al. A redundacy-reduced candidate box accelerator based on soft-non-maximum suppression[J]. Computer Engineering & Science, 2021, 43(4): 586-593.
25	张长伦, 张翠文, 王恒友, 等. 基于注意力机制的NMS在目标检测中的研究[J]. 电子测量技术, 2021, 44(19): 82-88.
	ZHANG C, ZHANG C, WANG H, et al. Research on non-maximum suppression based on attention mechanism in object detection[J]. Electronic Measurement Technology, 2021, 44(19): 82-88.

性能指标	召回率/%	精度/%	平均单张图像识别时间/s
原尺度SSD模型	94.6	87.3	0.046
原尺度YOLOv3模型	96.2	88.1	0.059
尺度自适应新模型（zr = zc = 300， os = 0.2）	98.4	89.9	0.970
尺度自适应新模型（zr = zc = 300， os = 0.05）	98.4	88.3	0.810
尺度自适应新模型（zr = zc = 500， os = 0.2）	98.4	89.9	0.512
尺度自适应新模型（zr = zc = 500， os = 0.05）	98.2	89.2	0.362
尺度自适应新模型（zr = zc = 700， os = 0.2）	97.8	89.6	0.315
尺度自适应新模型（zr = zc = 700， os = 0.05）	97.1	89.5	0.227

图像名称	分辨率	蜜蜂尺度均值	环境
pic1	670×420	4.975e^-3	野外蜂巢
pic2	1440×1080	0.900e^-3	野外蜂箱侧面
pic3	1920×1280	0.559e^-3	野外蜂箱全景

[1]	HE Ruimin, ZHENG Kefeng, WEI Qinyang, ZHANG Xiaobin, ZHANG Jun, ZHU Yihang, ZHAO Yiying, GU Qing. Identification and Counting of Silkworms in Factory Farm Using Improved Mask R-CNN Model [J]. Smart Agriculture, 2022, 4(2): 163-173.
[2]	LI Jiawei, MA Weihong, LI Qifeng, XUE Xianglong, WANG Zhiquan. Automatic Acquisition and Target Extraction of Beef Cattle 3D Point Cloud from Complex Environment [J]. Smart Agriculture, 2022, 4(2): 64-76.
[3]	GUO Zhiming, WANG Junyi, SONG Ye, ZOU Xiaobo, CAI Jianrong. Research Progress of Sensing Detection and Monitoring Technology for Fruit and Vegetable Quality Control [J]. Smart Agriculture, 2021, 3(4): 14-28.
[4]	LONG Jiehua, GUO Wenzhong, LIN Sen, WEN Chaowu, ZHANG Yu, ZHAO Chunjiang. Strawberry Growth Period Recognition Method Under Greenhouse Environment Based on Improved YOLOv4 [J]. Smart Agriculture, 2021, 3(4): 99-110.
[5]	HAN Shuqing, ZHANG Jing, CHENG Guodong, PENG Yingqi, ZHANG Jianhua, WU Jianzhai. Current State and Challenges of Automatic Lameness Detection in Dairy Cattle [J]. Smart Agriculture, 2020, 2(3): 21-36.
[6]	GUO Wei, WU Huarui, ZHU Huaji. Design and Application of Facility Greenhouse Image Collecting and Environmental Data Monitoring Robot System [J]. Smart Agriculture, 2020, 2(3): 48-60.
[7]	SUN Haoran, SUN Lin, BI Chunguang, YU Helong. Hybrid Multi-Hop Routing Algorithm for Farmland IoT based on Particle Swarm and Simulated Annealing Collaborative Optimization Method [J]. Smart Agriculture, 2020, 2(3): 98-107.
[8]	HONG Wei, XU Baohua, LIU Shengping. Design and Experimental Research of Long-Term Monitoring System for Bee Colony Multiple Features [J]. Smart Agriculture, 2020, 2(2): 105-114.
[9]	YANG XuanJiang, LI Hualong, LI Miao, HU Zelin, LIAO Jianjun, LIU Xianwang, GUO Panpan, YUE Xudong. Beehive Key Parameters Online Monitoring System and Performance Test [J]. Smart Agriculture, 2020, 2(2): 115-125.
[10]	ZHANG Zhongxiong, LI Bin, FENG Pan, ZHANG Pan, LAI Haibin, HU Jin, ZHANG Haihui. Stereoscopic Light Environment Intelligent Control System Based on Characteristic Differences of Facility Cucumber Plants Light Requirements [J]. Smart Agriculture, 2020, 2(2): 94-104.
[11]	Xia Xue, Sun Qixin, Shi Xiao, Chai Xiujuan. Apple detection model based on lightweight anchor-free deep convolutional neural network [J]. Smart Agriculture, 2020, 2(1): 99-110.
[12]	Zhang Xiaohan, Yin Changchuan, Wu Huarui. Energy optimization strategy for wireless sensor networks in large-scale farmland habitat monitoring [J]. Smart Agriculture, 2019, 1(2): 55-63.
[13]	Zhou Chao, Xu Daming, Lin Kai, Chen Lan, Zhang Song, Sun Chuanheng, Yang Xinting. Evaluation of fish feeding intensity in aquaculture based on near-infrared machine vision [J]. Smart Agriculture, 2019, 1(1): 76-84.