基于改进Mask R-CNN模型的工厂化养蚕蚕体识别与计数

doi:10.12133/j.smartag.SA202201012

摘要/Abstract

摘要：

精准饲喂是全龄人工饲料工厂化养蚕节本增效的核心技术之一，家蚕自动化识别与计数是实现精准饲喂的关键环节。本研究基于机器视觉系统获取工厂化养蚕过程中蚕在4龄和5龄期的数字图像，利用改进深度学习模型Mask R-CNN检测蚕体和残余饲料。通过在Mask R-CNN模型框架中加入像素重加权策略和边界框细调策略，从噪声数据中训练一个鲁棒性更好的目标检测模型，实现模型性能的优化，提高对蚕体和饲料边界的检测和分割能力。改进Mask R-CNN模型对蚕的检测和分割交并比阈值为0.5时的平均精度（Average Precision at IoU=0.5，AP50）分别为0.790和0.795，识别准确率为96.83%；对残余饲料的检测和分割AP50分别为0.641和0.653，识别准确率为87.71%。模型部署在NVIDIA Jetson AGX Xavier开发板上，单张图像平均检测时间为1.32 s，最长检测时间为2.05 s，运算速度可以满足养蚕盒单元在生产线上移动实时检测的要求。该研究为工厂化养蚕精准饲喂信息系统和投喂装置的研发提供了核心算法，可提高人工饲料的利用率，提升工厂化养蚕生产管理水平。

关键词: 家蚕, 人工饲料, 精准饲喂, 机器视觉, 深度学习, mask R-CNN, 噪声数据

Abstract:

Factory-like rearing of silkworm (Bombyx mori) using artificial diet for all instars is a brand-new rearing mode of silkworm. Accurate feeding is one of the core technologies to save cost and increase efficiency in factory silkworm rearing. Automatic identification and counting of silkworm play a key role to realize accurate feeding. In this study, a machine vision system was used to obtain digital images of silkworms during main instars, and an improved Mask R-CNN model was proposed to detect the silkworms and residual artificial diet. The original Mask R-CNN was improved using the noise data of annotations by adding a pixel reweighting strategy and a bounding box fine-tuning strategy to the model frame. A more robust model was trained to improve the detection and segmentation abilities of silkworm and residual feed. Three different data augmentation methods were used to expand the training dataset. The influences of silkworm instars, data augmentation, and the overlap between silkworms on the model performance were evaluated. Then the improved Mask R-CNN was used to detect silkworms and residual feed. The AP50 (Average Precision at IoU=0.5) of the model for silkworm detection and segmentation were 0.790 and 0.795, respectively, and the detection accuracy was 96.83%. The detection and segmentation AP50 of residual feed were 0.641 and 0.653, respectively, and the detection accuracy was 87.71%. The model was deployed on the NVIDIA Jetson AGX Xavier development board with an average detection time of 1.32 s and a maximum detection time of 2.05 s for a image. The computational speed of the improved Mask R-CNN can meet the requirement of real-time detection of the moving unit of the silkworm box on the production line. The model trained by the fifth instar data showed a better performance on test data than the fourth instar model. The brightness enhancement method had the greatest contribution to the model performance as compared to the other data augmentation methods. The overlap between silkworms also negatively affected the performance of the model. This study can provide a core algorithm for the research and development of the accurate feeding information system and feeding device for factory silkworm rearing, which can improve the utilization rate of artificial diet and improve the production and management level of factory silkworm rearing.

Key words: silkworm, artificial diet, accurate feeding, machine vision, deep learning, mask R-CNN, noise data

中图分类号:

S126

何锐敏, 郑可锋, 尉钦洋, 张小斌, 张俊, 朱怡航, 赵懿滢, 顾清. 基于改进Mask R-CNN模型的工厂化养蚕蚕体识别与计数[J]. 智慧农业(中英文), 2022, 4(2): 163-173.

HE Ruimin, ZHENG Kefeng, WEI Qinyang, ZHANG Xiaobin, ZHANG Jun, ZHU Yihang, ZHAO Yiying, GU Qing. Identification and Counting of Silkworms in Factory Farm Using Improved Mask R-CNN Model[J]. Smart Agriculture, 2022, 4(2): 163-173.

图/表 9

图1

图2

图3

表1

图 4

表2

表3

图5

表4

参考文献 22

1	TANAKA Y, SUDO M. Studies on the technology of artifical diet rearing for parental strains of the silkworm, 4: The relationship between the water content of artificial diets for the fifth larval instar and egg laying results[J]. Journal of Dainippon Silk Foundation, 2006, 53: 1-5.
2	吴亚群, 张升祥, 王洪江, 等. 家蚕不同品种对人工饲料摄食性的遗传模式[J]. 蚕业科学, 2017, 43(4): 603-609.
	WU Y, ZHANG S, WANG H, et al. Inheritance pattern of feeding habit on artificial diet in different bombyx mori varieties[J]. Science of Sericulture, 2017, 43(4): 603-609.
3	钱秋杰, 陈伟国. 家蚕人工饲料研究与应用进展[J]. 蚕桑通报, 2016, 47(2): 11-14.
	QIAN Q, CHEN W. Research and application progress of artificial diet for silkworm[J]. Bulletin of Sericulture, 2016, 47(2): 11-14.
4	董久鸣, 潘美良, 吴海平. 加快推进蚕桑产业转型发展的思考——巴贝工厂化养蚕的探索与启示[J]. 蚕桑通报, 2018, 49(2): 14-16.
	DONG J, PAN M, WU H. Thinking on speeding up the transformation and development of sericulture Industry—The exploration and enlightenment based on the BABEI's silkworm rearing in the factory[J]. Bulletin of Sericulture, 2018, 49(2): 14-16.
5	王亮, 胡帅栋. 推进全龄人工饲料工厂化养蚕的巴贝模式[J]. 蚕桑通报, 2020, 51(1): 37-45.
	WANG L, HU S. Babe model of promoting industrial sericulture by feeding artificial diet of full larval stage[J]. Bulletin of Sericulture, 2020, 51(1): 37-45.
6	DOU J, LI J. Robust object detection based on deformable part model and improved scale invariant feature transform[J]. Optik-International Journal for Light and Electron Optics, 2013, 124(24): 6485-6492.
7	HONG G S, KIM B G, HWANG Y S, et al. Fast multi-feature pedestrian detection algorithm based on histogram of oriented gradient using discrete wavelet transform[J]. Multimedia Tools and Applications, 2015, 75(23): 1-17.
8	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25: 1097-1105.
9	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2014.
10	HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 37(9): 1904-1916.
11	GIRSHICK R. Fast R-CNN[C]// 2015 IEEE International Conference on Computer Vision. Piscataway, New York, USA: IEEE, 2015.
12	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(6): 1137-1149.
13	HE K, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]// IEEE International Conference on Computer Vision. Piscataway, New York, USA: IEEE, 2017.
14	ZHOU C, HU J, XU Z, et al. A Novel greenhouse-based system for the detection and plumpness assessment of strawberry using an improved deep learning technique[J]. Frontiers in Plant Science, 2020, 11: ID 559.
15	TIAN Y, YANG G, WANG Z, et al. Apple detection during different growth stages in orchards using the improved YOLO-V3 model[J]. Computers and Electronics in Agriculture, 2019, 157: 417-426.
16	张远琴, 肖德琴, 陈焕坤, 等. 基于改进Faster R-CNN的水稻稻穗检测方法[J]. 农业机械学报, 52(8): 231-240.
	ZHANG Y, XIAO D, CHEN H, et al. Rice panicle detection method based on improved Faster R-CNN[J]. Transactions of the CSAM, 52(8): 231-240.
17	WEN Q, LUO Z, CHEN R, et al. Deep learning approaches on defect detection in high resolution aerial images of insulators[J]. Sensors, 2021, 21(4): ID 1033.
18	WJPD A, YT A, RONG L.B, et al. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot[J]. Computers and Electronics in Agriculture, 2020, 172(6): ID 105380.
19	M.AKHAN, ZHANG Y D, SHARIF M, et al. Pixels to classes: Intelligent learning framework for multiclass skin lesion localization and classification[J]. Computers and Electrical Engineering, 2021, 90: 1-20.
20	XU Y, ZHU L, YANG Y, et al. Training robust object detectors from noisy category labels and imprecise bounding boxes[J]. IEEE Transactions on Image Processing, 2021, 30: 5782-5792.
21	ZHANG Y, CHU J, LENG L, et al. Mask-refined R-CNN: A network for refining object details in instance segmentation[J]. Sensors, 2020, 20(4): ID 1010.
22	TIAN Y, YANG G, WANG Z, et al. Instance segmentation of apple flowers using the improved mask R-CNN model[J]. Biosystems Engineering, 2020, 193: 264-278.

龄期	准确率/%	检测AP50	分割AP50
4龄	95.21	0.768	0.774
5龄	97.82	0.801	0.815

数据集	准确率/%	检测 AP50	分割 AP50
完整数据集	96.83	0.790	0.795
剔除旋转与翻转处理数据	94.79	0.773	0.771
剔除亮度增强数据	93.34	0.747	0.751
剔除噪声添加数据	96.02	0.785	0.791

[1]	毛克彪, 张晨阳, 施建成, 王旭明, 郭中华, 李春树, 董立新, 吴门新, 孙瑞静, 武胜利, 姬大彬, 蒋玲梅, 赵天杰, 邱玉宝, 杜永明, 徐同仁. 基于人工智能的地球物理参数反演范式理论及判定条件[J]. 智慧农业(中英文), 2023, 5(2): 161-171.
[2]	夏雪, 柴秀娟, 张凝, 周硕, 孙琦鑫, 孙坦. 用于边缘计算设备的果树挂果量轻量化估测模型[J]. 智慧农业(中英文), 2023, 5(2): 1-12.
[3]	郭阳阳, 杜书增, 乔永亮, 梁栋. 深度学习在家畜智慧养殖中研究应用进展[J]. 智慧农业(中英文), 2023, 5(1): 52-65.
[4]	左敏, 胡天宇, 董微, 张可心, 张青川. 基于Informer神经网络的农产品物流需求预测分析——以华中地区为例[J]. 智慧农业(中英文), 2023, 5(1): 34-43.
[5]	计洁, 金洲, 王儒敬, 刘海燕, 李志远. 基于递进式卷积网络的农业命名实体识别方法[J]. 智慧农业(中英文), 2023, 5(1): 122-131.
[6]	胡松涛, 翟瑞芳, 王应华, 刘志, 朱剑忠, 任荷, 杨万能, 宋鹏. 基于多源数据的马铃薯植株表型参数提取[J]. 智慧农业(中英文), 2023, 5(1): 132-145.
[7]	刘晓航, 张昭, 刘嘉滢, 张漫, 李寒, FLORES Paulo, 韩雄哲. 基于多种深度学习算法的田间玉米籽粒检测与计数[J]. 智慧农业(中英文), 2022, 4(4): 49-60.
[8]	许钰林, 康孟珍, 王秀娟, 华净, 王浩宇, 沈震. 基于深度学习的玉米和大豆期货价格智能预测[J]. 智慧农业(中英文), 2022, 4(4): 156-163.
[9]	罗庆, 饶元, 金秀, 江朝晖, 王坦, 王丰仪, 张武. 基于改进YOLOv5s和多模态图像的树上毛桃检测[J]. 智慧农业(中英文), 2022, 4(4): 84-104.
[10]	张志博, 赵西宁, 高晓东, 张利, 杨孟豪. 基于改进Linknet网络的黄土高原苹果园精准提取[J]. 智慧农业(中英文), 2022, 4(3): 95-107.
[11]	商枫楠, 周学成, 梁英凯, 肖明玮, 陈桥, 罗陈迪. 基于改进YOLOX的自然环境中火龙果检测方法[J]. 智慧农业(中英文), 2022, 4(3): 120-131.
[12]	庄家煜, 许世卫, 李杨, 熊露, 刘克宝, 钟志平. 基于深度学习的多种农产品供需预测模型[J]. 智慧农业(中英文), 2022, 4(2): 174-182.
[13]	邵明月, 张建华, 冯全, 柴秀娟, 张凝, 张文蓉. 深度学习在植物叶部病害检测与识别的研究进展[J]. 智慧农业(中英文), 2022, 4(1): 29-46.
[14]	郭秀明, 诸叶平, 李世娟, 张杰, 吕纯阳, 刘升平. 农业复杂环境下尺度自适应小目标识别算法——以蜜蜂为研究对象[J]. 智慧农业(中英文), 2022, 4(1): 140-149.
[15]	杨国峰, 何勇, 冯旭萍, 李禧尧, 张金诺, 俞泽宇. 无人机遥感监测作物病虫害胁迫方法与最新研究进展[J]. 智慧农业(中英文), 2022, 4(1): 1-16.