Identification and Counting of Silkworms in Factory Farm Using Improved Mask R-CNN Model

doi:10.12133/j.smartag.SA202201012

Abstract

Abstract:

Factory-like rearing of silkworm (Bombyx mori) using artificial diet for all instars is a brand-new rearing mode of silkworm. Accurate feeding is one of the core technologies to save cost and increase efficiency in factory silkworm rearing. Automatic identification and counting of silkworm play a key role to realize accurate feeding. In this study, a machine vision system was used to obtain digital images of silkworms during main instars, and an improved Mask R-CNN model was proposed to detect the silkworms and residual artificial diet. The original Mask R-CNN was improved using the noise data of annotations by adding a pixel reweighting strategy and a bounding box fine-tuning strategy to the model frame. A more robust model was trained to improve the detection and segmentation abilities of silkworm and residual feed. Three different data augmentation methods were used to expand the training dataset. The influences of silkworm instars, data augmentation, and the overlap between silkworms on the model performance were evaluated. Then the improved Mask R-CNN was used to detect silkworms and residual feed. The AP50 (Average Precision at IoU=0.5) of the model for silkworm detection and segmentation were 0.790 and 0.795, respectively, and the detection accuracy was 96.83%. The detection and segmentation AP50 of residual feed were 0.641 and 0.653, respectively, and the detection accuracy was 87.71%. The model was deployed on the NVIDIA Jetson AGX Xavier development board with an average detection time of 1.32 s and a maximum detection time of 2.05 s for a image. The computational speed of the improved Mask R-CNN can meet the requirement of real-time detection of the moving unit of the silkworm box on the production line. The model trained by the fifth instar data showed a better performance on test data than the fourth instar model. The brightness enhancement method had the greatest contribution to the model performance as compared to the other data augmentation methods. The overlap between silkworms also negatively affected the performance of the model. This study can provide a core algorithm for the research and development of the accurate feeding information system and feeding device for factory silkworm rearing, which can improve the utilization rate of artificial diet and improve the production and management level of factory silkworm rearing.

Key words: silkworm, artificial diet, accurate feeding, machine vision, deep learning, mask R-CNN, noise data

CLC Number:

S126

HE Ruimin, ZHENG Kefeng, WEI Qinyang, ZHANG Xiaobin, ZHANG Jun, ZHU Yihang, ZHAO Yiying, GU Qing. Identification and Counting of Silkworms in Factory Farm Using Improved Mask R-CNN Model[J]. Smart Agriculture, 2022, 4(2): 163-173.

Figures/Tables 9

Fig. 1

Fig. 2

Fig. 3

Table 1

Fig. 4

Table 2

Table 3

Fig. 5

Table 4

References 22

1	TANAKA Y, SUDO M. Studies on the technology of artifical diet rearing for parental strains of the silkworm, 4: The relationship between the water content of artificial diets for the fifth larval instar and egg laying results[J]. Journal of Dainippon Silk Foundation, 2006, 53: 1-5.
2	吴亚群, 张升祥, 王洪江, 等. 家蚕不同品种对人工饲料摄食性的遗传模式[J]. 蚕业科学, 2017, 43(4): 603-609.
	WU Y, ZHANG S, WANG H, et al. Inheritance pattern of feeding habit on artificial diet in different bombyx mori varieties[J]. Science of Sericulture, 2017, 43(4): 603-609.
3	钱秋杰, 陈伟国. 家蚕人工饲料研究与应用进展[J]. 蚕桑通报, 2016, 47(2): 11-14.
	QIAN Q, CHEN W. Research and application progress of artificial diet for silkworm[J]. Bulletin of Sericulture, 2016, 47(2): 11-14.
4	董久鸣, 潘美良, 吴海平. 加快推进蚕桑产业转型发展的思考——巴贝工厂化养蚕的探索与启示[J]. 蚕桑通报, 2018, 49(2): 14-16.
	DONG J, PAN M, WU H. Thinking on speeding up the transformation and development of sericulture Industry—The exploration and enlightenment based on the BABEI's silkworm rearing in the factory[J]. Bulletin of Sericulture, 2018, 49(2): 14-16.
5	王亮, 胡帅栋. 推进全龄人工饲料工厂化养蚕的巴贝模式[J]. 蚕桑通报, 2020, 51(1): 37-45.
	WANG L, HU S. Babe model of promoting industrial sericulture by feeding artificial diet of full larval stage[J]. Bulletin of Sericulture, 2020, 51(1): 37-45.
6	DOU J, LI J. Robust object detection based on deformable part model and improved scale invariant feature transform[J]. Optik-International Journal for Light and Electron Optics, 2013, 124(24): 6485-6492.
7	HONG G S, KIM B G, HWANG Y S, et al. Fast multi-feature pedestrian detection algorithm based on histogram of oriented gradient using discrete wavelet transform[J]. Multimedia Tools and Applications, 2015, 75(23): 1-17.
8	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25: 1097-1105.
9	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2014.
10	HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 37(9): 1904-1916.
11	GIRSHICK R. Fast R-CNN[C]// 2015 IEEE International Conference on Computer Vision. Piscataway, New York, USA: IEEE, 2015.
12	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(6): 1137-1149.
13	HE K, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]// IEEE International Conference on Computer Vision. Piscataway, New York, USA: IEEE, 2017.
14	ZHOU C, HU J, XU Z, et al. A Novel greenhouse-based system for the detection and plumpness assessment of strawberry using an improved deep learning technique[J]. Frontiers in Plant Science, 2020, 11: ID 559.
15	TIAN Y, YANG G, WANG Z, et al. Apple detection during different growth stages in orchards using the improved YOLO-V3 model[J]. Computers and Electronics in Agriculture, 2019, 157: 417-426.
16	张远琴, 肖德琴, 陈焕坤, 等. 基于改进Faster R-CNN的水稻稻穗检测方法[J]. 农业机械学报, 52(8): 231-240.
	ZHANG Y, XIAO D, CHEN H, et al. Rice panicle detection method based on improved Faster R-CNN[J]. Transactions of the CSAM, 52(8): 231-240.
17	WEN Q, LUO Z, CHEN R, et al. Deep learning approaches on defect detection in high resolution aerial images of insulators[J]. Sensors, 2021, 21(4): ID 1033.
18	WJPD A, YT A, RONG L.B, et al. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot[J]. Computers and Electronics in Agriculture, 2020, 172(6): ID 105380.
19	M.AKHAN, ZHANG Y D, SHARIF M, et al. Pixels to classes: Intelligent learning framework for multiclass skin lesion localization and classification[J]. Computers and Electrical Engineering, 2021, 90: 1-20.
20	XU Y, ZHU L, YANG Y, et al. Training robust object detectors from noisy category labels and imprecise bounding boxes[J]. IEEE Transactions on Image Processing, 2021, 30: 5782-5792.
21	ZHANG Y, CHU J, LENG L, et al. Mask-refined R-CNN: A network for refining object details in instance segmentation[J]. Sensors, 2020, 20(4): ID 1010.
22	TIAN Y, YANG G, WANG Z, et al. Instance segmentation of apple flowers using the improved mask R-CNN model[J]. Biosystems Engineering, 2020, 193: 264-278.

模型	目标	准确率/%	检测 AP50	分割AP50
Mask R-CNN	蚕	95.23	0.764	0.768
Mask R-CNN	饲料	85.35	0.602	0.611
改进Mask R-CNN	蚕	96.83	0.790	0.795
改进Mask R-CNN	饲料	87.71	0.641	0.653

龄期	准确率/%	检测AP50	分割AP50
4龄	95.21	0.768	0.774
5龄	97.82	0.801	0.815

数据集	准确率/%	检测 AP50	分割 AP50
完整数据集	96.83	0.790	0.795
剔除旋转与翻转处理数据	94.79	0.773	0.771
剔除亮度增强数据	93.34	0.747	0.751
剔除噪声添加数据	96.02	0.785	0.791

[1]	LI Hao, DU Yuqiu, XIAO Xingzhu, CHEN Yanxi. Remote Sensing Identification Method of Cultivated Land at Hill County of Sichuan Basin Based on Deep Learning [J]. Smart Agriculture, 2024, 6(3): 34-45.
[2]	NIE Ganggang, RAO Honghui, LI Zefeng, LIU Muhua. Severity Grading Model for Camellia Oleifera Anthracnose Infection Based on Improved YOLACT [J]. Smart Agriculture, 2024, 6(3): 138-147.
[3]	ZHANG Jing, ZHAO Zexuan, ZHAO Yanru, BU Hongchao, WU Xingyu. Oilseed Rape Sclerotinia in Hyperspectral Images Segmentation Method Based on Bi-GRU and Spatial-Spectral Information Fusion [J]. Smart Agriculture, 2024, 6(2): 40-48.
[4]	FAN Jiangchuan, WANG Yuanqiao, GOU Wenbo, CAI Shuangze, GUO Xinyu, ZHAO Chunjiang. Fast Extracting Method for Strawberry Leaf Age and Canopy Width Based on Instance Segmentation Technology [J]. Smart Agriculture, 2024, 6(2): 95-106.
[5]	PANG Chunhui, CHEN Peng, XIA Yi, ZHANG Jun, WANG Bing, ZOU Yan, CHEN Tianjiao, KANG Chenrui, LIANG Dong. HI-FPN: A Hierarchical Interactive Feature Pyramid Network for Accurate Wheat Lodging Localization Across Multiple Growth Periods [J]. Smart Agriculture, 2024, 6(2): 128-139.
[6]	ZHANG Yuyu, BING Shuying, JI Yuanhao, YAN Beibei, XU Jinpu. Grading Method of Fresh Cut Rose Flowers Based on Improved YOLOv8s [J]. Smart Agriculture, 2024, 6(2): 118-127.
[7]	ZHANG Jianhua, YAO Qiong, ZHOU Guomin, WU Wendi, XIU Xiaojie, WANG Jian. Intelligent Identification of Crop Agronomic Traits and Morphological Structure Phenotypes: A Review [J]. Smart Agriculture, 2024, 6(2): 14-27.
[8]	GUO Wang, YANG Yusen, WU Huarui, ZHU Huaji, MIAO Yisheng, GU Jingqiu. Big Models in Agriculture: Key Technologies, Application and Future Directions [J]. Smart Agriculture, 2024, 6(2): 1-13.
[9]	MAO Yongwen, HAN Junying, LIU Chengzhong. Automated Flax Seeds Testing Methods Based on Machine Vision [J]. Smart Agriculture, 2024, 6(1): 135-146.
[10]	ZHOU Huamao, WANG Jing, YIN Hua, CHEN Qi. Phenotype Analysis of Pleurotus Geesteranus Based on Improved Mask R-CNN [J]. Smart Agriculture, 2023, 5(4): 117-126.
[11]	WANG Herong, CHEN Yingyi, CHAI Yingqian, XU Ling, YU Huihui. Image Segmentation Method Combined with VoVNetv2 and Shuffle Attention Mechanism for Fish Feeding in Aquaculture [J]. Smart Agriculture, 2023, 5(4): 137-149.
[12]	LI Zhengkai, YU Jiahui, PAN Shijia, JIA Zefeng, NIU Zijie. Individual Tree Skeleton Extraction and Crown Prediction Method of Winter Kiwifruit Trees [J]. Smart Agriculture, 2023, 5(4): 92-104.
[13]	TANG Hui, WANG Ming, YU Qiushi, ZHANG Jiaxi, LIU Liantao, WANG Nan. Root Image Segmentation Method Based on Improved UNet and Transfer Learning [J]. Smart Agriculture, 2023, 5(3): 96-109.
[14]	PAN Weiting, SUN Mengli, YUN Yan, LIU Ping. Identification Method of Wheat Grain Phenotype Based on Deep Learning of ImCascade R-CNN [J]. Smart Agriculture, 2023, 5(3): 110-120.
[15]	GUAN Bolun, ZHANG Liping, ZHU Jingbo, LI Runmei, KONG Juanjuan, WANG Yan, DONG Wei. The Key Issues and Evaluation Methods for Constructing Agricultural Pest and Disease Image Datasets: A Review [J]. Smart Agriculture, 2023, 5(3): 17-34.