Identification and Counting of Silkworms in Factory Farm Using Improved Mask R-CNN Model

doi:10.12133/j.smartag.SA202201012

Abstract

Abstract:

Factory-like rearing of silkworm (Bombyx mori) using artificial diet for all instars is a brand-new rearing mode of silkworm. Accurate feeding is one of the core technologies to save cost and increase efficiency in factory silkworm rearing. Automatic identification and counting of silkworm play a key role to realize accurate feeding. In this study, a machine vision system was used to obtain digital images of silkworms during main instars, and an improved Mask R-CNN model was proposed to detect the silkworms and residual artificial diet. The original Mask R-CNN was improved using the noise data of annotations by adding a pixel reweighting strategy and a bounding box fine-tuning strategy to the model frame. A more robust model was trained to improve the detection and segmentation abilities of silkworm and residual feed. Three different data augmentation methods were used to expand the training dataset. The influences of silkworm instars, data augmentation, and the overlap between silkworms on the model performance were evaluated. Then the improved Mask R-CNN was used to detect silkworms and residual feed. The AP50 (Average Precision at IoU=0.5) of the model for silkworm detection and segmentation were 0.790 and 0.795, respectively, and the detection accuracy was 96.83%. The detection and segmentation AP50 of residual feed were 0.641 and 0.653, respectively, and the detection accuracy was 87.71%. The model was deployed on the NVIDIA Jetson AGX Xavier development board with an average detection time of 1.32 s and a maximum detection time of 2.05 s for a image. The computational speed of the improved Mask R-CNN can meet the requirement of real-time detection of the moving unit of the silkworm box on the production line. The model trained by the fifth instar data showed a better performance on test data than the fourth instar model. The brightness enhancement method had the greatest contribution to the model performance as compared to the other data augmentation methods. The overlap between silkworms also negatively affected the performance of the model. This study can provide a core algorithm for the research and development of the accurate feeding information system and feeding device for factory silkworm rearing, which can improve the utilization rate of artificial diet and improve the production and management level of factory silkworm rearing.

Key words: silkworm, artificial diet, accurate feeding, machine vision, deep learning, mask R-CNN, noise data

CLC Number:

S126

HE Ruimin, ZHENG Kefeng, WEI Qinyang, ZHANG Xiaobin, ZHANG Jun, ZHU Yihang, ZHAO Yiying, GU Qing. Identification and Counting of Silkworms in Factory Farm Using Improved Mask R-CNN Model[J]. Smart Agriculture, 2022, 4(2): 163-173.

Figures/Tables 9

Fig. 1

Fig. 2

Fig. 3

Table 1

Fig. 4

Table 2

Table 3

Fig. 5

Table 4

References 22

1	TANAKA Y, SUDO M. Studies on the technology of artifical diet rearing for parental strains of the silkworm, 4: The relationship between the water content of artificial diets for the fifth larval instar and egg laying results[J]. Journal of Dainippon Silk Foundation, 2006, 53: 1-5.
2	吴亚群, 张升祥, 王洪江, 等. 家蚕不同品种对人工饲料摄食性的遗传模式[J]. 蚕业科学, 2017, 43(4): 603-609.
	WU Y, ZHANG S, WANG H, et al. Inheritance pattern of feeding habit on artificial diet in different bombyx mori varieties[J]. Science of Sericulture, 2017, 43(4): 603-609.
3	钱秋杰, 陈伟国. 家蚕人工饲料研究与应用进展[J]. 蚕桑通报, 2016, 47(2): 11-14.
	QIAN Q, CHEN W. Research and application progress of artificial diet for silkworm[J]. Bulletin of Sericulture, 2016, 47(2): 11-14.
4	董久鸣, 潘美良, 吴海平. 加快推进蚕桑产业转型发展的思考——巴贝工厂化养蚕的探索与启示[J]. 蚕桑通报, 2018, 49(2): 14-16.
	DONG J, PAN M, WU H. Thinking on speeding up the transformation and development of sericulture Industry—The exploration and enlightenment based on the BABEI's silkworm rearing in the factory[J]. Bulletin of Sericulture, 2018, 49(2): 14-16.
5	王亮, 胡帅栋. 推进全龄人工饲料工厂化养蚕的巴贝模式[J]. 蚕桑通报, 2020, 51(1): 37-45.
	WANG L, HU S. Babe model of promoting industrial sericulture by feeding artificial diet of full larval stage[J]. Bulletin of Sericulture, 2020, 51(1): 37-45.
6	DOU J, LI J. Robust object detection based on deformable part model and improved scale invariant feature transform[J]. Optik-International Journal for Light and Electron Optics, 2013, 124(24): 6485-6492.
7	HONG G S, KIM B G, HWANG Y S, et al. Fast multi-feature pedestrian detection algorithm based on histogram of oriented gradient using discrete wavelet transform[J]. Multimedia Tools and Applications, 2015, 75(23): 1-17.
8	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25: 1097-1105.
9	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2014.
10	HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 37(9): 1904-1916.
11	GIRSHICK R. Fast R-CNN[C]// 2015 IEEE International Conference on Computer Vision. Piscataway, New York, USA: IEEE, 2015.
12	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(6): 1137-1149.
13	HE K, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]// IEEE International Conference on Computer Vision. Piscataway, New York, USA: IEEE, 2017.
14	ZHOU C, HU J, XU Z, et al. A Novel greenhouse-based system for the detection and plumpness assessment of strawberry using an improved deep learning technique[J]. Frontiers in Plant Science, 2020, 11: ID 559.
15	TIAN Y, YANG G, WANG Z, et al. Apple detection during different growth stages in orchards using the improved YOLO-V3 model[J]. Computers and Electronics in Agriculture, 2019, 157: 417-426.
16	张远琴, 肖德琴, 陈焕坤, 等. 基于改进Faster R-CNN的水稻稻穗检测方法[J]. 农业机械学报, 52(8): 231-240.
	ZHANG Y, XIAO D, CHEN H, et al. Rice panicle detection method based on improved Faster R-CNN[J]. Transactions of the CSAM, 52(8): 231-240.
17	WEN Q, LUO Z, CHEN R, et al. Deep learning approaches on defect detection in high resolution aerial images of insulators[J]. Sensors, 2021, 21(4): ID 1033.
18	WJPD A, YT A, RONG L.B, et al. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot[J]. Computers and Electronics in Agriculture, 2020, 172(6): ID 105380.
19	M.AKHAN, ZHANG Y D, SHARIF M, et al. Pixels to classes: Intelligent learning framework for multiclass skin lesion localization and classification[J]. Computers and Electrical Engineering, 2021, 90: 1-20.
20	XU Y, ZHU L, YANG Y, et al. Training robust object detectors from noisy category labels and imprecise bounding boxes[J]. IEEE Transactions on Image Processing, 2021, 30: 5782-5792.
21	ZHANG Y, CHU J, LENG L, et al. Mask-refined R-CNN: A network for refining object details in instance segmentation[J]. Sensors, 2020, 20(4): ID 1010.
22	TIAN Y, YANG G, WANG Z, et al. Instance segmentation of apple flowers using the improved mask R-CNN model[J]. Biosystems Engineering, 2020, 193: 264-278.

模型	目标	准确率/%	检测 AP50	分割AP50
Mask R-CNN	蚕	95.23	0.764	0.768
Mask R-CNN	饲料	85.35	0.602	0.611
改进Mask R-CNN	蚕	96.83	0.790	0.795
改进Mask R-CNN	饲料	87.71	0.641	0.653

龄期	准确率/%	检测AP50	分割AP50
4龄	95.21	0.768	0.774
5龄	97.82	0.801	0.815

数据集	准确率/%	检测 AP50	分割 AP50
完整数据集	96.83	0.790	0.795
剔除旋转与翻转处理数据	94.79	0.773	0.771
剔除亮度增强数据	93.34	0.747	0.751
剔除噪声添加数据	96.02	0.785	0.791

[1]	WU Zhangbin, HE Ning, WU Yandong, GUO Xinyu, WEN Weiliang. Point Cloud Data-driven Methods for Estimating Maize Leaf Biomass [J]. Smart Agriculture, 2026, 8(1): 156-166.
[2]	YAO Xiaotong, QU Shaoye. Lightweight Detection Method for Pepper Leaf Diseases and Pests Based on Improved YOLOv12s [J]. Smart Agriculture, 2026, 8(1): 1-14.
[3]	ZHANG Yun, ZHANG Lumin, XU Guangtao, HAO Jiahui. Remote Sensing Extraction Method of Rice-Crayfish Fields Based on Dual-Branch and Multi-Scale Attention [J]. Smart Agriculture, 2025, 7(6): 185-195.
[4]	ZHAO Jun, NIE Zhigang, LI Guang, LIU Jiayu. Corn Borer Pests Infestations Detection Method Using Low-Altitude Close-Range UAV Imagery [J]. Smart Agriculture, 2025, 7(6): 111-123.
[5]	LI Wenzheng, YANG Xinting, SUN Chuanheng, CUI Tengpeng, WANG Hui, LI Shanshan, LI Wenyong. Light-Trapping Rice Planthopper Detection Method by Combining Spatial Depth Transform Convolution and Multi-scale Attention Mechanism [J]. Smart Agriculture, 2025, 7(5): 169-181.
[6]	HAN Wenkai, LI Tao, FENG Qingchun, CHEN Liping. Lightweight Apple Instance Segmentation Algorithm Based on SSW-YOLOv11n for Complex Orchard Environments [J]. Smart Agriculture, 2025, 7(5): 114-123.
[7]	WANG Fengyun, WANG Xuanyu, AN Lei, FENG Wenjie. Detection Method for Log-Cultivated Shiitake Mushrooms Based on Improved RT-DETR [J]. Smart Agriculture, 2025, 7(5): 67-77.
[8]	ZHAO Yingping, LIANG Jinming, CHEN Beizhang, DENG Xiaoling, ZHANG Yi, XIONG Zheng, PAN Ming, MENG Xiangbao. Applications Research Progress and Prospects of Multi-Agent Large Language Models in Agricultural [J]. Smart Agriculture, 2025, 7(5): 37-51.
[9]	HU Yan, WANG Yujie, ZHANG Xuechen, ZHANG Yiqiang, YU Huahao, SONG Xinbei, YE Sitan, ZHOU Jihong, CHEN Zhenlin, ZONG Weiwei, HE Yong, LI Xiaoli. Non-Destructive Inspection and Intelligent Grading Method of Fu Brick Tea at Fungal Fermentation Stage Based on Hyperspectral Imaging Technology [J]. Smart Agriculture, 2025, 7(4): 71-83.
[10]	GUO Qi, FAN Yixuan, YAN Xinhuan, LIU Xuemei, CAO Ning, WANG Zhen, PAN Shaoxiang, TAN Mengnan, ZHENG Xiaodong, SONG Ye. Advances in the Application of Multi-source Data Fusion Technology in Non-Destructive Detection of Apple [J]. Smart Agriculture, 2025, 7(4): 31-46.
[11]	LI Ruijie, WANG Aidong, WU Huaxing, LI Ziqiu, FENG Xiangqian, HONG Weiyuan, TANG Xuejun, QIN Jinhua, WANG Danying, CHU Guang, ZHANG Yunbo, CHEN Song. Remote Sensing for Rice Growth Stages Monitoring: Research Progress, Bottleneck Problems and Technical Optimization Paths [J]. Smart Agriculture, 2025, 7(3): 89-107.
[12]	HAN Yu, QI Kangkang, ZHENG Jiye, LI Jinai, JIANG Fugui, ZHANG Xianglun, YOU Wei, ZHANG Xia. Lightweight Cattle Facial Recognition Method Based on Improved YOLOv11 [J]. Smart Agriculture, 2025, 7(3): 173-184.
[13]	MA Liu, MAO Kebiao, GUO Zhonghua. Defogging Remote Sensing Images Method Based on a Hybrid Attention-Based Generative Adversarial Network [J]. Smart Agriculture, 2025, 7(2): 172-182.
[14]	XU Shiwei, LI Qianchuan, LUAN Rupeng, ZHUANG Jiayu, LIU Jiajia, XIONG Lu. Agricultural Market Monitoring and Early Warning: An Integrated Forecasting Approach Based on Deep Learning [J]. Smart Agriculture, 2025, 7(1): 57-69.
[15]	GONG Yu, WANG Ling, ZHAO Rongqiang, YOU Haibo, ZHOU Mo, LIU Jie. Tomato Growth Height Prediction Method by Phenotypic Feature Extraction Using Multi-modal Data [J]. Smart Agriculture, 2025, 7(1): 97-110.