Identification and Counting of Silkworms in Factory Farm Using Improved Mask R-CNN Model

doi:10.12133/j.smartag.SA202201012

Abstract

Abstract:

Factory-like rearing of silkworm (Bombyx mori) using artificial diet for all instars is a brand-new rearing mode of silkworm. Accurate feeding is one of the core technologies to save cost and increase efficiency in factory silkworm rearing. Automatic identification and counting of silkworm play a key role to realize accurate feeding. In this study, a machine vision system was used to obtain digital images of silkworms during main instars, and an improved Mask R-CNN model was proposed to detect the silkworms and residual artificial diet. The original Mask R-CNN was improved using the noise data of annotations by adding a pixel reweighting strategy and a bounding box fine-tuning strategy to the model frame. A more robust model was trained to improve the detection and segmentation abilities of silkworm and residual feed. Three different data augmentation methods were used to expand the training dataset. The influences of silkworm instars, data augmentation, and the overlap between silkworms on the model performance were evaluated. Then the improved Mask R-CNN was used to detect silkworms and residual feed. The AP50 (Average Precision at IoU=0.5) of the model for silkworm detection and segmentation were 0.790 and 0.795, respectively, and the detection accuracy was 96.83%. The detection and segmentation AP50 of residual feed were 0.641 and 0.653, respectively, and the detection accuracy was 87.71%. The model was deployed on the NVIDIA Jetson AGX Xavier development board with an average detection time of 1.32 s and a maximum detection time of 2.05 s for a image. The computational speed of the improved Mask R-CNN can meet the requirement of real-time detection of the moving unit of the silkworm box on the production line. The model trained by the fifth instar data showed a better performance on test data than the fourth instar model. The brightness enhancement method had the greatest contribution to the model performance as compared to the other data augmentation methods. The overlap between silkworms also negatively affected the performance of the model. This study can provide a core algorithm for the research and development of the accurate feeding information system and feeding device for factory silkworm rearing, which can improve the utilization rate of artificial diet and improve the production and management level of factory silkworm rearing.

Key words: silkworm, artificial diet, accurate feeding, machine vision, deep learning, mask R-CNN, noise data

CLC Number:

S126

HE Ruimin, ZHENG Kefeng, WEI Qinyang, ZHANG Xiaobin, ZHANG Jun, ZHU Yihang, ZHAO Yiying, GU Qing. Identification and Counting of Silkworms in Factory Farm Using Improved Mask R-CNN Model[J]. Smart Agriculture, 2022, 4(2): 163-173.

Figures/Tables 9

Fig. 1

Fig. 2

Fig. 3

Table 1

Fig. 4

Table 2

Table 3

Fig. 5

Table 4

References 22

1	TANAKA Y, SUDO M. Studies on the technology of artifical diet rearing for parental strains of the silkworm, 4: The relationship between the water content of artificial diets for the fifth larval instar and egg laying results[J]. Journal of Dainippon Silk Foundation, 2006, 53: 1-5.
2	吴亚群, 张升祥, 王洪江, 等. 家蚕不同品种对人工饲料摄食性的遗传模式[J]. 蚕业科学, 2017, 43(4): 603-609.
	WU Y, ZHANG S, WANG H, et al. Inheritance pattern of feeding habit on artificial diet in different bombyx mori varieties[J]. Science of Sericulture, 2017, 43(4): 603-609.
3	钱秋杰, 陈伟国. 家蚕人工饲料研究与应用进展[J]. 蚕桑通报, 2016, 47(2): 11-14.
	QIAN Q, CHEN W. Research and application progress of artificial diet for silkworm[J]. Bulletin of Sericulture, 2016, 47(2): 11-14.
4	董久鸣, 潘美良, 吴海平. 加快推进蚕桑产业转型发展的思考——巴贝工厂化养蚕的探索与启示[J]. 蚕桑通报, 2018, 49(2): 14-16.
	DONG J, PAN M, WU H. Thinking on speeding up the transformation and development of sericulture Industry—The exploration and enlightenment based on the BABEI's silkworm rearing in the factory[J]. Bulletin of Sericulture, 2018, 49(2): 14-16.
5	王亮, 胡帅栋. 推进全龄人工饲料工厂化养蚕的巴贝模式[J]. 蚕桑通报, 2020, 51(1): 37-45.
	WANG L, HU S. Babe model of promoting industrial sericulture by feeding artificial diet of full larval stage[J]. Bulletin of Sericulture, 2020, 51(1): 37-45.
6	DOU J, LI J. Robust object detection based on deformable part model and improved scale invariant feature transform[J]. Optik-International Journal for Light and Electron Optics, 2013, 124(24): 6485-6492.
7	HONG G S, KIM B G, HWANG Y S, et al. Fast multi-feature pedestrian detection algorithm based on histogram of oriented gradient using discrete wavelet transform[J]. Multimedia Tools and Applications, 2015, 75(23): 1-17.
8	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25: 1097-1105.
9	GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2014.
10	HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 37(9): 1904-1916.
11	GIRSHICK R. Fast R-CNN[C]// 2015 IEEE International Conference on Computer Vision. Piscataway, New York, USA: IEEE, 2015.
12	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(6): 1137-1149.
13	HE K, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]// IEEE International Conference on Computer Vision. Piscataway, New York, USA: IEEE, 2017.
14	ZHOU C, HU J, XU Z, et al. A Novel greenhouse-based system for the detection and plumpness assessment of strawberry using an improved deep learning technique[J]. Frontiers in Plant Science, 2020, 11: ID 559.
15	TIAN Y, YANG G, WANG Z, et al. Apple detection during different growth stages in orchards using the improved YOLO-V3 model[J]. Computers and Electronics in Agriculture, 2019, 157: 417-426.
16	张远琴, 肖德琴, 陈焕坤, 等. 基于改进Faster R-CNN的水稻稻穗检测方法[J]. 农业机械学报, 52(8): 231-240.
	ZHANG Y, XIAO D, CHEN H, et al. Rice panicle detection method based on improved Faster R-CNN[J]. Transactions of the CSAM, 52(8): 231-240.
17	WEN Q, LUO Z, CHEN R, et al. Deep learning approaches on defect detection in high resolution aerial images of insulators[J]. Sensors, 2021, 21(4): ID 1033.
18	WJPD A, YT A, RONG L.B, et al. Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot[J]. Computers and Electronics in Agriculture, 2020, 172(6): ID 105380.
19	M.AKHAN, ZHANG Y D, SHARIF M, et al. Pixels to classes: Intelligent learning framework for multiclass skin lesion localization and classification[J]. Computers and Electrical Engineering, 2021, 90: 1-20.
20	XU Y, ZHU L, YANG Y, et al. Training robust object detectors from noisy category labels and imprecise bounding boxes[J]. IEEE Transactions on Image Processing, 2021, 30: 5782-5792.
21	ZHANG Y, CHU J, LENG L, et al. Mask-refined R-CNN: A network for refining object details in instance segmentation[J]. Sensors, 2020, 20(4): ID 1010.
22	TIAN Y, YANG G, WANG Z, et al. Instance segmentation of apple flowers using the improved mask R-CNN model[J]. Biosystems Engineering, 2020, 193: 264-278.

模型	目标	准确率/%	检测 AP50	分割AP50
Mask R-CNN	蚕	95.23	0.764	0.768
Mask R-CNN	饲料	85.35	0.602	0.611
改进Mask R-CNN	蚕	96.83	0.790	0.795
改进Mask R-CNN	饲料	87.71	0.641	0.653

龄期	准确率/%	检测AP50	分割AP50
4龄	95.21	0.768	0.774
5龄	97.82	0.801	0.815

数据集	准确率/%	检测 AP50	分割 AP50
完整数据集	96.83	0.790	0.795
剔除旋转与翻转处理数据	94.79	0.773	0.771
剔除亮度增强数据	93.34	0.747	0.751
剔除噪声添加数据	96.02	0.785	0.791

[1]	MAO Kebiao, ZHANG Chenyang, SHI Jiancheng, WANG Xuming, GUO Zhonghua, LI Chunshu, DONG Lixin, WU Menxin, SUN Ruijing, WU Shengli, JI Dabin, JIANG Lingmei, ZHAO Tianjie, QIU Yubao, DU Yongming, XU Tongren. The Paradigm Theory and Judgment Conditions of Geophysical Parameter Retrieval Based on Artificial Intelligence [J]. Smart Agriculture, 2023, 5(2): 161-171.
[2]	XIA Xue, CHAI Xiujuan, ZHANG Ning, ZHOU Shuo, SUN Qixin, SUN Tan. A Lightweight Fruit Load Estimation Model for Edge Computing Equipment [J]. Smart Agriculture, 2023, 5(2): 1-12.
[3]	GUO Yangyang, DU Shuzeng, QIAO Yongliang, LIANG Dong. Advances in the Applications of Deep Learning Technology for Livestock Smart Farming [J]. Smart Agriculture, 2023, 5(1): 52-65.
[4]	ZUO Min, HU Tianyu, DONG Wei, ZHANG Kexin, ZHANG Qingchuan. Forecast and Analysis of Agricultural Products Logistics Demand Based on Informer Neural Network: Take the Central China Aera as An Example [J]. Smart Agriculture, 2023, 5(1): 34-43.
[5]	JI Jie, JIN Zhou, WANG Rujing, LIU Haiyan, LI Zhiyuan. Progressive Convolutional Net Based Method for Agricultural Named Entity Recognition [J]. Smart Agriculture, 2023, 5(1): 122-131.
[6]	HU Songtao, ZHAI Ruifang, WANG Yinghua, LIU Zhi, ZHU Jianzhong, REN He, YANG Wanneng, SONG Peng. Extraction of Potato Plant Phenotypic Parameters Based on Multi-Source Data [J]. Smart Agriculture, 2023, 5(1): 132-145.
[7]	LIU Xiaohang, ZHANG Zhao, LIU Jiaying, ZHANG Man, LI Han, FLORES Paulo, HAN Xiongzhe. Infield Corn Kernel Detection and Counting Based on Multiple Deep Learning Networks [J]. Smart Agriculture, 2022, 4(4): 49-60.
[8]	XU Yulin, KANG Mengzhen, WANG Xiujuan, HUA Jing, WANG Haoyu, SHEN Zhen. Corn and Soybean Futures Price Intelligent Forecasting Based on Deep Learning [J]. Smart Agriculture, 2022, 4(4): 156-163.
[9]	LUO Qing, RAO Yuan, JIN Xiu, JIANG Zhaohui, WANG Tan, WANG Fengyi, ZHANG Wu. Multi-Class on-Tree Peach Detection Using Improved YOLOv5s and Multi-Modal Images [J]. Smart Agriculture, 2022, 4(4): 84-104.
[10]	ZHANG Zhibo, ZHAO Xining, GAO Xiaodong, ZHANG Li, YANG Menghao. Accurate Extraction of Apple Orchard on the Loess Plateau Based on Improved Linknet Network [J]. Smart Agriculture, 2022, 4(3): 95-107.
[11]	SHANG Fengnan, ZHOU Xuecheng, LIANG Yingkai, XIAO Mingwei, CHEN Qiao, LUO Chendi. Detection Method for Dragon Fruit in Natural Environment Based on Improved YOLOX [J]. Smart Agriculture, 2022, 4(3): 120-131.
[12]	ZHUANG Jiayu, XU Shiwei, LI Yang, XIONG Lu, LIU Kebao, ZHONG Zhiping. Supply and Demand Forecasting Model of Multi-Agricultural Products Based on Deep Learning [J]. Smart Agriculture, 2022, 4(2): 174-182.
[13]	SHAO Mingyue, ZHANG Jianhua, FENG Quan, CHAI Xiujuan, ZHANG Ning, ZHANG Wenrong. Research Progress of Deep Learning in Detection and Recognition of Plant Leaf Diseases [J]. Smart Agriculture, 2022, 4(1): 29-46.
[14]	GUO Xiuming, ZHU Yeping, LI Shijuan, ZHANG Jie, LYU Chunyang, LIU Shengping. Scale Adaptive Small Objects Detection Method in Complex Agricultural Environment: Taking Bees as Research Object [J]. Smart Agriculture, 2022, 4(1): 140-149.
[15]	YANG Guofeng, HE Yong, FENG Xuping, LI Xiyao, ZHANG Jinnuo, YU Zeyu. Methods and New Research Progress of Remote Sensing Monitoring of Crop Disease and Pest Stress Using Unmanned Aerial Vehicle [J]. Smart Agriculture, 2022, 4(1): 1-16.