基于迁移学习的多尺度特征融合牦牛脸部识别算法

doi:10.12133/j.smartag.SA202201001

摘要/Abstract

摘要：

牦牛个体身份标识是实现个体建档、行为监测、精准饲喂、疫病防控及食品溯源的前提。针对智慧畜牧智能化、信息化等养殖平台中动物个体识别技术应用需求，本研究提出一种基于迁移学习的多尺度特征融合牦牛脸部识别算法（Transfer Learning-Multiscale Feature Fusion-VGG， T-M-VGG）。以预训练的视觉几何组网络（Visual Geometry Group Network，VGG）为骨干网络构建基于迁移学习的卷积神经网络模型，获取其Block3、Block4、Block5输出的特征图，分别用F3、F4、F5表示，将F3和F5经过三个不同膨胀系数的空洞卷积组成的并行空洞卷积模块增大感受野后，送入改进的特征金字塔进行多尺度特征融合；最后利用全局平均池化代替全连接层分类输出。试验结果表明，本研究提出的T-M-VGG算法在194头牦牛的38,800张数据集中识别准确率达到96.01%，模型大小为70.75 MB。随机选取12张不同类别牦牛图像进行面部遮挡测试，识别准确率为83.33%。本算法可以为牦牛脸部识别研究提供参考。

关键词: 牦牛, 脸部识别, 迁移学习, 特征金字塔, T-M-VGG

Abstract:

Identifying of yak is indispensable for individual documentation, behavior monitoring, precise feeding, disease prevention and control, food traceability, and individualized breeding. Aiming at the application requirements of animal individual identification technology in intelligent informatization animal breeding platforms, a yak face recognition algorithm based on transfer learning and multiscale feature fusion, i.e., transfer learning-multiscale feature fusion-VGG(T-M-VGG) was proposed. The sample data set of yak facial images was produced by a camera named GoPro HERO8 BLACK. Then, a part of dataset was increased by the data enhancement ways that involved rotating, adjusting the brightness and adding noise to improve the robustness and accuracy of model. T-M-VGG, a kind of convolutional neural network based on pre-trained visual geometry group network and transfer learning was input with normalized dataset samples. The feature map of Block3, Block4 and Block5 were considered as F3, F4 and F5, respectively. What's more, F3 and F5 were taken by the structure that composed of three parallel dilated convolutions, the dilation rate were one, two and three, respectively, to dilate the receptive filed which was the map size of feature map. Further, the multiscale feature maps were fused by the improved feature pyramid which was in the shape of stacked hourglass structure. Finally, the fully connected layer was replaced by the global average pooling to classify and reduce a large number of parameters. To verify the effectiveness of the proposed model, a comparative experiment was conducted. The experimental results showed that recognition accuracy rate in 38,800 data sets of 194 yaks reached 96.01%, but the storage size was 70.75 MB. Twelve images representing different yak categories from dataset were chosen randomly for occlusion test. The origin images were masked with different shape of occlusions. The accuracy of identifying yak individuals was 83.33% in the occlusion test, which showed that the model had mainly learned facial features. The proposed algorithm could provide a reference for research of yak face recognition and would be the foundation for the establishment of smart management platform.

Key words: yak, face recognition, transfer learning, feature pyramid structure, T-M-VGG

中图分类号:

TP391

陈占琦, 张玉安, 王文志, 李丹, 何杰, 宋仁德. 基于迁移学习的多尺度特征融合牦牛脸部识别算法[J]. 智慧农业(中英文), 2022, 4(2): 77-85.

CHEN Zhanqi, ZHANG Yu'an, WANG Wenzhi, LI Dan, HE Jie, SONG Rende. Multiscale Feature Fusion Yak Face Recognition Algorithm Based on Transfer Learning[J]. Smart Agriculture, 2022, 4(2): 77-85.

图/表 10

图1

图2

图3

图4

图5

表1

试验方案参数设置

试验方案	图像训练形式	优化函数	学习率	批量	迁移学习-全连接层
文献［9］结构	256 $×$ 256 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
文献［10］结构	128 $×$ 128 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
文献［11］结构	256 $×$ 256 $×$ 1	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
文献［25］结构	256 $×$ 256 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
VGG16	128 $×$ 128 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
Mb-Net-L	128 $×$ 128 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
Mb-Net-S	256 $×$ 256 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
InceptionV3	128 $×$ 128 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
FaceNet结构	128 $×$ 128 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
Tr-L-VGG16	256 $×$ 256 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	x=Activation（'relu'）（output） x=GlobalAveragePooling2D（x） x=Dense（194，activation='softmax'）（x）
T-M-VGG	256 $×$ 256 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	x=Activation（'relu'）（output） x=GlobalAveragePooling2D（x） x=Dense（194，activation='softmax'）（x）

表1

图6

图7

表2

图8

参考文献 28

1	秦兴, 宋各方. 基于双线性卷积神经网络的猪脸识别算法[J]. 杭州电子科技大学学报(自然科学版), 2019, 39(2): 12-17.
	QIN X, SONG G. Pig face recognition algorithm based on bilinear convolution neural network[J]. Journal of Hangzhou Dianzi University (Natural Sciences), 2019, 39(2): 12-17.
2	何屿彤, 李斌, 张锋, 等. 基于改进YOLOv3的猪脸识别[J]. 中国农业大学学报, 2021, 26(3): 53-62.
	HE Y, LI B, ZHANG F, et al. Pig face recognition based on improved YOLOv3[J]. Journal of China Agricultural University, 2021, 26(3): 53-62.
3	刘忠超, 何东健. 基于卷积神经网络的奶牛发情行为识别方法[J]. 农业机械学报, 2019, 50(7): 186-193.
	LIU Z, HE D. Recognition method of cow estrus behavior based on convolutional neural network[J]. Transactions of the CSAM, 2019, 50(7): 186-193.
4	杨秋妹, 肖德琴, 张根兴. 猪只饮水行为机器视觉自动识别[J]. 农业机械学报, 2018, 49(6): 232-238.
	YANG Q, XIAO D, ZHANG G. Automatic pig drinking behavior recognition with machine vision[J]. Transactions of the CSAM, 2018, 49(6): 232-238.
5	张宏鸣, 周利香, 李永恒, 等. 基于改进MobileFaceNet的羊脸识别方法研究[J/OL]. 农业机械学报:1-10 [2022-05-13].
	ZHANG H, ZHOU L, LI Y, et al. Sheep face recognition method based on improved mobilefacenet[J/OL]. Transactions of the CSAM, 1-10 [2022-05-13]. .
6	魏征. 基于全局和局部特征相结合的不完美牛眼虹膜识别技术研究[D]. 南京: 东南大学, 2017.
	WEI Z. Research on iris recognition technology of imperfect bull's eye based on the combination of global and local features[D]. Nanjing: Southeast University, 2017.
7	赵凯旋, 何东健. 基于卷积神经网络的奶牛个体身份识别方法[J]. 农业工程学报, 2015, 31(5): 181-187.
	ZHAO K, HE D. Recognition of individual dairy cattle based on convolutional neural networks[J]. Transactions of the CSAE, 2015, 31(5): 181-187.
8	何东健, 刘建敏, 熊虹婷, 等. 基于改进YOLOv3模型的挤奶奶牛个体识别方法[J]. 农业机械学报, 2020, 51(4): 250-260.
	HE D, LIU J, XIONG H, et al. Individual identification of dairy cows based on improved YOLOv3[J]. Transactions of the CSAM, 2020, 51(4): 250-260.
9	陈争涛, 黄灿, 杨波, 等. 基于迁移学习的并行卷积神经网络牦牛脸识别算法[J]. 计算机应用, 2021, 41(5): 1332-1336.
	CHEN Z, HUANG C, YANG B, et al. Parallel convolutional neural network yak face recognition algorithm based on transfer learning[J]. Journal of Computer Applications, 2020, 41(5): 1332-1336.
10	HANSEN M F, SMITH M L, SMITH L N, et al. Towards on-farm pig face recognition using convolutional neural networks[J]. Computers in Industry, 2018, 98: 145-152.
11	MARSOT M, MEI J, SHAN X, et al. An adaptive pig face recognition approach using convolutional neural networks[J]. Computers and Electronics in Agriculture, 2020, 173: ID 105386.
12	KUMAR S, SINGH S K, SINGH R, et al. Deep learning framework for recognition of cattle using muzzle point image pattern[J]. Measurement, 2018, 116: 1-17.
13	JUNG D H, KIM N Y, MOON S H, et al. Deep learning-based cattle vocal classification model and real-time livestock monitoring system with noise filtering[J]. Animals, 2021, 11(2): ID 357.
14	SALAMA A, HASSANIEN A E, FAHMY A. Sheep identification using a hybrid deep learning and Bayesian optimization approach[J]. IEEE Access, 2019, 7: 31681-31687.
15	WADA N, SHINYA M, SHIRAISHI M. Pig face recognition using eigenspace method[J]. ITE Transactions on Media Technology & Applications, 2013, 1(4): 328-332.
16	RASHID M, GU X, LEE Y J. Interspecies knowledge transfer for facial keypoint detection[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2017: 1600-1609.
17	WANG Z, BOVIK A C, SHEIKN H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
18	陈桂芬, 赵姗, 曹丽英, 等. 基于迁移学习与卷积神经网络的玉米植株病害识别[J]. 智慧农业, 2019, 1(2): 34-44.
	CHEN G, ZHAO S, CAO L, et al. Corn plant disease recognition based on migration learning and convolutional neural network[J]. Smart Agriculture, 2019, 1 (2): 34-44.
19	李淼, 王敬贤, 李华龙, 等. 基于CNN和迁移学习的农作物病害识别方法研究[J]. 智慧农业, 2019, 1(3):46-55.
	LI M, WANG J, LI H, et al. Method for identifying crop disease based on cnn and transfer learning[J]. Smart Agriculture, 2019, 1(3):46-55.
20	LIU S, HUANG D. Receptive field block net for accurate and fast object detection[C]// The European Conference on Computer Vision. Cham, Switzerland: Springer, 2018: 385-400.
21	SZEGEDY C, WEI L, JIA Y, et al. Going deeper with convolutions[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2015: 1-9.
22	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2017: 2117-2125.
23	LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2018: 8759-8768.
24	TAN M, PANG R, LE Q V. Efficientdet: Scalable and efficient object detection[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2020: 10781-10790.
25	樊湘鹏, 周建平, 许燕, 等. 基于改进卷积神经网络的复杂背景下玉米病害识别[J]. 农业机械学报, 2021, 52(3): 210-217.
	FAN X, ZHOU J, XU Y, et al. Corn disease recognition under complicated background based on improved convolutional neural network[J]. Transactions of the CSAM, 2021, 52(3): 210-217.
26	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J/OL]. arXiv:.
27	HOWARD A, SANDLER M, CHEN B, et al. Searching for Mobilenetv3[C]// The IEEE International Conference on Computer Vision. Piscataway, USA: IEEE, 2019: 1314-1324.
28	SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2016: 2818-2826.

试验方案	F1值/%	模型大小/MB	准确率/%	可训练参数量/M
T-M-VGG	95.43	70.75	96.01	3.73
文献［9］结构	88.03	166.33	88.03	7.07
文献［10］结构	82.07	263.02	82.89	34.47
文献［11］结构	81.92	263.16	82.63	34.48
文献［25］结构	83.57	74.85	84.24	9.80
VGG16	92.85	502.48	93.02	65.85
Mb-Net-L	93.29	34.74	93.91	4.46
Mb-Net-S	94.60	13.65	94.62	1.72
InceptionV3	95.01	170.13	95.16	22.17
FaceNet结构	95.60	418.71	95.68	54.57
Tr-L-VGG16	20.64	56.96	28.38	0.10

[1]	杨霖, 刘双印, 徐龙琴, 赫敏, 绳庆峰, 韩佳伟. 多源信息融合下冷链配送车辆碳排放动态预测方法[J]. 智慧农业(中英文), 2024, 6(4): 138-148.
[2]	庞春晖, 陈鹏, 夏懿, 章军, 王兵, 邹岩, 陈天娇, 康辰瑞, 梁栋. 用于小麦多生长阶段倒伏边界精准检测的分层交互特征金字塔网络[J]. 智慧农业(中英文), 2024, 6(2): 128-139.
[3]	周华茂, 王婧, 殷华, 陈琦. 基于改进Mask R-CNN模型的秀珍菇表型参数自动测量方法[J]. 智慧农业(中英文), 2023, 5(4): 117-126.
[4]	张淦, 严海峰, 胡根生, 张东彦, 程涛, 潘正高, 许海峰, 沈书豪, 朱科宇. 基于深度学习语义分割和迁移学习策略的麦田倒伏面积识别方法[J]. 智慧农业(中英文), 2023, 5(3): 75-85.
[5]	唐辉, 王铭, 于秋实, 张佳茜, 刘连涛, 王楠. 融合改进UNet和迁移学习的棉花根系图像分割方法[J]. 智慧农业(中英文), 2023, 5(3): 96-109.
[6]	王亚鹏, 曹姗姗, 李全胜, 孙伟. 融合迁移学习和集成学习的自然背景下荒漠植物识别方法[J]. 智慧农业(中英文), 2023, 5(2): 93-103.
[7]	朱海鹏, 张玉安, 李欢欢, 王建文, 杨英魁, 宋仁德. 基于改进残差网络模型的不同部位牦牛肉分类识别方法[J]. 智慧农业(中英文), 2023, 5(2): 115-125.
[8]	潘晨露, 张正华, 桂文豪, 马家俊, 严晨曦, 张晓敏. 融合ECA机制与DenseNet201的水稻病虫害识别方法[J]. 智慧农业(中英文), 2023, 5(2): 45-55.
[9]	张文景, 蒋泽中, 秦立峰. 基于弱监督下改进的CBAM-ResNet18模型识别苹果多种叶部病害[J]. 智慧农业(中英文), 2023, 5(1): 111-121.
[10]	夏烨, 雷哓晖, 祁雁楠, 徐陶, 袁全春, 潘健, 姜赛珂, 吕晓兰. 基于改进Ghost-YOLOv5s-BiFPN算法检测梨树花序[J]. 智慧农业(中英文), 2022, 4(3): 108-119.
[11]	张志博, 赵西宁, 高晓东, 张利, 杨孟豪. 基于改进Linknet网络的黄土高原苹果园精准提取[J]. 智慧农业(中英文), 2022, 4(3): 95-107.
[12]	周巧黎, 马丽, 曹丽英, 于合龙. 基于改进轻量级卷积神经网络MobileNetV3的番茄叶片病害识别[J]. 智慧农业(中英文), 2022, 4(1): 47-56.
[13]	魏靖, 王玉亭, 袁会珠, 张梦蕾, 王振营. 基于深度学习与特征可视化方法的草地贪夜蛾及其近缘种成虫识别[J]. 智慧农业(中英文), 2020, 2(3): 75-85.
[14]	李淼, 王敬贤, 李华龙, 胡泽林, 杨选将, 黄小平, 曾伟辉, 张建, 房思思. 基于CNN和迁移学习的农作物病害识别方法研究[J]. 智慧农业(中英文), 2019, 1(3): 46-55.
[15]	陈桂芬, 赵姗, 曹丽英, 傅思维, 周佳鑫. 基于迁移学习与卷积神经网络的玉米植株病害识别[J]. 智慧农业(中英文), 2019, 1(2): 34-44.