Multiscale Feature Fusion Yak Face Recognition Algorithm Based on Transfer Learning

doi:10.12133/j.smartag.SA202201001

Abstract

Abstract:

Identifying of yak is indispensable for individual documentation, behavior monitoring, precise feeding, disease prevention and control, food traceability, and individualized breeding. Aiming at the application requirements of animal individual identification technology in intelligent informatization animal breeding platforms, a yak face recognition algorithm based on transfer learning and multiscale feature fusion, i.e., transfer learning-multiscale feature fusion-VGG(T-M-VGG) was proposed. The sample data set of yak facial images was produced by a camera named GoPro HERO8 BLACK. Then, a part of dataset was increased by the data enhancement ways that involved rotating, adjusting the brightness and adding noise to improve the robustness and accuracy of model. T-M-VGG, a kind of convolutional neural network based on pre-trained visual geometry group network and transfer learning was input with normalized dataset samples. The feature map of Block3, Block4 and Block5 were considered as F3, F4 and F5, respectively. What's more, F3 and F5 were taken by the structure that composed of three parallel dilated convolutions, the dilation rate were one, two and three, respectively, to dilate the receptive filed which was the map size of feature map. Further, the multiscale feature maps were fused by the improved feature pyramid which was in the shape of stacked hourglass structure. Finally, the fully connected layer was replaced by the global average pooling to classify and reduce a large number of parameters. To verify the effectiveness of the proposed model, a comparative experiment was conducted. The experimental results showed that recognition accuracy rate in 38,800 data sets of 194 yaks reached 96.01%, but the storage size was 70.75 MB. Twelve images representing different yak categories from dataset were chosen randomly for occlusion test. The origin images were masked with different shape of occlusions. The accuracy of identifying yak individuals was 83.33% in the occlusion test, which showed that the model had mainly learned facial features. The proposed algorithm could provide a reference for research of yak face recognition and would be the foundation for the establishment of smart management platform.

Key words: yak, face recognition, transfer learning, feature pyramid structure, T-M-VGG

CLC Number:

TP391

CHEN Zhanqi, ZHANG Yu'an, WANG Wenzhi, LI Dan, HE Jie, SONG Rende. Multiscale Feature Fusion Yak Face Recognition Algorithm Based on Transfer Learning[J]. Smart Agriculture, 2022, 4(2): 77-85.

Figures/Tables 10

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Table 1

Parameters setting of experimental schemes

试验方案	图像训练形式	优化函数	学习率	批量	迁移学习-全连接层
文献［9］结构	256 $×$ 256 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
文献［10］结构	128 $×$ 128 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
文献［11］结构	256 $×$ 256 $×$ 1	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
文献［25］结构	256 $×$ 256 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
VGG16	128 $×$ 128 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
Mb-Net-L	128 $×$ 128 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
Mb-Net-S	256 $×$ 256 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
InceptionV3	128 $×$ 128 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
FaceNet结构	128 $×$ 128 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	——
Tr-L-VGG16	256 $×$ 256 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	x=Activation（'relu'）（output） x=GlobalAveragePooling2D（x） x=Dense（194，activation='softmax'）（x）
T-M-VGG	256 $×$ 256 $×$ 3	SGD（momentum=0.9，decay=0.00001）	0.001	128	x=Activation（'relu'）（output） x=GlobalAveragePooling2D（x） x=Dense（194，activation='softmax'）（x）

Table 1

Fig.6

Fig.7

Table 2

Fig. 8

References 28

1	秦兴, 宋各方. 基于双线性卷积神经网络的猪脸识别算法[J]. 杭州电子科技大学学报(自然科学版), 2019, 39(2): 12-17.
	QIN X, SONG G. Pig face recognition algorithm based on bilinear convolution neural network[J]. Journal of Hangzhou Dianzi University (Natural Sciences), 2019, 39(2): 12-17.
2	何屿彤, 李斌, 张锋, 等. 基于改进YOLOv3的猪脸识别[J]. 中国农业大学学报, 2021, 26(3): 53-62.
	HE Y, LI B, ZHANG F, et al. Pig face recognition based on improved YOLOv3[J]. Journal of China Agricultural University, 2021, 26(3): 53-62.
3	刘忠超, 何东健. 基于卷积神经网络的奶牛发情行为识别方法[J]. 农业机械学报, 2019, 50(7): 186-193.
	LIU Z, HE D. Recognition method of cow estrus behavior based on convolutional neural network[J]. Transactions of the CSAM, 2019, 50(7): 186-193.
4	杨秋妹, 肖德琴, 张根兴. 猪只饮水行为机器视觉自动识别[J]. 农业机械学报, 2018, 49(6): 232-238.
	YANG Q, XIAO D, ZHANG G. Automatic pig drinking behavior recognition with machine vision[J]. Transactions of the CSAM, 2018, 49(6): 232-238.
5	张宏鸣, 周利香, 李永恒, 等. 基于改进MobileFaceNet的羊脸识别方法研究[J/OL]. 农业机械学报:1-10 [2022-05-13].
	ZHANG H, ZHOU L, LI Y, et al. Sheep face recognition method based on improved mobilefacenet[J/OL]. Transactions of the CSAM, 1-10 [2022-05-13]. .
6	魏征. 基于全局和局部特征相结合的不完美牛眼虹膜识别技术研究[D]. 南京: 东南大学, 2017.
	WEI Z. Research on iris recognition technology of imperfect bull's eye based on the combination of global and local features[D]. Nanjing: Southeast University, 2017.
7	赵凯旋, 何东健. 基于卷积神经网络的奶牛个体身份识别方法[J]. 农业工程学报, 2015, 31(5): 181-187.
	ZHAO K, HE D. Recognition of individual dairy cattle based on convolutional neural networks[J]. Transactions of the CSAE, 2015, 31(5): 181-187.
8	何东健, 刘建敏, 熊虹婷, 等. 基于改进YOLOv3模型的挤奶奶牛个体识别方法[J]. 农业机械学报, 2020, 51(4): 250-260.
	HE D, LIU J, XIONG H, et al. Individual identification of dairy cows based on improved YOLOv3[J]. Transactions of the CSAM, 2020, 51(4): 250-260.
9	陈争涛, 黄灿, 杨波, 等. 基于迁移学习的并行卷积神经网络牦牛脸识别算法[J]. 计算机应用, 2021, 41(5): 1332-1336.
	CHEN Z, HUANG C, YANG B, et al. Parallel convolutional neural network yak face recognition algorithm based on transfer learning[J]. Journal of Computer Applications, 2020, 41(5): 1332-1336.
10	HANSEN M F, SMITH M L, SMITH L N, et al. Towards on-farm pig face recognition using convolutional neural networks[J]. Computers in Industry, 2018, 98: 145-152.
11	MARSOT M, MEI J, SHAN X, et al. An adaptive pig face recognition approach using convolutional neural networks[J]. Computers and Electronics in Agriculture, 2020, 173: ID 105386.
12	KUMAR S, SINGH S K, SINGH R, et al. Deep learning framework for recognition of cattle using muzzle point image pattern[J]. Measurement, 2018, 116: 1-17.
13	JUNG D H, KIM N Y, MOON S H, et al. Deep learning-based cattle vocal classification model and real-time livestock monitoring system with noise filtering[J]. Animals, 2021, 11(2): ID 357.
14	SALAMA A, HASSANIEN A E, FAHMY A. Sheep identification using a hybrid deep learning and Bayesian optimization approach[J]. IEEE Access, 2019, 7: 31681-31687.
15	WADA N, SHINYA M, SHIRAISHI M. Pig face recognition using eigenspace method[J]. ITE Transactions on Media Technology & Applications, 2013, 1(4): 328-332.
16	RASHID M, GU X, LEE Y J. Interspecies knowledge transfer for facial keypoint detection[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2017: 1600-1609.
17	WANG Z, BOVIK A C, SHEIKN H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
18	陈桂芬, 赵姗, 曹丽英, 等. 基于迁移学习与卷积神经网络的玉米植株病害识别[J]. 智慧农业, 2019, 1(2): 34-44.
	CHEN G, ZHAO S, CAO L, et al. Corn plant disease recognition based on migration learning and convolutional neural network[J]. Smart Agriculture, 2019, 1 (2): 34-44.
19	李淼, 王敬贤, 李华龙, 等. 基于CNN和迁移学习的农作物病害识别方法研究[J]. 智慧农业, 2019, 1(3):46-55.
	LI M, WANG J, LI H, et al. Method for identifying crop disease based on cnn and transfer learning[J]. Smart Agriculture, 2019, 1(3):46-55.
20	LIU S, HUANG D. Receptive field block net for accurate and fast object detection[C]// The European Conference on Computer Vision. Cham, Switzerland: Springer, 2018: 385-400.
21	SZEGEDY C, WEI L, JIA Y, et al. Going deeper with convolutions[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2015: 1-9.
22	LIN T Y, DOLLAR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2017: 2117-2125.
23	LIU S, QI L, QIN H, et al. Path aggregation network for instance segmentation[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2018: 8759-8768.
24	TAN M, PANG R, LE Q V. Efficientdet: Scalable and efficient object detection[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2020: 10781-10790.
25	樊湘鹏, 周建平, 许燕, 等. 基于改进卷积神经网络的复杂背景下玉米病害识别[J]. 农业机械学报, 2021, 52(3): 210-217.
	FAN X, ZHOU J, XU Y, et al. Corn disease recognition under complicated background based on improved convolutional neural network[J]. Transactions of the CSAM, 2021, 52(3): 210-217.
26	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J/OL]. arXiv:.
27	HOWARD A, SANDLER M, CHEN B, et al. Searching for Mobilenetv3[C]// The IEEE International Conference on Computer Vision. Piscataway, USA: IEEE, 2019: 1314-1324.
28	SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2016: 2818-2826.

试验方案	F1值/%	模型大小/MB	准确率/%	可训练参数量/M
T-M-VGG	95.43	70.75	96.01	3.73
文献［9］结构	88.03	166.33	88.03	7.07
文献［10］结构	82.07	263.02	82.89	34.47
文献［11］结构	81.92	263.16	82.63	34.48
文献［25］结构	83.57	74.85	84.24	9.80
VGG16	92.85	502.48	93.02	65.85
Mb-Net-L	93.29	34.74	93.91	4.46
Mb-Net-S	94.60	13.65	94.62	1.72
InceptionV3	95.01	170.13	95.16	22.17
FaceNet结构	95.60	418.71	95.68	54.57
Tr-L-VGG16	20.64	56.96	28.38	0.10

[1]	ZHANG Gan, YAN Haifeng, HU Gensheng, ZHANG Dongyan, CHENG Tao, PAN Zhenggao, XU Haifeng, SHEN Shuhao, ZHU Keyu. Identification Method of Wheat Field Lodging Area Based on Deep Learning Semantic Segmentation and Transfer Learning [J]. Smart Agriculture, 2023, 5(3): 75-85.
[2]	TANG Hui, WANG Ming, YU Qiushi, ZHANG Jiaxi, LIU Liantao, WANG Nan. Root Image Segmentation Method Based on Improved UNet and Transfer Learning [J]. Smart Agriculture, 2023, 5(3): 96-109.
[3]	WANG Yapeng, CAO Shanshan, LI Quansheng, SUN Wei. Desert Plant Recognition Method Under Natural Background Incorporating Transfer Learning and Ensemble Learning [J]. Smart Agriculture, 2023, 5(2): 93-103.
[4]	ZHU Haipeng, ZHANG Yu'an, LI Huanhuan, WANG Jianwen, YANG Yingkui, SONG Rende. Classification and Recognition Method for Yak Meat Parts Based on Improved Residual Network Model [J]. Smart Agriculture, 2023, 5(2): 115-125.
[5]	PAN Chenlu, ZHANG Zhenghua, GUI Wenhao, MA Jiajun, YAN Chenxi, ZHANG Xiaomin. Rice Disease and Pest Recognition Method Integrating ECA Mechanism and DenseNet201 [J]. Smart Agriculture, 2023, 5(2): 45-55.
[6]	ZHANG Wenjing, JIANG Zezhong, QIN Lifeng. Identifying Multiple Apple Leaf Diseases Based on the Improved CBAM-ResNet18 Model Under Weak Supervision [J]. Smart Agriculture, 2023, 5(1): 111-121.
[7]	ZHANG Zhibo, ZHAO Xining, GAO Xiaodong, ZHANG Li, YANG Menghao. Accurate Extraction of Apple Orchard on the Loess Plateau Based on Improved Linknet Network [J]. Smart Agriculture, 2022, 4(3): 95-107.
[8]	ZHOU Qiaoli, MA Li, CAO Liying, YU Helong. Identification of Tomato Leaf Diseases Based on Improved Lightweight Convolutional Neural Networks MobileNetV3 [J]. Smart Agriculture, 2022, 4(1): 47-56.
[9]	WEI Jing, WANG Yuting, YUAN Huizhu, ZHANG Menglei, WANG Zhenying. Identification and Morphological Analysis of Adult Spodoptera Frugiperda and Its Close Related Species Using Deep Learning [J]. Smart Agriculture, 2020, 2(3): 75-85.
[10]	Li Miao, Wang Jingxian, Li Hualong, Hu Zelin, Yang XuanJiang, Huang Xiaoping, Zeng Weihui, Zhang Jian, Fang Sisi. Method for identifying crop disease based on CNN and transfer learning [J]. Smart Agriculture, 2019, 1(3): 46-55.
[11]	Chen Guifen, Zhao Shan, Cao Liying, Fu Siwei, Zhou Jiaxin. Corn plant disease recognition based on migration learning and convolutional neural network [J]. Smart Agriculture, 2019, 1(2): 34-44.