Underwater Fish Species Identification Model and Real-Time Identification System

doi:10.12133/j.smartag.SA202202006

Abstract

Abstract:

Convolutional neural network models have different advantages and disadvantages, it is becoming more and more difficult to select an appropriate convolutional neural network model in an actual fish identification project. The identification of underwater fish is a challenge task due to varies in illumination, low contrast, high noise, low resolution and sample imbalance between each type of image from the real underwater environment. In addition, deploying models to mobile devices directly will reduce the accuracy of the model sharply. In order to solve the above problems, Fish Recognition Ground-Truth dataset was used to training model in this study, which is provided by Fish4Knowledge project from University of Edinburgh. It contains 27,370 images with 23 fish species, and has been labeled manually by marine biologists. AlexNet, GoogLeNet, ResNet and DenseNet models were selected initially according to the characteristics of real-time underwater fish identification task, then a comparative experiment was designed to explore the best network model. Random image flipping, rotation and color dithering were used to enhance data based on ground-truth fish dataset in response to the limited number of underwater fish images. Considering that there was a serious imbalance in the number of samples in each category, the label smoothing technology was used to alleviate model overfitting. The Ranger optimizer and Cosine learning rate attenuation strategy were used to further improve the training effect of the models. The accuracy and recall rate information of each model were recorded and counted. The results showed that, the accuracy and recall rate of the fish recognition model based on DenseNet reached 99.21% and 96.77% in train set and validation set respectively, its F₁ value reached 0.9742, which was the best model obtained in the experiment. Finally, a remote fish identification system was designed based on Python language, in this system the model was deployed to linux server and the Android APP was responsible for uploading fish images via http to request server to identify the fishes and displaying the identification information returned by server, such as fish species, profiles, habits, distribution, etc. A set of recognition tests were performed on real Android phone and the results showed that in the same local area net the APP could show fish information rapidly and exactly within 1 s.

Key words: fish identification model, CNN, model evaluation, Android, Ground-Truth, real-time identification system

CLC Number:

S964

LI Shaobo, YANG Ling, YU Huihui, CHEN Yingyi. Underwater Fish Species Identification Model and Real-Time Identification System[J]. Smart Agriculture, 2022, 4(1): 130-139.

Figures/Tables 16

Fig. 1

Fig. 2

Fig.3

Table 1

Fig. 4

Table 2

Table 3

Table 4

Fig. 5

Fig. 6

Table 5

Fig. 7

Fig. 8

Fig.9

Fig.10

Table 6

References 40

1	NELSON J S, GRANDE T C, WILSON M V. Fishes of the world[M]. New Jersey: John Wiley & Sons, 2016.
2	YANG L, LIU Y, YU H, et al. Computer vision models in intelligent aquaculture with emphasis on fish detection and behavior analysis: A review[J]. Archives of Computational Methods in Engineering, 2021, 28(4): 2785-2816.
3	LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International journal of computer vision, 2004, 60(2): 91-110.
4	DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]// 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). Piscataway, New York, USA: IEEE, 2005: 886-893.
5	卢宏涛, 张秦川. 深度卷积神经网络在计算机视觉中的应用研究综述[J]. 数据采集与处理, 2016, 31(1): 1-17.
	LU H, ZHANG Q. Applications of deep convolutional neural network in computer vision[J]. Data Acquisition and Processing, 2016, 31(1): 1-17.
6	AL-SAFFAR A A M, TAO H, TALAB M A. Review of deep convolution neural network in image classification[C]// 2017 International Conference on Radar, Antenna, Microwave, Electronics, and Telecommunications (ICRAMET). Piscataway, New York, USA: IEEE, 2017: 26-31.
7	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25: 1097-1105.
8	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J/OL]. arXiv: , 2014.
9	SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]// The IEEE Conference on Computer vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2015: 1-9.
10	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2016: 770-778.
11	HUANG G, LIU Z, MAATEN L VAN DER, et al. Densely connected convolutional networks[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2017:4700-4708.
12	HU H, YANG Y. A Combined GLQP and DBN-DRF for face recognition in unconstrained environments[C]// 2017 2nd International Conference on Control, Automation and Artificial Intelligence (CAAI 2017). Paris: Atlantis Press, 2017: 553-557.
13	NOVAKOVIĆ J D, VELJOVIĆ A, ILIĆ S S, et al. Evaluation of classification models in machine learning[J]. Theory and Applications of Mathematics & Computer Science, 2017, 7(1): 39-46.
14	ERTOSUN M G, RUBIN D L. Probabilistic visual search for masses within mammography images using deep learning[C]// 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Piscataway, New York, USA: IEEE, 2015: 1310-1315.
15	LIN C, LIN C, WANG S, et al. Multiple convolutional neural networks fusion using improved fuzzy integral for facial emotion recognition[J]. Applied Sciences, 2019, 9(13): ID 2593.
16	BHATIA G S, AHUJA P, CHAUDHARI D, et al. Farmguide-one-stop solution to farmers[J/OL]. Asian Journal For Convergence In Technology, 2019, 4(1). [2021-09-06]. .
17	中国互联网络信息中心. 中国互联网络发展状况统计报告[EB/OL]. [2021-09-07]. .
18	孙彦博. 2021年中国手机操作系统行业研究报告[R]. 南京: 头豹研究院, 2021.
19	BOOM B J, HUANG P X, HE J, et al. Supporting ground-truth annotation of image datasets using clustering[C]// The 21st International Conference on Pattern Recognition (ICPR2012). Piscataway, New York, USA: IEEE, 2012: 1542-1545.
20	MAKANTASIS K, KARANTZALOS K, DOULAMIS A, et al. Deep supervised learning for hyperspectral data classification through convolutional neural networks[C]// 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Piscataway, New York, USA: IEEE, 2015: 4959-4962.
21	魏秀参. 解析深度学习: 卷积神经网络原理与视觉实践[M]. 北京: 电子工业出版社, 2018: 13-14.
22	RUMELHART D E, HINTON G E, WILLIAMS R J. Learning representations by back-propagating errors[J]. nature, 1986, 323(6088): 533-536.
23	BOUVRIE J. Notes on convolutional neural networks[J/OL]. (2006-11-22)[2021-09-06]. .
24	KHAN A, SOHAIL A, ZAHOORA U, et al. A survey of the recent architectures of deep convolutional neural networks[J]. Artificial Intelligence Review, 2020, 53(8): 5455-5516.
25	RAHMAD F, SURYANTO Y, RAMLI K. Performance comparison of anti-spam technology using confusion matrix classification[C]// IOP Conference Series: Materials Science and Engineering. Bandung, Indonesia: IOP Publishing, 2020: 12076.
26	WANG H, ZHAO T, LI L C, et al. A hybrid CNN feature model for pulmonary nodule malignancy risk differentiation[J]. Journal of X-ray Science and Technology, 2018, 26(2): 171-187.
27	KAMILARIS A, PRENAFETA-BOLDÚ F X. A review of the use of convolutional neural networks in agriculture[J]. The Journal of Agricultural Science, 2018, 156(3): 312-322.
28	梁红, 金磊磊, 杨长生. 小样本情况基于深度学习的水下目标识别研究[J]. 武汉理工大学学报(交通科学与工程版), 2019, 43(1): 6-10.
	LIANG H, JIN L, YANG C. Research on underwater target recognition based on depth learning with small sample[J]. Journal of Wuhan University of Technology(Transportation Science & Engineering), 2019, 43(1): 6-10.
29	ZHANG C, JIANG P, HOU Q, et al. Delving deep into label smoothing[J]. IEEE Transactions on Image Processing, 2021, 30: 5984-5996.
30	GitHub-lessw2020/Ranger-Deep-Learning-Optimizer: Ranger — A synergistic optimizer using RAdam (Rectified Adam), Gradient Centralization and LookAhead in one codebase[EB/OL]. [2021/8/30]. .
31	LIU L, JIANG H, HE P, et al. On the variance of the adaptive learning rate and beyond[J/OL]. arXiv:, 2019.
32	ZHANG M R, LUCAS J, HINTON G, et al. Lookahead optimizer: K steps forward, 1 step back[J/OL]. arXiv:, 2019.
33	Django overview[EB/OL]. [2021-07-28]. .
34	Gunicorn-Python WSGI HTTP Server for UNIX[EB/OL]. [2021-07-28]. .
35	PyTorch documentation[EB/OL]. [2021-07-28]. .
36	中文海洋鱼类资料库[EB/OL]. [2021-08-30]. .
37	台湾鱼类资料库[EB/OL]. [2021-08-30]. .
38	Android developers[EB/OL]. [2021-07-28]. .
39	OkHttp[EB/OL]. [2021-08-30]. .
40	GitHub-asaskevich/EventBus: [Go]
	Lightweight eventbus with async compatibility for Go[EB/OL]. [2021-08-30]. .

CNN名称	模型大小/Mb
AlexNet	60
VGGNet	138
GoogLeNet	4
ResNet	1.7~25.6
DenseNet	15.3~25.6

超参数	值
训练轮数/轮	30
批处理大小	64
学习率	0.0001
优化器	Ranger
学习率衰减	Cosine
损失	Label Smoothing

模型	训练集精度	验证集表现			模型参数量/百万	浮点计算次数/G
模型	训练集精度	准确度/%	精确度/%	召回率/%	模型参数量/百万	浮点计算次数/G
AlexNet	99.42	99.08	95.59	93.76	61.10	0.93
GoogLeNet	99.81	99.51	98.37	94.75	5.62	1.98
ResNet50	99.83	99.58	99.26	96.64	23.56	5.98
DenseNet169	99.90	99.64	99.21	96.77	14.15	4.49

模型	整体F₁ 值
AlexNet	0.9449
GoogLeNet	0.9537
ResNet50	0.9739
DenseNet169	0.9742

字段名	字段类型	完整性约束	是否为主键	字段含义
fish_id	int	非空	是	鱼类编号
type	varchar	非空	否	鱼的种类
profile	varchar	非空	否	简介信息
outline	varchar	非空	否	外形介绍
distribution	varchar	非空	否	分布情况
life_cycle	varchar	非空	否	生长周期
disease	varchar	非空	否	常见病害