Underwater Fish Species Identification Model and Real-Time Identification System

doi:10.12133/j.smartag.SA202202006

Abstract

Abstract:

Convolutional neural network models have different advantages and disadvantages, it is becoming more and more difficult to select an appropriate convolutional neural network model in an actual fish identification project. The identification of underwater fish is a challenge task due to varies in illumination, low contrast, high noise, low resolution and sample imbalance between each type of image from the real underwater environment. In addition, deploying models to mobile devices directly will reduce the accuracy of the model sharply. In order to solve the above problems, Fish Recognition Ground-Truth dataset was used to training model in this study, which is provided by Fish4Knowledge project from University of Edinburgh. It contains 27,370 images with 23 fish species, and has been labeled manually by marine biologists. AlexNet, GoogLeNet, ResNet and DenseNet models were selected initially according to the characteristics of real-time underwater fish identification task, then a comparative experiment was designed to explore the best network model. Random image flipping, rotation and color dithering were used to enhance data based on ground-truth fish dataset in response to the limited number of underwater fish images. Considering that there was a serious imbalance in the number of samples in each category, the label smoothing technology was used to alleviate model overfitting. The Ranger optimizer and Cosine learning rate attenuation strategy were used to further improve the training effect of the models. The accuracy and recall rate information of each model were recorded and counted. The results showed that, the accuracy and recall rate of the fish recognition model based on DenseNet reached 99.21% and 96.77% in train set and validation set respectively, its F₁ value reached 0.9742, which was the best model obtained in the experiment. Finally, a remote fish identification system was designed based on Python language, in this system the model was deployed to linux server and the Android APP was responsible for uploading fish images via http to request server to identify the fishes and displaying the identification information returned by server, such as fish species, profiles, habits, distribution, etc. A set of recognition tests were performed on real Android phone and the results showed that in the same local area net the APP could show fish information rapidly and exactly within 1 s.

Key words: fish identification model, CNN, model evaluation, Android, Ground-Truth, real-time identification system

CLC Number:

S964

LI Shaobo, YANG Ling, YU Huihui, CHEN Yingyi. Underwater Fish Species Identification Model and Real-Time Identification System[J]. Smart Agriculture, 2022, 4(1): 130-139.

References

1	NELSON J S, GRANDE T C, WILSON M V. Fishes of the world[M]. New Jersey: John Wiley & Sons, 2016.
2	YANG L, LIU Y, YU H, et al. Computer vision models in intelligent aquaculture with emphasis on fish detection and behavior analysis: A review[J]. Archives of Computational Methods in Engineering, 2021, 28(4): 2785-2816.
3	LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International journal of computer vision, 2004, 60(2): 91-110.
4	DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]// 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). Piscataway, New York, USA: IEEE, 2005: 886-893.
5	卢宏涛, 张秦川. 深度卷积神经网络在计算机视觉中的应用研究综述[J]. 数据采集与处理, 2016, 31(1): 1-17.
5	LU H, ZHANG Q. Applications of deep convolutional neural network in computer vision[J]. Data Acquisition and Processing, 2016, 31(1): 1-17.
6	AL-SAFFAR A A M, TAO H, TALAB M A. Review of deep convolution neural network in image classification[C]// 2017 International Conference on Radar, Antenna, Microwave, Electronics, and Telecommunications (ICRAMET). Piscataway, New York, USA: IEEE, 2017: 26-31.
7	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25: 1097-1105.
8	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J/OL]. arXiv: , 2014.
9	SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]// The IEEE Conference on Computer vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2015: 1-9.
10	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2016: 770-778.
11	HUANG G, LIU Z, MAATEN L VAN DER, et al. Densely connected convolutional networks[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2017:4700-4708.
12	HU H, YANG Y. A Combined GLQP and DBN-DRF for face recognition in unconstrained environments[C]// 2017 2nd International Conference on Control, Automation and Artificial Intelligence (CAAI 2017). Paris: Atlantis Press, 2017: 553-557.
13	NOVAKOVI? J D, VELJOVI? A, ILI? S S, et al. Evaluation of classification models in machine learning[J]. Theory and Applications of Mathematics & Computer Science, 2017, 7(1): 39-46.
14	ERTOSUN M G, RUBIN D L. Probabilistic visual search for masses within mammography images using deep learning[C]// 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Piscataway, New York, USA: IEEE, 2015: 1310-1315.
15	LIN C, LIN C, WANG S, et al. Multiple convolutional neural networks fusion using improved fuzzy integral for facial emotion recognition[J]. Applied Sciences, 2019, 9(13): ID 2593.
16	BHATIA G S, AHUJA P, CHAUDHARI D, et al. Farmguide-one-stop solution to farmers[J/OL]. Asian Journal For Convergence In Technology, 2019, 4(1). [2021-09-06]. .
17	中国互联网络信息中心. 中国互联网络发展状况统计报告[EB/OL]. [2021-09-07]. .
18	孙彦博. 2021年中国手机操作系统行业研究报告[R]. 南京: 头豹研究院, 2021.
19	BOOM B J, HUANG P X, HE J, et al. Supporting ground-truth annotation of image datasets using clustering[C]// The 21st International Conference on Pattern Recognition (ICPR2012). Piscataway, New York, USA: IEEE, 2012: 1542-1545.
20	MAKANTASIS K, KARANTZALOS K, DOULAMIS A, et al. Deep supervised learning for hyperspectral data classification through convolutional neural networks[C]// 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Piscataway, New York, USA: IEEE, 2015: 4959-4962.
21	魏秀参. 解析深度学习: 卷积神经网络原理与视觉实践[M]. 北京: 电子工业出版社, 2018: 13-14.
22	RUMELHART D E, HINTON G E, WILLIAMS R J. Learning representations by back-propagating errors[J]. nature, 1986, 323(6088): 533-536.
23	BOUVRIE J. Notes on convolutional neural networks[J/OL]. (2006-11-22)[2021-09-06]. .
24	KHAN A, SOHAIL A, ZAHOORA U, et al. A survey of the recent architectures of deep convolutional neural networks[J]. Artificial Intelligence Review, 2020, 53(8): 5455-5516.
25	RAHMAD F, SURYANTO Y, RAMLI K. Performance comparison of anti-spam technology using confusion matrix classification[C]// IOP Conference Series: Materials Science and Engineering. Bandung, Indonesia: IOP Publishing, 2020: 12076.
26	WANG H, ZHAO T, LI L C, et al. A hybrid CNN feature model for pulmonary nodule malignancy risk differentiation[J]. Journal of X-ray Science and Technology, 2018, 26(2): 171-187.
27	KAMILARIS A, PRENAFETA-BOLDú F X. A review of the use of convolutional neural networks in agriculture[J]. The Journal of Agricultural Science, 2018, 156(3): 312-322.
28	梁红, 金磊磊, 杨长生. 小样本情况基于深度学习的水下目标识别研究[J]. 武汉理工大学学报(交通科学与工程版), 2019, 43(1): 6-10.
28	LIANG H, JIN L, YANG C. Research on underwater target recognition based on depth learning with small sample[J]. Journal of Wuhan University of Technology(Transportation Science & Engineering), 2019, 43(1): 6-10.
29	ZHANG C, JIANG P, HOU Q, et al. Delving deep into label smoothing[J]. IEEE Transactions on Image Processing, 2021, 30: 5984-5996.
30	GitHub-lessw2020/Ranger-Deep-Learning-Optimizer: Ranger — A synergistic optimizer using RAdam (Rectified Adam), Gradient Centralization and LookAhead in one codebase[EB/OL]. [2021/8/30]. .
31	LIU L, JIANG H, HE P, et al. On the variance of the adaptive learning rate and beyond[J/OL]. arXiv:, 2019.
32	ZHANG M R, LUCAS J, HINTON G, et al. Lookahead optimizer: K steps forward, 1 step back[J/OL]. arXiv:, 2019.
33	Django overview[EB/OL]. [2021-07-28]. .
34	Gunicorn-Python WSGI HTTP Server for UNIX[EB/OL]. [2021-07-28]. .
35	PyTorch documentation[EB/OL]. [2021-07-28]. .
36	中文海洋鱼类资料库[EB/OL]. [2021-08-30]. .
37	台湾鱼类资料库[EB/OL]. [2021-08-30]. .
38	Android developers[EB/OL]. [2021-07-28]. .
39	OkHttp[EB/OL]. [2021-08-30]. .
40	GitHub-asaskevich/EventBus: [Go]
40	Lightweight eventbus with async compatibility for Go[EB/OL]. [2021-08-30]. .

[1]	PAN Weiting, SUN Mengli, YUN Yan, LIU Ping. Identification Method of Wheat Grain Phenotype Based on Deep Learning of ImCascade R-CNN [J]. Smart Agriculture, 2023, 5(3): 110-120.
[2]	LI Yangde, MA Xiaohui, WANG Ji. Pineapple Maturity Analysis in Natural Environment Based on MobileNet V3-YOLOv4 [J]. Smart Agriculture, 2023, 5(2): 35-44.
[3]	PAN Chenlu, ZHANG Zhenghua, GUI Wenhao, MA Jiajun, YAN Chenxi, ZHANG Xiaomin. Rice Disease and Pest Recognition Method Integrating ECA Mechanism and DenseNet201 [J]. Smart Agriculture, 2023, 5(2): 45-55.
[4]	LIU Xiaohang, ZHANG Zhao, LIU Jiaying, ZHANG Man, LI Han, FLORES Paulo, HAN Xiongzhe. Infield Corn Kernel Detection and Counting Based on Multiple Deep Learning Networks [J]. Smart Agriculture, 2022, 4(4): 49-60.
[5]	HE Ruimin, ZHENG Kefeng, WEI Qinyang, ZHANG Xiaobin, ZHANG Jun, ZHU Yihang, ZHAO Yiying, GU Qing. Identification and Counting of Silkworms in Factory Farm Using Improved Mask R-CNN Model [J]. Smart Agriculture, 2022, 4(2): 163-173.
[6]	ZHANG Xiaoqing, SHAO Song, GUO Xinyu, FAN Jiangchuan. High-Throughput Dynamic Monitoring Method of Field Maize Seedling [J]. Smart Agriculture, 2021, 3(2): 88-99.
[7]	Li Miao, Wang Jingxian, Li Hualong, Hu Zelin, Yang XuanJiang, Huang Xiaoping, Zeng Weihui, Zhang Jian, Fang Sisi. Method for identifying crop disease based on CNN and transfer learning [J]. Smart Agriculture, 2019, 1(3): 46-55.