Fast Counting Method of Soybean Seeds Based on Density Estimation and VGG-Two

doi:10.12133/j.smartag.2021.3.4.202101-SA002

Abstract

Abstract:

In order to count soybean seeds quickly and accurately, improve the speed of seed test and the level of soybean breeding, a method of soybean seed counting based on VGG-Two (VGG-T) was developed in this research. Firstly, in view of the lack of available image dataset in the field of soybean seed counting, a fast target point labeling method of combining pre-annotation based on digital image processing technology with manual correction annotation was proposed to speed up the establishment of publicly available soybean seed image dataset with annotation. Only 197 min were taken to mark 37,563 seeds when using this method, which saved 1592 min than ordinary manual marking and could reduce 96% of manual workload. At the same time, the dataset in this research is the largest annotated data set for soybean seed counting so far. Secondly, a method that combined the density estimation-based and the convolution neural network (CNN) was developed to accurately estimate the seed count from an individual threshed seed image with a single perspective. Thereinto, a CNN architecture consisting of two columns of the same network structure was used to learn the mapping from the original pixel to the density map. Due to the very limited number of training samples and the effect of vanishing gradients on deep neural networks, it is not easy for the network to learn all parameters at the same time. Inspired by the success of pre-training, this research pre-trained the CNN in each column by directly mapping the output of the fourth convolutional layer to the density map. Then these pre-trained CNNs were used to initialize CNNs in these two columns and fine-tune all parameters. Finally, the model was tested, and the effectiveness of the algorithm through three comparative experiments (with and without data enhancement, VGG16 and VGG-T, multiple sets of test set) was verified, which respectively provided 0.6 and 0.2 mean absolute error (MAE) in the original image and patch cases, while mean squared error (MSE) were 0.6 and 0.3. Compared with traditional image morphology operations, ResNet18, ResNet18-T and VGG16, the method proposed improving the accuracy of soybean seed counting. In the testset containing soybean seeds of different densities, the error fluctuation was small, and it still had excellent counting performance. At the same time, compared with manual counting and photoelectric seed counter, it saved about 2.493 h and 0.203 h respectively for counting 11,350 soybean seeds, realizing rapid soybean seeds counting.

Key words: convolutional neural network, seed counting, seed image, point labeling, density map, VGG-Two, breeding

CLC Number:

TP391.4

WANG Ying, LI Yue, WU Tingting, SUN Shi, WANG Minjuan. Fast Counting Method of Soybean Seeds Based on Density Estimation and VGG-Two[J]. Smart Agriculture, 2021, 3(4): 111-122.

Figures/Tables 17

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Table 1

Fig. 10

Fig. 11

Table 2

Network structure comparison of VGG-T and VGG

$V G G 16$	$V G G - T$
$I n p u t ? 224 × 224 ? R G B ? i m a g e$	$I n p u t ? R G B ? i m a g e$
$C o n v 1_3 - 64$	$C o n v 1_3 - 64$
$C o n v 1_3 - 64$	$C o n v 1_3 - 64$
$P o o l 1$	$P o o l 1$
$C o n v 2_3 - 128$	$C o n v 2_3 - 128$
$C o n v 2_3 - 128$	$C o n v 2_3 - 128$
$P o o l 2$	$P o o l 2$
$C o n v 3_3 - 256$	$C o n v 3_3 - 256$
$C o n v 3_3 - 256$	$C o n v 3_3 - 256$
$C o n v 3_3 - 256$	$C o n v 3_3 - 256$
$P o o l 3$	$P o o l 3$
$C o n v 4_3 - 512$	$C o n v 4_3 - 512$
$C o n v 4_3 - 512$	$C o n v 4_3 - 512$
$C o n v 4_3 - 512$	$C o n v 4_3 - 512$
$P o o l 4$	1	2
$C o n v 5_3 - 512$	$C o n v$	$P o o l 4$
$C o n v 5_3 - 512$		$C o n v 5_3 - 512$
$C o n v 5_3 - 512$		$C o n v 5_3 - 512$
$P o o l 5$		$C o n v 5_3 - 512$
$F C - 4096$		$C o n v$
$F C - 4096$		$D e c o n v$
$F C - 1000$	$C o m b i n e$
$S o f t - m a x$	$C o m b i n e$

Table 2

Table 3

Table 4

Table 5

Table 6

Cost comparison of 11,350 soybean seed counting time

指标	人工计数	光电种子数粒仪	VGG-T
效率/（粒·s^-1）	1.25	5.56	116.690
总耗时/h	$2.52 ?$	$0.23 ?$	$0.027$

Table 6

References 28

1	韩昕儒, 梅旭荣, 李思经, 等. 中国农业产业发展战略前瞻[J]. 智库理论与实践, 2019, 4(6): 2-7.
	HAN X, MEI X, LI S, et al. The development strategy of China's agricultural industry[J]. Think Tank Theory & Practice, 2019, 4(6): 2-7.
2	ALI A, KHAN S A, EHSANULLAH, et al. Estimation of genetic parameters in soybean for yield and morphological characters[J]. Pakistan Journal of Agriculture, Agricultural Engineering, Veterinary Sciences, 2016, 32(2): 162-168.
3	何进. 不同年代大豆品种籽粒产量差异及其水磷亏缺适应机制[D]. 兰州: 兰州大学, 2016.
	HE J. Grain yield difference of soybean varieties in different ages and its adaptation mechanism to water and phosphorus deficiency[D]. Lanzhou: Lanzhou University, 2016.
4	周洪垒. 基于图像处理的水稻考种系统的设计与实现[D]. 成都: 电子科技大学, 2019.
	ZHOU H. Design and implementation of rice seed test system based on image processing[D]. Chengdu: University of Electronic Science and Technology, 2019.
5	宋礽苏, 华娇, 蓝景针, 等. 转盘斜刮式光电自动数粒仪设计[J]. 农业机械学报, 2011, 42(11): 89-92.
	SONG R, HUA J, LAN J, et al. Design of photoelectric automatic particle counting instrument with rotary table[J]. Transactions of the CSAM, 2011, 42(11): 89-92.
6	荣斐. 基于图像处理的作物种子自动计数软件开发[J]. 工业设计, 2011(7): 126-127.
	RONG F. Development of crop seed automatic counting software based on image processing[J]. Industrial Design, 2011(7): 126-127.
7	崔亮. 基于机器视觉的农作物种子计数检测系统[D]. 太原: 中北大学, 2016.
	CUI L. Crop seed counting detection system based on machine vision[D]. Taiyuan: North China University, 2016.
8	LIU T, CHEN W, WANG Y, et al. Rice and wheat grain counting method and software development based on Android system[J]. Computers and Electronics in Agriculture, 2017(141): 302-309.
9	TAN S, MA X, MAI Z, et al. Segmentation and counting algorithm for touching hybrid rice grains[J]. Computers and Electronics in Agriculture, 2019(162): 493-504.
10	潘锐, 熊勤学, 张文英. 数字图像技术及其在作物表型研究中的应用研究进展[J]. 长江大学学报(自科版), 2016, 13(21): 38-41.
	PAN R, XIONG Q, ZHANG W. Digital image technology and its application in crop phenotype research[J]. Journal of Changjiang University, 2016, 13(21): 38-41.
11	章琳, 袁非牛, 张文睿, 等. 全卷积神经网络研究综述[J]. 计算机工程与应用, 2020, 56(1): 25-37.
	ZHANG L, YUAN F, ZHANG W, et al. A survey of total convolution neural networks [J]. Computer Engineering and Application, 2020, 56(1): 25-37.
12	ALSMIRAT M A, AL-ALEM F, AL-AYYOUB M, et al. Impact of digital fingerprint image quality on the fingerprint recognition accuracy[J]. Multimedia Tools and Applications, 2019, 78(3): 3649-3688.
13	MEDEN B, MALLI R C, FABIJAN S, et al. Face deidentification with generative deep neural networks[J]. IET Signal Processing, 2017, 11(9): 1046-1054.
14	YU H, HE F, PAN Y. A novel segmentation model for medical images with intensity inhomogeneity based on adaptive perturbation[J]. Multimedia Tools and Applications, 2019, 78(9): 11779-11798.
15	LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
16	AICH S, STAVNESS I. Global sum pooling: A generalization trick for object counting with small datasets of large images[J/OL]. arXiv:1805.11123. 2018.
17	POUND M P, ATKINSON J A, WELLS D M, et al. Deep learning for multi-task plant phenotyping[C]// The IEEE International Conference on Computer Vision Workshops. Piscataway, New York, USA: IEEE, 2017: 2055-2063.
18	DENG R, TAO M, HUANG X, et al. Automated counting grains on the rice panicle based on deep learning method[J]. Sensors, 2021, 21(1): 281.
19	WU W, LIU T, ZHOU P, et al. Image analysis-based recognition and quantification of grain number per panicle in rice[J]. Plant Methods, 2019, 15: ID 122.
20	WU W, YANG T, LI RUI, et al. Detection and enumeration of wheat grains based on a deep learning method under various scenarios and scales[J]. Journal of Integrative Agriculture, 2020, 19(8): 1998-2008.
21	翟强, 王陆洋, 殷保群, 等. 基于尺度自适应卷积神经网络的人群计数算法[J]. 计算机工程, 2020, 46(2): 250-254.
	ZHAI Q, WANG L, YIN B, et al. Crowd counting algorithm based on scale adaptive convolution neural network[J]. Computer Engineering, 2020, 46(2): 250-254.
22	AICH S, STAVNESS I. Improving object counting with heatmap regulation[J/OL]. ArXiv:abs/1803. 05494. 2018.
23	LIU Y, SUN P, WERGELES N, et al. A survey and performance evaluation of deep learning methods for small object detection[J]. Expert Systems with Applications, 2021, 172: ID 114602.
24	BABU SAM D, SURYA S, VENKATESH BABU R. Switching convolutional neural network for crowd counting[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2017: 5744-5752.
25	MA Z, WEI X, HONG X, et al. Bayesian loss for crowd count estimation with point supervision[C]// The IEEE/CVF International Conference on Computer Vision. Piscataway, New York, USA: IEEE, 2019: 6142-6151.
26	VARIOR R R, SHUAI B, TIGHE J, et al. Multi-scale attention network for crowd counting[J/OL]. arXiv: 1901.06026. 2019.
27	ZHU L, ZHAO Z, LU C, et al. Dual path multi-scale fusion networks with attention for crowd counting[J/OL]. arXiv: 1902.01115. 2019.
28	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J/OL]. arXiv: 1409.1556. 2014.

数据集	数量/张	籽粒数/个	最少籽粒数/个	最多籽粒数/个	平均籽粒数/个
训练集	206	22,582	67	201	109
验证集	33	3631	70	200	110
测试集	103	11,350	70	200	110
总数	342	37,563	67	201	109

Method	Patch	MAE/（粒·图^-1）	MSE/（粒²·图^-1）
传统形态学操作	无	4.36	7.17
ResNet18	无	2.1	2.4
ResNet18	有	0.7	0.8
VGG16	无	2.2	2.9
VGG16	有	0.4	0.6
ResNet18-T	无	0.7	0.8
ResNet18-T	有	0.2	0.3
VGG-T	无	0.6	0.6
VGG-T	有	0.2	0.3

组别	MAE/ （粒·图^-1）	MSE/ （粒²·图^-1）	真值种子数/粒	估值种子数/粒
组1	0.46	0.53	71	71
组2	0.50	0.59	80	79
组3	0.61	0.57	90	90
组4	0.58	0.52	100	101
组5	0.63	0.55	110	109
组6	0.68	0.65	120	118
组7	0.73	0.63	200	199

[1]	QIU Wenjie, YE Jin, HU Liangqing, YANG Juan, LI Qili, MO Jianyou, YI Wanmao. Distilled-MobileNet Model of Convolutional Neural Network Simplified Structure for Plant Disease Recognition [J]. Smart Agriculture, 2021, 3(1): 109-117.
[2]	LI Hualong, LI Miao, ZHAN Kai, LIU Xianwang, YANG Xuanjiang, HU Zelin, GUO Panpan. Construction Method and Performance Test of Prediction Model for Laying Hen Breeding Environmental Quality Evaluation [J]. Smart Agriculture, 2020, 2(3): 37-47.
[3]	Chen Guifen, Zhao Shan, Cao Liying, Fu Siwei, Zhou Jiaxin. Corn plant disease recognition based on migration learning and convolutional neural network [J]. Smart Agriculture, 2019, 1(2): 34-44.