High-Throughput Dynamic Monitoring Method of Field Maize Seedling

doi:10.12133/j.smartag.2021.3.2.202103-SA003

Abstract

Abstract:

At present, the dynamic detection and monitoring of maize seedling mainly rely on manual observation, which is time-consuming and laborious, and only small quadrats can be selected to estimate the overall emergence situation. In this research, two kinds of data sources, the high-time-series RGB images obtained by the plant high-throughput phenotypic platform (HTPP) and the RGB images obtained by the unmanned aerial vehicle (UAV) platform, were used to construct the image data set of maize seedling process under different light conditions. Considering the complex background and uneven illumination in the field environment, a residual unit based on the Faster R-CNN was built and ResNet50 was used as a new feature extraction network to optimize Faster R-CNN to realize the detection and counting of maize seedlings in complex field environment. Then, based on the high time series image data obtained by the HTPP, the dynamic continuous monitoring of maize seedlings of different varieties and densities was carried out, and the seedling duration and uniformity of each maize variety were evaluated and analyzed. The experimental results showed that the recognition accuracy of the proposed method was 95.67% in sunny days and 91.36% in cloudy days when it was applied to the phenotypic platform in the field. When applied to the UAV platform to monitor the emergence of maize, the recognition accuracy of sunny and cloudy days was 91.43% and 89.77% respectively. The detection accuracy of the phenotyping platform image was higher, which could meet the needs of automatic detection of maize emergence in actual application scenarios. In order to further verify the robustness and generalization of the model, HTPP was used to obtain time series data, and the dynamic emergence of maize was analyzed. The results showed that the dynamic emergence results obtained by HTPP were consistent with the manual observation results, which shows that the model proposed in this research is robust and generalizable.

Key words: field maize, Faster R-CNN, recognition, counting, dynamic seedling detection

CLC Number:

ZHANG Xiaoqing, SHAO Song, GUO Xinyu, FAN Jiangchuan. High-Throughput Dynamic Monitoring Method of Field Maize Seedling[J]. Smart Agriculture, 2021, 3(2): 88-99.

Figures/Tables 18

Fig. 1

Fig. 2

Fig.3

Fig. 4

Table 1

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 2

Fig. 9

Fig. 10

Table 3

Table 4

Table 5

Fig. 11

Fig. 12

Table 6

References 27

1	JIM S. New systems enhance seed and product-placement at planting[J]. Cotton Grower, 2021, 57(1): 16-18.
2	WILES L J, SCHWEIZER E E. The cost of counting and identifying weed seeds and seedlings[J]. Weed Science, 1999, 47(6): 667-673.
3	ZHAO W, YAMADA W, Li T, et al. Augmenting crop detection for precision agriculture with deep visual transfer learning: A case study of bale detection[J]. Remote Sensing, 2021, 13: ID 23.
4	SAKO Y, MCDONALD M, FUJIMURA K, et al. A system for automated seed vigour assessment[J]. Seed Science and Technology, 2001, 29(3): 625-636.
5	HOFMASTER A, FUJIMURA K, MCDONALD M, et al. An automated system for vigor testing three-day-old soybean seedlings[J]. Seed Science and Technology, 2003, 31(3): 701-713.
6	MARCOS-FILHO J, BENNETT M, MCDONALD M, et al. Assessment of melon seed vigour by an automated computer imaging system compared to traditional procedures[J]. Seed Science and Technology, 2006, 34(2): 485-497.
7	FORCELLA F, ARNOLD R L B, SANCHEZ R, et al. Modeling seedling emergence[J]. Field Crops Research, 2000, 67(2): 123-139.
8	FENG A, ZHOU J, VORIES E, et al. Evaluation of cotton emergence using UAV-based narrow-band spectral imagery with customized image alignment and stitching algorithms[J]. Remote Sensing, 2020, 12(11): ID 1764.
9	GNADINGER F, SCHMIDHALTER U. Digital counts of maize plants by unmanned aerial vehicles (UAVs)[J]. Remote sensing, 2017, 9(6): ID 544.
10	REUZEAU C, FRANKARD V, HATZEELD Y, et al. Traitmill™: A functional genomics platform for the phenotypic analysis of cereals[J]. Plant Genetic Resources: Characterisation and Utilisation, 2006, 4: 20-24.
11	VIRLET N, SABERMANESH K, SADEGHI - TEHRAN P, et al. Field scanalyzer: An automated robotic field phenotyping platform for detailed crop monitoring[J]. Functional Plant Biology, 2017, 44(1): 143-153.
12	李寒冰. 基于深度学习的舆情监测系统[D]. 北京: 北京交通大学, 2019.
	LI H. Public opinion monitoring system based on deep learning[D]. Beijing: Beijing Jiaotong University, 2019.
13	李卫. 深度学习在图像识别中的研究及应用[D]. 武汉: 武汉理工大学, 2014.
	LI W. The research and application of deep learning in image recognition[D]. Wuhan: WuhanUniversity of Technology, 2014.
14	史璐璐. 深度学习及其在视频目标跟踪中的应用研究[D]. 南京: 南京邮电大学, 2019.
	SHI L. Research on deep learning and its applications in video object tracking[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2019.
15	GRINBLAT G L, UZAL L, GLARESE M. Deep learning for plant identification using vein morphological patterns[J]. Computers and Electronics in Agriculture, 2016, 127: 418-424.
16	杨建宇, 周振旭, 杜贞容, 等. 基于SegNet语义模型的高分辨率遥感影像农村建设用地提取[J]. 农业工程学报, 2019, 35(5): 251-258.
	YANG J, ZHOU Z, DU Z, et al. Rural construction land extraction from high spatial resolution remote sensing image based on SegNet semantic segmentation model[J]. Transactions of the CSAE, 2019, 35(5): 251-258.
17	HUMPHREY E J, BELLO J P, LECUN Y. Moving beyond feature design: Deep architectures and automatic feature learning in music informatics[C]// The 13th International Society for Music Information Retrieval Conference. Piscataway, New York, USA: IEEE, 2012: 403-408.
18	NANNI L, GHIDONI S, BRAHNAM S. Handcrafted vs. non-handcrafted features for computer vision classification[J]. Pattern Recognition, 2017, 71: 158-172.
19	KRIZHEVSKY A. One weird trick for parallelizing convolutional neural networks[EB/OL]. 2014. .
20	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. 2014. arXiv:1409.1556 [cs.CV].
21	SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]// The IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2015: 1-9.
22	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// The IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC, USA: IEEE Computer Society, 2016: 770-778.
23	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection[C]// The IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway, New York, USA: IEEE, 2016: 779-788.
24	LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot multibox detector[C]// European Conference on Computer Vision. Springer, Cham, Switzerland: 2016: 21-37.
25	REN S, HE K, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(6): 1137-1149.
26	MADEC S, JIN X, LU H, et al. Ear density estimation from high resolution RGB imagery using deep learning technique[J]. Agricultural and Forest Meteorology, 2019, 264: 225-234.
27	ZOU H, LU H, LI Y, et al. Maize tassels detection: A benchmark of the state of the art[J]. Plant Methods, 2020, 16(1): 1-15.

数据来源	训练集图像数量/张	测试集图像数量/张	总数/张
无人机平台	832	356	1188
田间表型平台	840	360	1200
总数	1672	716	2388

网络层	卷积核	通道数	残差结构数量
Layer-1	卷积操作（卷积核7×7，64），Stride=2
Layer-1	最大池化操作（池化矩形3×3），Stride=2
Layer-2	1×1	64	3
	3×3	64
	1×1	256
Layer-3	1×1	128	4
	3×3	128
	1×1	512
Layer-4	1×1	256	6
	3×3	256
	1×1	1024
Layer-5	1×1	512	3
	3×3	512
	1×1	2048
Layer-6	平均池化操作1×1，FC-1000，Softmax

特征提取网络	P/%	R/%	IOU	MAE
VGG	89.42	86.81	75.04	6.2816
ResNet	91.87	90.13	82.57	5.6361

光照情况	P/%	R/%	IOU	MAE
晴天	95.67	93.36	85.57	4.3682
阴天	91.36	89.23	83.04	5.2314

光照情况	P/%	R/%	IOU	MAE
晴天	91.43	89.46	86.01	5.3521
阴天	89.77	87.81	83.04	5.9564