Accurate Extraction of Apple Orchard on the Loess Plateau Based on Improved Linknet Network

doi:10.12133/j.smartag.SA202206001

Abstract

Abstract:

The rapid increasing of apple planting area on the Loess Plateau has exerted an important influence on the regional eco-hydrology and socio-economic development. However, the orchards in this area are small and complex, and there are only county or city scale statistical data, lack of actual spatial distribution information. To this end, for the extraction of apple orchards on the Loess Plateau, in this study, a professional dataset of low-altitude remote sensing images acquired by unmanned aerial vehicle was firstly established. The R_34_Linknet network and other five commonly used deep learning semantic segmentation models SegNet, FCN_8s, DeeplabV3+, UNet and Linknet were applied to the spatial distribution extraction of apple orchards on the Loess Plateau, and the best-performing model was R_34_Linknet, with a F1 score of 87.1%, a pixel accuracy (PA) of 92.3%, an mean intersection over union (MioU) of 81.2%, a frequency weighted intersection over union (FWIoU) of 85.7%, and the mean pixel accuracy (MPA) was 89.6%. The spatial pyramid pool structure (ASPP) and R_34_Linknet network was combined to expand the receptive field of the network and get R_34_Linknet_ASPP network, and then ASPP structure was improved. Combining the spatial pyramid pooling (ASPP) with the R_34_Linknet network to expand the receptive field of the network and obtain a R_34_Linknet_ASPP network; Then the ASPP structure was improved to get a R_34_Linknet_ASPP+ network. The performance of the three networks were compared. R_34_Linknet_ASPP+ got the best performance, with 86.3% for F₁, 94.7% for PA, 82.7% for MIoU, 89.0% for FWIoU, and 92.3% for MPA on the test set. The accuracy of apple orchard extraction in Wangdonggou, Changwu County and Tongji Village, Baishui County using R_34_Linknet_ASPP+ were 94.22% and 95.66%, respectively. In Wangdonggou, it was 1.21% and 0.58% higher than R_34_Linknet and R_34_Linknet_ASPP, respectively. In Tongji village, it was 1.70% and 0.90% higher than R_34_Linknet and R_34_Linknet_ASPP, respectively. The results show that the proposed R_34_Linknet_ASPP+ method can extract apple orchards accurately, the edge treatment of apple orchard plots is better, the method can be used as the technical support and theoretical basis for research on the spatial distribution mapping of apple orchards on the Loess Plateau.

Key words: UAV remote sensing, apple orchard extraction, deep learning, Loess Plateau, transfer learning, residual neural network, semantic segmentation

CLC Number:

S126

ZHANG Zhibo, ZHAO Xining, GAO Xiaodong, ZHANG Li, YANG Menghao. Accurate Extraction of Apple Orchard on the Loess Plateau Based on Improved Linknet Network[J]. Smart Agriculture, 2022, 4(3): 95-107.

Figures/Tables 12

Table 1

Fig. 1

Fig. 2

Table 2

Fig.3

Fig. 4

Table 3

Fig. 5

Fig. 6

Table 4

Table 5

Fig. 7

References 33

1	周江涛, 赵德英, 陈艳辉, 等. 中国苹果产区变动分析[J]. 果树学报, 2021, 38(3): 372-384.
	ZHOU J, ZHAO D, CHEN Y, et al. Analysis of apple producing area changes in China[J]. Journal of Fruit Science, 2021, 38(3): 372-384.
2	GAO X, ZHAO X, WU P, et al. The economic-environmental trade-off of growing apple trees in the drylands of China: A conceptual framework for sustainable intensification[J]. Journal of Cleaner Production, 2021(3): 126-497.
3	康凌艳, 雷玉平, 郑力, 等. 在GIS支持下利用MODIS数据监测多种作物和果树种植面积[J]. 遥感技术与应用, 2007(3): 361-366.
	KNAG L, LEI Y, ZHENG L, alel. Vegetation classification based on modis data and the accuracy evaluation in the pixel scale[J]. Remote Sensing Technology and Application, 2007(3): 361-366.
4	邬明权, 杨良闯, 于博, 等. 基于遥感与多变量概率抽样调查的作物种植面积测量[J]. 农业工程学报, 2014, 30(2): 146-152.
	WU M, YANG L, YU B, et al. Mapping crops acreages based on remote sensing and sampling investigation by multivariate probability proportional to size[J]. Transactions of the CSAE, 2014, 30(2): 146-152.
5	刘佳岐. 基于Landsat8遥感影像的扶风县苹果园地信息提取研究[D]. 杨凌: 西北农林科技大学, 2015.
	LIU J. Research on apple oechards information extraction of fufeng countybased on landsat8 remote sensing image[D]. Yangling: Northwest A&F University, 2015.
6	KUMAR A, SINGH K, LAL B, et al. Mapping of apple orchards using remote sensing techniques in cold desert of himachal pradesh, India[J]. Journal of the Indian Society of Remote Sensing, 2008, 36(4): 387-392.
7	YUAN H, MA R, LUO J. Mapping orchards on plain terrains using multi-temporal medium-resolution satellite imagery[J]. Applied Engineering in Agriculture, 2015, 31: 351-362.
8	徐晗泽宇, 刘冲, 王军邦, 等. Google Earth Engine平台支持下的赣南柑橘果园遥感提取研究[J]. 地球信息科学学报, 2018, 20(3): 396-404.
	XU H, LIU C, WANG J, et al. Study on extraction of citrus orchard in Gannan region based on Google Earth engine platform[J]. Journal of Geo-information Science, 2018, 20(3): 396-404.
9	万亚玲, 钟锡武, 刘慧, 等. 卷积神经网络在高光谱图像分类中的应用综述[J]. 计算机工程与应用, 2021. 57(4): 1-10.
	WAN Y, ZHONG X, LIU H, et al. Survey of application of convolutional neural network in classification of hyperspectral images[J]. Computer Engineering and Applications, 2021, 57(4): 1-10.
10	BERA S, SHRIVASTAVA V K. Analysis of various optimizers on deep convolutional neural network model in the application of hyperspectral remote sensing image classification[J]. International Journal of Remote Sensing, 2020, 41: 2664-2683.
11	MAJEED Y, ZHANG J, ZHANG X, et al. Deep learning based segmentation for automated training of apple trees on trellis wires[J]. Computers and Electronics in Agriculture, 2020, 170: ID 170.
12	JEONGEUN K, JEAHWI S, SUKWOO L, et al. An intelligent spraying system with deep learning-based semantic segmentation of fruit trees in orchards[C]// IEEE International Conference on Robotics and Automation (ICRA). Piscataway, New York, USA: IEEE, 2020: 3923-3929.
13	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2016, 39: 640-651.
14	OLAF R, PHILIPP F, THOMAS B. U-Net: convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Piscataway, New York, USA: IEEE, 2015: 234-241.
15	马金林, 邓媛媛, 马自萍. 肝脏肿瘤CT图像深度学习分割方法综述[J]. 中国图象图形学报, 2020, 25(10): 2024-2046.
	MA J, DENG Y, MA Z. Review of deep learning segmentation methods for CT images of liver tumors[J]. Journal of Image and Graphics, 2020, 25(10): 2024-2046.
16	张凯航, 冀杰, 蒋骆, 等. 基于SegNet的非结构道路可行驶区域语义分割[J]. 重庆大学学报, 2020, 43(3): 79-87.
	ZHANG K, JI J, JIANG L, et al. The segmentation of driving regions on unstructured road based on SegNet architecture[J]. Journal of Chongqing University, 2020, 43(3): 79-87.
17	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J/OL]. arXiv: , 2014.
18	HE K, ZHANG X, REN S, et al. Identity mappings in deep residual networks[J/OL]. arXiv: 1603. 05027, 2016.
19	ANAGNOSTIS A, TAGARAKIS A C, ASIMINARI G, et al. A deep learning approach for anthracnose infected trees classification in walnut orchards[J]. Computers and Electronics in Agriculture, 2021, 182: ID 182.
20	PAPANDREOU G, CHEN L, KEVIN P, et al. Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation[C]// IEEE International Conference on Computer Vision. Piscataway, New York, USA: IEEE, 2015: 1742-1750.
21	CHEN L, PAPANDREOU G, KOKKINOS I. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 40: 834-848.
22	CHEN L, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[J/OL]. arXiv:1706.05587v3, 2017.
23	CHAURASIA A, CULURCIELLO E. LinkNet: exploiting encoder representations for efficient semantic segmentation[C]// IEEE Visual Communications and Image Processing. Piscataway, New York, USA: IEEE, 2017.
24	DENG W, MOU Y, KASHIWA T, et al. Vision based pixel-level bridge structural damage detection using a link ASPP network[J]. Automation in Construction, 2020, 110: ID 102973.
25	YU Y, LI H, WANG J, et al. A multilayer pyramid network based on learning for vehicle logo recognition[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(5): 3123-3134.
26	MO J, LAN Y, YANG D, et al. Deep learning-based instance segmentation method of litchi canopy from UAV-acquired images[J]. Remote Sensing, 2021, 13(19): ID 3919.
27	王爱. 黄土高原苹果树识别与蒸散发过程模拟[D]. 杨凌: 西北农林科技大学, 2020.
	WANG A. Apple tree identification and evapotranspiration process simulation on the Loess Plateau[D]. Yang ling: Northwest A&F University, 2020.
28	王培周. 黄土高原苹果主产区养分投入和土壤养分状况及其空间分布特征[D]. 杨凌: 西北农林科技大学, 2021.
	WANG P. Nutrient input, soil nutrient contents and their spatial distribution characteristics in the main apple producing areas of the Loess Plateau[D]. Yangling: Northwest A&F University, 2021.
29	晏利斌. 1961—2014年黄土高原气温和降水变化趋势[J]. 地球环境学报, 2015, (5): 276-282.
	YAN L. Characteristics of temperature and precipitation on the Loess Plateau from 1961 to 2014[J]. Journal of Earth Environment, 2015, (5): 276-282.
30	吴乾慧, 张勃, 马彬, 等. 气候变暖对黄土高原冬小麦种植区的影响[J]. 生态环境学报, 2017, 26: 429-436.
	WU Q, ZHANG B, MA B, et al. Impact of climate warming on winter wheat planting in the Loess Plateau[J]. Ecology and Environmental Sciences, 2017, 26: 429-436.
31	CHEN L, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[J]. Computer vision-ECCV, 2018, 11211: 833-851.
32	陈鹏飞, 徐新刚. 无人机影像拼接软件在农业中应用的比较研究[J]. 作物学报, 2020, 46(7): 1112-1119.
	CHEN P, XU X. A comparison of photogrammetric software packages for mosaicking unmanned aerial vehicle (UAV) images in agricultural application[J]. Acta Agronomica Sinica, 2020, 46(7): 1112-1119.
33	王慧. PhotoScan在无人机遥感影像数据处理中的应用[J]. 测绘与空间地理信息, 2017, 40(5): 109-111.
	WANG H. Application of PhotoScan in UAV remote sensing image data processing[J]. Geomatics & Spatial Information Technology, 2017, 40(5): 109-111.

分割方法	F₁/%	PA/%	MIoU/%	FWIoU/%	mPA/%
SegNet	84.2	86.0	67.6	76.0	77.9
FCN_8s	85.0	87.4	69.2	76.9	78.6
DeeplabV3+	85.9	89.9	75.8	81.2	88.3
UNet	84.6	89.2	73.8	80.7	82.4
Linknet	86.3	91.7	80.0	85.0	88.6
R_34_linknet	87.1	92.3	81.2	85.7	89.6

分割方法	提取面积/hm²	目视解译面积/hm²	面积精度/%
SegNet	157.17	139.41	87.26
FCN_8s	125.48		90.01
DeeplabV3+	127.30		91.32
UNet	150.48		92.06
Linknet	129.21		92.68
R_34_linknet	129.67		93.01
R_34_Linknet_ASPP	130.54		93.64
R_34_Linknet_ASPP+	131.35		94.22

分割方法	提取面积/hm²	目视解译面积/hm²	面积精度/%
SegNet	49.62	44.97	89.65
FCN_8s	41.04		91.27
DeeplabV3+	41.94		93.26
UNet	48.31		92.57
Linknet	42.16		93.76
R_34_linknet	42.24		93.94
R_34_Linknet_ASPP	42.52		94.56
R_34_Linknet_ASPP+	42.93		95.46

[1]	HE Ruimin, ZHENG Kefeng, WEI Qinyang, ZHANG Xiaobin, ZHANG Jun, ZHU Yihang, ZHAO Yiying, GU Qing. Identification and Counting of Silkworms in Factory Farm Using Improved Mask R-CNN Model [J]. Smart Agriculture, 2022, 4(2): 163-173.
[2]	ZHUANG Jiayu, XU Shiwei, LI Yang, XIONG Lu, LIU Kebao, ZHONG Zhiping. Supply and Demand Forecasting Model of Multi-Agricultural Products Based on Deep Learning [J]. Smart Agriculture, 2022, 4(2): 174-182.
[3]	CHEN Zhanqi, ZHANG Yu'an, WANG Wenzhi, LI Dan, HE Jie, SONG Rende. Multiscale Feature Fusion Yak Face Recognition Algorithm Based on Transfer Learning [J]. Smart Agriculture, 2022, 4(2): 77-85.
[4]	SHAO Mingyue, ZHANG Jianhua, FENG Quan, CHAI Xiujuan, ZHANG Ning, ZHANG Wenrong. Research Progress of Deep Learning in Detection and Recognition of Plant Leaf Diseases [J]. Smart Agriculture, 2022, 4(1): 29-46.
[5]	YANG Guofeng, HE Yong, FENG Xuping, LI Xiyao, ZHANG Jinnuo, YU Zeyu. Methods and New Research Progress of Remote Sensing Monitoring of Crop Disease and Pest Stress Using Unmanned Aerial Vehicle [J]. Smart Agriculture, 2022, 4(1): 1-16.
[6]	ZHOU Qiaoli, MA Li, CAO Liying, YU Helong. Identification of Tomato Leaf Diseases Based on Improved Lightweight Convolutional Neural Networks MobileNetV3 [J]. Smart Agriculture, 2022, 4(1): 47-56.
[7]	LI Zhijun, YANG Shenghui, SHI Deshuai, LIU Xingxing, ZHENG Yongjun. Yield Estimation Method of Apple Tree Based on Improved Lightweight YOLOv5 [J]. Smart Agriculture, 2021, 3(2): 100-114.
[8]	FLORES Paulo, ZHANG Zhao. Wheat Lodging Ratio Detection Based on UAS Imagery Coupled with Different Machine Learning and Deep Learning Algorithms [J]. Smart Agriculture, 2021, 3(2): 23-34.
[9]	QIU Wenjie, YE Jin, HU Liangqing, YANG Juan, LI Qili, MO Jianyou, YI Wanmao. Distilled-MobileNet Model of Convolutional Neural Network Simplified Structure for Plant Disease Recognition [J]. Smart Agriculture, 2021, 3(1): 109-117.
[10]	LI Liangde, WANG Xiujuan, KANG Mengzhen, HUA Jing, FAN Menghan. Agricultural Named Entity Recognition Based on Semantic Aggregation and Model Distillation [J]. Smart Agriculture, 2021, 3(1): 118-128.
[11]	SHU Meiyan, CHEN Xiangyang, WANG Xiqing, MA Yuntao. Estimation of Maize Leaf Area Index and Aboveground Biomass Based on Hyperspectral Data [J]. Smart Agriculture, 2021, 3(1): 29-39.
[12]	GUO Wei, WU Huarui, ZHU Huaji. Design and Application of Facility Greenhouse Image Collecting and Environmental Data Monitoring Robot System [J]. Smart Agriculture, 2020, 2(3): 48-60.
[13]	LI Hualong, LI Miao, ZHAN Kai, LIU Xianwang, YANG Xuanjiang, HU Zelin, GUO Panpan. Construction Method and Performance Test of Prediction Model for Laying Hen Breeding Environmental Quality Evaluation [J]. Smart Agriculture, 2020, 2(3): 37-47.
[14]	WEI Jing, WANG Yuting, YUAN Huizhu, ZHANG Menglei, WANG Zhenying. Identification and Morphological Analysis of Adult Spodoptera Frugiperda and Its Close Related Species Using Deep Learning [J]. Smart Agriculture, 2020, 2(3): 75-85.
[15]	FLORES Paulo, ZHANG Zhao, MATHEW Jithin, JAHAN Nusrat, STENGER John. Distinguishing Volunteer Corn from Soybean at Seedling Stage Using Images and Machine Learning [J]. Smart Agriculture, 2020, 2(3): 61-74.