基于改进Linknet网络的黄土高原苹果园精准提取

doi:10.12133/j.smartag.SA202206001

摘要/Abstract

摘要：

黄土高原近20年来苹果栽植面积迅猛增加，对区域生态水文和社会经济发展均产生了重要影响。但该区域果园地块小且场景复杂，仅有县/市尺度统计数据，尚无苹果园实际的空间分布信息。为此，本研究建立了无人机低空遥感影像专业数据集。融合迁移学习与深度学习方法，将残差神经网络ResNet34网络迁移到Linknet网络，得到R_34_Linknet网络。将R_34_Linknet网络与5种常用的深度学习语义分割模型SegNet、FCN_8s、DeeplabV3+、UNet和Linknet应用于黄土高原苹果园空间分布提取，表现最好的模型为R_34_Linknet，其在测试集上的调和平均值F₁为87.1%，像素准确度PA为92.3%，均交并比MIoU为81.2%，频权交并比FWIoU为85.7%，平均像素准确度MPA为89.6%。将空间金字塔池化结构（Atrous Spatial Pyramid Pooling，ASPP）与R_34_Linknet网络相结合，扩大网络的感受野，得到R_34_Linknet_ASPP网络；然后对ASPP结构进行改进，得到R_34_Linknet_ASPP+网络。对比三种网络性能，表现最优的为R_34_Linknet_ASPP+，在测试集上F₁为86.3%，PA为94.7%，MIoU为82.7%，FWIoU为89.0%，MPA为92.3%。使用R_34_Linknet_ASPP+在长武县王东沟和白水县通积村提取苹果园面积精度分别为94.22%和95.66%。本研究提出的R_34_Linknet_ASPP+方法提取到的苹果园更加准确，苹果园地块边缘处效果更好，可作为黄土高原苹果园空间分布制图等研究的技术支撑和理论依据。

关键词: 无人机遥感, 苹果园提取, 深度学习, 黄土高原, 迁移学习, 残差神经网络, 语义分割

Abstract:

The rapid increasing of apple planting area on the Loess Plateau has exerted an important influence on the regional eco-hydrology and socio-economic development. However, the orchards in this area are small and complex, and there are only county or city scale statistical data, lack of actual spatial distribution information. To this end, for the extraction of apple orchards on the Loess Plateau, in this study, a professional dataset of low-altitude remote sensing images acquired by unmanned aerial vehicle was firstly established. The R_34_Linknet network and other five commonly used deep learning semantic segmentation models SegNet, FCN_8s, DeeplabV3+, UNet and Linknet were applied to the spatial distribution extraction of apple orchards on the Loess Plateau, and the best-performing model was R_34_Linknet, with a F1 score of 87.1%, a pixel accuracy (PA) of 92.3%, an mean intersection over union (MioU) of 81.2%, a frequency weighted intersection over union (FWIoU) of 85.7%, and the mean pixel accuracy (MPA) was 89.6%. The spatial pyramid pool structure (ASPP) and R_34_Linknet network was combined to expand the receptive field of the network and get R_34_Linknet_ASPP network, and then ASPP structure was improved. Combining the spatial pyramid pooling (ASPP) with the R_34_Linknet network to expand the receptive field of the network and obtain a R_34_Linknet_ASPP network; Then the ASPP structure was improved to get a R_34_Linknet_ASPP+ network. The performance of the three networks were compared. R_34_Linknet_ASPP+ got the best performance, with 86.3% for F₁, 94.7% for PA, 82.7% for MIoU, 89.0% for FWIoU, and 92.3% for MPA on the test set. The accuracy of apple orchard extraction in Wangdonggou, Changwu County and Tongji Village, Baishui County using R_34_Linknet_ASPP+ were 94.22% and 95.66%, respectively. In Wangdonggou, it was 1.21% and 0.58% higher than R_34_Linknet and R_34_Linknet_ASPP, respectively. In Tongji village, it was 1.70% and 0.90% higher than R_34_Linknet and R_34_Linknet_ASPP, respectively. The results show that the proposed R_34_Linknet_ASPP+ method can extract apple orchards accurately, the edge treatment of apple orchard plots is better, the method can be used as the technical support and theoretical basis for research on the spatial distribution mapping of apple orchards on the Loess Plateau.

Key words: UAV remote sensing, apple orchard extraction, deep learning, Loess Plateau, transfer learning, residual neural network, semantic segmentation

中图分类号:

S126

张志博, 赵西宁, 高晓东, 张利, 杨孟豪. 基于改进Linknet网络的黄土高原苹果园精准提取[J]. 智慧农业(中英文), 2022, 4(3): 95-107.

ZHANG Zhibo, ZHAO Xining, GAO Xiaodong, ZHANG Li, YANG Menghao. Accurate Extraction of Apple Orchard on the Loess Plateau Based on Improved Linknet Network[J]. Smart Agriculture, 2022, 4(3): 95-107.

图/表 12

表1

图1

图2

表2

图3

图4

表3

图5

图6

表4

表5

图7

参考文献 33

1	周江涛, 赵德英, 陈艳辉, 等. 中国苹果产区变动分析[J]. 果树学报, 2021, 38(3): 372-384.
	ZHOU J, ZHAO D, CHEN Y, et al. Analysis of apple producing area changes in China[J]. Journal of Fruit Science, 2021, 38(3): 372-384.
2	GAO X, ZHAO X, WU P, et al. The economic-environmental trade-off of growing apple trees in the drylands of China: A conceptual framework for sustainable intensification[J]. Journal of Cleaner Production, 2021(3): 126-497.
3	康凌艳, 雷玉平, 郑力, 等. 在GIS支持下利用MODIS数据监测多种作物和果树种植面积[J]. 遥感技术与应用, 2007(3): 361-366.
	KNAG L, LEI Y, ZHENG L, alel. Vegetation classification based on modis data and the accuracy evaluation in the pixel scale[J]. Remote Sensing Technology and Application, 2007(3): 361-366.
4	邬明权, 杨良闯, 于博, 等. 基于遥感与多变量概率抽样调查的作物种植面积测量[J]. 农业工程学报, 2014, 30(2): 146-152.
	WU M, YANG L, YU B, et al. Mapping crops acreages based on remote sensing and sampling investigation by multivariate probability proportional to size[J]. Transactions of the CSAE, 2014, 30(2): 146-152.
5	刘佳岐. 基于Landsat8遥感影像的扶风县苹果园地信息提取研究[D]. 杨凌: 西北农林科技大学, 2015.
	LIU J. Research on apple oechards information extraction of fufeng countybased on landsat8 remote sensing image[D]. Yangling: Northwest A&F University, 2015.
6	KUMAR A, SINGH K, LAL B, et al. Mapping of apple orchards using remote sensing techniques in cold desert of himachal pradesh, India[J]. Journal of the Indian Society of Remote Sensing, 2008, 36(4): 387-392.
7	YUAN H, MA R, LUO J. Mapping orchards on plain terrains using multi-temporal medium-resolution satellite imagery[J]. Applied Engineering in Agriculture, 2015, 31: 351-362.
8	徐晗泽宇, 刘冲, 王军邦, 等. Google Earth Engine平台支持下的赣南柑橘果园遥感提取研究[J]. 地球信息科学学报, 2018, 20(3): 396-404.
	XU H, LIU C, WANG J, et al. Study on extraction of citrus orchard in Gannan region based on Google Earth engine platform[J]. Journal of Geo-information Science, 2018, 20(3): 396-404.
9	万亚玲, 钟锡武, 刘慧, 等. 卷积神经网络在高光谱图像分类中的应用综述[J]. 计算机工程与应用, 2021. 57(4): 1-10.
	WAN Y, ZHONG X, LIU H, et al. Survey of application of convolutional neural network in classification of hyperspectral images[J]. Computer Engineering and Applications, 2021, 57(4): 1-10.
10	BERA S, SHRIVASTAVA V K. Analysis of various optimizers on deep convolutional neural network model in the application of hyperspectral remote sensing image classification[J]. International Journal of Remote Sensing, 2020, 41: 2664-2683.
11	MAJEED Y, ZHANG J, ZHANG X, et al. Deep learning based segmentation for automated training of apple trees on trellis wires[J]. Computers and Electronics in Agriculture, 2020, 170: ID 170.
12	JEONGEUN K, JEAHWI S, SUKWOO L, et al. An intelligent spraying system with deep learning-based semantic segmentation of fruit trees in orchards[C]// IEEE International Conference on Robotics and Automation (ICRA). Piscataway, New York, USA: IEEE, 2020: 3923-3929.
13	LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2016, 39: 640-651.
14	OLAF R, PHILIPP F, THOMAS B. U-Net: convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Piscataway, New York, USA: IEEE, 2015: 234-241.
15	马金林, 邓媛媛, 马自萍. 肝脏肿瘤CT图像深度学习分割方法综述[J]. 中国图象图形学报, 2020, 25(10): 2024-2046.
	MA J, DENG Y, MA Z. Review of deep learning segmentation methods for CT images of liver tumors[J]. Journal of Image and Graphics, 2020, 25(10): 2024-2046.
16	张凯航, 冀杰, 蒋骆, 等. 基于SegNet的非结构道路可行驶区域语义分割[J]. 重庆大学学报, 2020, 43(3): 79-87.
	ZHANG K, JI J, JIANG L, et al. The segmentation of driving regions on unstructured road based on SegNet architecture[J]. Journal of Chongqing University, 2020, 43(3): 79-87.
17	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J/OL]. arXiv: , 2014.
18	HE K, ZHANG X, REN S, et al. Identity mappings in deep residual networks[J/OL]. arXiv: 1603. 05027, 2016.
19	ANAGNOSTIS A, TAGARAKIS A C, ASIMINARI G, et al. A deep learning approach for anthracnose infected trees classification in walnut orchards[J]. Computers and Electronics in Agriculture, 2021, 182: ID 182.
20	PAPANDREOU G, CHEN L, KEVIN P, et al. Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation[C]// IEEE International Conference on Computer Vision. Piscataway, New York, USA: IEEE, 2015: 1742-1750.
21	CHEN L, PAPANDREOU G, KOKKINOS I. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 40: 834-848.
22	CHEN L, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[J/OL]. arXiv:1706.05587v3, 2017.
23	CHAURASIA A, CULURCIELLO E. LinkNet: exploiting encoder representations for efficient semantic segmentation[C]// IEEE Visual Communications and Image Processing. Piscataway, New York, USA: IEEE, 2017.
24	DENG W, MOU Y, KASHIWA T, et al. Vision based pixel-level bridge structural damage detection using a link ASPP network[J]. Automation in Construction, 2020, 110: ID 102973.
25	YU Y, LI H, WANG J, et al. A multilayer pyramid network based on learning for vehicle logo recognition[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(5): 3123-3134.
26	MO J, LAN Y, YANG D, et al. Deep learning-based instance segmentation method of litchi canopy from UAV-acquired images[J]. Remote Sensing, 2021, 13(19): ID 3919.
27	王爱. 黄土高原苹果树识别与蒸散发过程模拟[D]. 杨凌: 西北农林科技大学, 2020.
	WANG A. Apple tree identification and evapotranspiration process simulation on the Loess Plateau[D]. Yang ling: Northwest A&F University, 2020.
28	王培周. 黄土高原苹果主产区养分投入和土壤养分状况及其空间分布特征[D]. 杨凌: 西北农林科技大学, 2021.
	WANG P. Nutrient input, soil nutrient contents and their spatial distribution characteristics in the main apple producing areas of the Loess Plateau[D]. Yangling: Northwest A&F University, 2021.
29	晏利斌. 1961—2014年黄土高原气温和降水变化趋势[J]. 地球环境学报, 2015, (5): 276-282.
	YAN L. Characteristics of temperature and precipitation on the Loess Plateau from 1961 to 2014[J]. Journal of Earth Environment, 2015, (5): 276-282.
30	吴乾慧, 张勃, 马彬, 等. 气候变暖对黄土高原冬小麦种植区的影响[J]. 生态环境学报, 2017, 26: 429-436.
	WU Q, ZHANG B, MA B, et al. Impact of climate warming on winter wheat planting in the Loess Plateau[J]. Ecology and Environmental Sciences, 2017, 26: 429-436.
31	CHEN L, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[J]. Computer vision-ECCV, 2018, 11211: 833-851.
32	陈鹏飞, 徐新刚. 无人机影像拼接软件在农业中应用的比较研究[J]. 作物学报, 2020, 46(7): 1112-1119.
	CHEN P, XU X. A comparison of photogrammetric software packages for mosaicking unmanned aerial vehicle (UAV) images in agricultural application[J]. Acta Agronomica Sinica, 2020, 46(7): 1112-1119.
33	王慧. PhotoScan在无人机遥感影像数据处理中的应用[J]. 测绘与空间地理信息, 2017, 40(5): 109-111.
	WANG H. Application of PhotoScan in UAV remote sensing image data processing[J]. Geomatics & Spatial Information Technology, 2017, 40(5): 109-111.

分割方法	F₁/%	PA/%	MIoU/%	FWIoU/%	mPA/%
SegNet	84.2	86.0	67.6	76.0	77.9
FCN_8s	85.0	87.4	69.2	76.9	78.6
DeeplabV3+	85.9	89.9	75.8	81.2	88.3
UNet	84.6	89.2	73.8	80.7	82.4
Linknet	86.3	91.7	80.0	85.0	88.6
R_34_linknet	87.1	92.3	81.2	85.7	89.6

分割方法	提取面积/hm²	目视解译面积/hm²	面积精度/%
SegNet	157.17	139.41	87.26
FCN_8s	125.48		90.01
DeeplabV3+	127.30		91.32
UNet	150.48		92.06
Linknet	129.21		92.68
R_34_linknet	129.67		93.01
R_34_Linknet_ASPP	130.54		93.64
R_34_Linknet_ASPP+	131.35		94.22

分割方法	提取面积/hm²	目视解译面积/hm²	面积精度/%
SegNet	49.62	44.97	89.65
FCN_8s	41.04		91.27
DeeplabV3+	41.94		93.26
UNet	48.31		92.57
Linknet	42.16		93.76
R_34_linknet	42.24		93.94
R_34_Linknet_ASPP	42.52		94.56
R_34_Linknet_ASPP+	42.93		95.46

[1]	何锐敏, 郑可锋, 尉钦洋, 张小斌, 张俊, 朱怡航, 赵懿滢, 顾清. 基于改进Mask R-CNN模型的工厂化养蚕蚕体识别与计数[J]. 智慧农业(中英文), 2022, 4(2): 163-173.
[2]	庄家煜, 许世卫, 李杨, 熊露, 刘克宝, 钟志平. 基于深度学习的多种农产品供需预测模型[J]. 智慧农业(中英文), 2022, 4(2): 174-182.
[3]	陈占琦, 张玉安, 王文志, 李丹, 何杰, 宋仁德. 基于迁移学习的多尺度特征融合牦牛脸部识别算法[J]. 智慧农业(中英文), 2022, 4(2): 77-85.
[4]	邵明月, 张建华, 冯全, 柴秀娟, 张凝, 张文蓉. 深度学习在植物叶部病害检测与识别的研究进展[J]. 智慧农业(中英文), 2022, 4(1): 29-46.
[5]	杨国峰, 何勇, 冯旭萍, 李禧尧, 张金诺, 俞泽宇. 无人机遥感监测作物病虫害胁迫方法与最新研究进展[J]. 智慧农业(中英文), 2022, 4(1): 1-16.
[6]	周巧黎, 马丽, 曹丽英, 于合龙. 基于改进轻量级卷积神经网络MobileNetV3的番茄叶片病害识别[J]. 智慧农业(中英文), 2022, 4(1): 47-56.
[7]	李志军, 杨圣慧, 史德帅, 刘星星, 郑永军. 基于轻量化改进YOLOv5的苹果树产量测定方法[J]. 智慧农业（中英文）, 2021, 3(2): 100-114.
[8]	. 基于无人机图像以及不同机器学习和深度学习模型的小麦倒伏率检测[J]. 智慧农业（中英文）, 2021, 3(2): 23-34.
[9]	邱文杰, 叶进, 胡亮青, 杨娟, 李其利, 莫贱友, 易万茂. 面向植物病害识别的卷积神经网络精简结构Distilled-MobileNet模型[J]. 智慧农业(中英文), 2021, 3(1): 109-117.
[10]	李亮德, 王秀娟, 康孟珍, 华净, 樊梦涵. 基于语义融合与模型蒸馏的农业实体识别[J]. 智慧农业(中英文), 2021, 3(1): 118-128.
[11]	束美艳, 陈向阳, 王喜庆, 马韫韬. 基于高光谱数据的玉米叶面积指数和生物量评估[J]. 智慧农业(中英文), 2021, 3(1): 29-39.
[12]	郭威, 吴华瑞, 朱华吉. 设施温室影像采集与环境监测机器人系统设计及应用[J]. 智慧农业（中英文）, 2020, 2(3): 48-60.
[13]	李华龙, 李淼, 詹凯, 刘先旺, 杨选将, 胡泽林, 郭盼盼. 蛋鸡设施养殖环境质量评价预测模型构建方法及性能测试[J]. 智慧农业（中英文）, 2020, 2(3): 37-47.
[14]	魏靖, 王玉亭, 袁会珠, 张梦蕾, 王振营. 基于深度学习与特征可视化方法的草地贪夜蛾及其近缘种成虫识别[J]. 智慧农业（中英文）, 2020, 2(3): 75-85.
[15]	FLORESPaulo, MATHEWJithin, JAHANNusrat, STENGERJohn. 利用图像和机器学习检测大豆作物幼苗期玉米杂苗[J]. 智慧农业（中英文）, 2020, 2(3): 61-74.