农业大模型：关键技术、应用分析与发展方向

doi:10.12133/j.smartag.SA202403015

Smart Agriculture ›› 2024, Vol. 6 ›› Issue (2): 1-13.doi: 10.12133/j.smartag.SA202403015

• 专刊--农业信息感知与模型 • 下一篇

农业大模型：关键技术、应用分析与发展方向

郭旺¹^,²^,³, 杨雨森¹^,⁴, 吴华瑞¹^,²^,³(), 朱华吉¹^,²^,³, 缪祎晟¹^,²^,³, 顾静秋¹^,²^,³

^1. 国家农业信息化工程技术研究中心, 北京 100097, 中国
^2. 北京市农林科学院信息技术研究中心, 北京 100097, 中国
^3. 农业农村部数字乡村技术重点实验室, 北京 100097, 中国
^4. 新加坡国立大学设计与工程学院, 新加坡 117583, 新加坡

收稿日期:2024-03-13 出版日期:2024-03-30
基金资助:
科技创新2030“新一代人工智能”重大项目(2021ZD0113604); 财政部和农业农村部：国家现代农业产业技术体系资助(CARS-23-D07); 北京市农林科学院创新能力建设项目(KJCX20230219)
作者简介:
郭旺，研究方向为农业人工智能与智能系统。E-mail：guow007@nercita.org.cn
通信作者:
吴华瑞，博士，研究员，研究方向为农业人工智能与大模型。E-mail：wuhr007@nercita.org.cn

Big Models in Agriculture: Key Technologies, Application and Future Directions

GUO Wang¹^,²^,³, YANG Yusen¹^,⁴, WU Huarui¹^,²^,³(), ZHU Huaji¹^,²^,³, MIAO Yisheng¹^,²^,³, GU Jingqiu¹^,²^,³

^1. National Engineering Research Center for Information Technology in Agriculture, Beijing 100097, China
^2. Information Technology Research Center, Beijing Academy of Agriculture and Forestry Sciences, Beijing 100097, China
^3. Key Laboratory of Digital Village Technology, Ministry of Agriculture and Rural Affairs, Beijing 100097, China
^4. Collage of Design and Engineering, National University of Singapore, Singapore 117583, Singapore

Received:2024-03-13 Online:2024-03-30
corresponding author:
WU Huarui, E-mail: wuhr007@nercita.org.cn
About author:
GUO Wang, E-mail: guow007@nercita.org.cn
Supported by:
Innovation 2030 Major S&T Projects of China(2021ZD0113604); China Agriculture Research System of MOF and MARA Grant(CARS-23-D07); Beijing Academy of Agricultural and Forestry Sciences: Innovation Capacity Building Project(KJCX20230219)

摘要/Abstract

摘要：

［目的/意义］ 近年来，人工智能在农业领域的应用取得了显著进展，但仍面临诸如模型数据收集标记困难、模型泛化能力弱等挑战。大模型技术作为近期人工智能领域新的热点技术，已在多个行业的垂直领域中展现出了良好性能，尤其在复杂关联表示、模型泛化、多模态信息处理等方面较传统机器学习方法有着较大优势。［进展］ 本文首先阐述了大模型的基本概念和核心技术方法，展示了在参数规模扩大与自监督训练下，模型通用能力与下游适应能力的显著提升。随后，分析了大模型在农业领域应用的主要场景；按照语言大模型、视觉大模型和多模态大模型三大类，在阐述模型发展的同时重点介绍在农业领域的应用现状，展示了大模型在农业上取得的研究进展。［结论/展望］ 对农业大模型数据集少而分散、模型部署难度大、农业应用场景复杂等困难提出见解，展望了农业大模型未来的发展重点方向。预计大模型将在未来提供全面综合的农业决策系统，并为公众提供专业优质的农业服务。

关键词: 生成式人工智能, 大模型, 农业知识服务, 机器学习, 自主决策, 多模态, 深度学习

Abstract:

[Significance] Big Models, or Foundation Models, have offered a new paradigm in smart agriculture. These models, built on the Transformer architecture, incorporate numerous parameters and have undergone extensive training, often showing excellent performance and adaptability, making them effective in addressing agricultural issues where data is limited. Integrating big models in agriculture promises to pave the way for a more comprehensive form of agricultural intelligence, capable of processing diverse inputs, making informed decisions, and potentially overseeing entire farming systems autonomously. [Progress] The fundamental concepts and core technologies of big models are initially elaborated from five aspects: the generation and core principles of the Transformer architecture, scaling laws of extending big models, large-scale self-supervised learning, the general capabilities and adaptions of big models, and the emerging capabilities of big models. Subsequently, the possible application scenarios of the big model in the agricultural field are analyzed in detail, the development status of big models is described based on three types of the models: Large language models (LLMs), large vision models (LVMs), and large multi-modal models (LMMs). The progress of applying big models in agriculture is discussed, and the achievements are presented. [Conclusions and Prospects] The challenges and key tasks of applying big models technology in agriculture are analyzed. Firstly, the current datasets used for agricultural big models are somewhat limited, and the process of constructing these datasets can be both expensive and potentially problematic in terms of copyright issues. There is a call for creating more extensive, more openly accessible datasets to facilitate future advancements. Secondly, the complexity of big models, due to their extensive parameter counts, poses significant challenges in terms of training and deployment. However, there is optimism that future methodological improvements will streamline these processes by optimizing memory and computational efficiency, thereby enhancing the performance of big models in agriculture. Thirdly, these advanced models demonstrate strong proficiency in analyzing image and text data, suggesting potential future applications in integrating real-time data from IoT devices and the Internet to make informed decisions, manage multi-modal data, and potentially operate machinery within autonomous agricultural systems. Finally, the dissemination and implementation of these big models in the public agricultural sphere are deemed crucial. The public availability of these models is expected to refine their capabilities through user feedback and alleviate the workload on humans by providing sophisticated and accurate agricultural advice, which could revolutionize agricultural practices.

Key words: artificial intelligence generated content (AIGC), big models, agricultural knowledge services, machine learning, autonomous decision-making, multi-modality, deep learning

郭旺, 杨雨森, 吴华瑞, 朱华吉, 缪祎晟, 顾静秋. 农业大模型：关键技术、应用分析与发展方向[J]. 智慧农业(中英文), 2024, 6(2): 1-13.

GUO Wang, YANG Yusen, WU Huarui, ZHU Huaji, MIAO Yisheng, GU Jingqiu. Big Models in Agriculture: Key Technologies, Application and Future Directions[J]. Smart Agriculture, 2024, 6(2): 1-13.

图/表 4

参考文献 60

1	YAO J, YI X, WANG X, et al. From instructions to intrinsic human values: A survey of alignment goals for big models[EB/OL]. arXiv, 2023.
2	BOMMASANI R, HUDSON D A, ADELI E, et al. On the opportunities and risks of foundation models[EB/OL]. arXiv, 2022.
3	VASWANI A, SHAZEER N, PARMAR N,et al. Attention is all you need[J]. Advances in neural information processing systems, 2017, 30.
4	RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative pre-training[EB/OL]. OpenAI, 2018.
5	TAN C, CAO Q, LI Y, et al. On the promises and challenges of multimodal foundation models for geographical, environmental, agricultural, and urban planning applications[EB/OL]. arXiv, 2023.
6	AWAIS M, NASEER M, KHAN S, et al. Foundational models defining a new era in vision: A survey and outlook[EB/OL]. arXiv, 2023.
7	ZHAO W X, ZHOU K, LI J, et al. A Survey of large language models[EB/OL]. arXiv, 2023.
8	WEI J, TAY Y, BOMMASANI R, et al. Emergent abilities of large language models[EB/OL]. arXiv, 2022[2024-03-06].
9	DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Minneapolis, USA: Association for Computational Linguistics, 2019: 4171-4186.
10	BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[EB/OL]. arXiv, 2020.
11	TOUVRON H, LAVRIL T, IZACARD G, et al. LLaMA: Open and efficient foundation language models[EB/OL]. arXiv, 2023.
12	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[EB/OL]. arXiv, 2021.
13	CARON M, TOUVRON H, MISRA I,et al. Emerging properties in self-supervised vision transformers[C]// 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, New Jersey, USA: IEEE, 2021: 9630-9640.
14	KAPLAN J, MCCANDLISH S, HENIGHAN T, et al. Scaling laws for neural language models[EB/OL]. arXiv, 2020.
15	HOFFMANN J, BORGEAUD S, MENSCH A, et al. Training compute-optimal large language models[EB/OL]. arXiv, 2022.
16	CHEN T, KORNBLITH S, NOROUZI M, et al. A simple framework for contrastive learning of visual representations[EB/OL]. arXiv, 2020.
17	RAJBHANDARI S, RASLEY J, RUWASE O, et al. ZeRO: Memory optimizations toward training trillion parameter models[C]// Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. Piscataway, New Jersey, USA: IEEE, 2020: 1-16.
18	KIRILLOV A, MINTUN E, RAVI N, et al. Segment anything[EB/OL]. arXiv, 2023.
19	OUYANG L, WU J, JIANG X, et al. Training language models to follow instructions with human feedback[EB/OL]. arXiv, 2022.
20	HU E J, SHEN Y, WALLIS P, et al. LoRA: Low-rank adaptation of large language models[EB/OL]. arXiv, 2021.
21	WEI J, WANG X, SCHUURMANS D, et al. Chain-of-thought prompting elicits reasoning in large language models[EB/OL]. arXiv, 2023.
22	RADFORD A, WU J, CHILD R, et al. Language models are unsupervised multitask learners[R/OL]. 2019.
23	OPENAI, ACHIAM J, ADLER S, et al. GPT-4 technical report[EB/OL]. arXiv, 2023.
24	LEWIS M, LIU Y H, GOYAL N, et al. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Minneapolis, Minnesota, USA: Association for Computational Linguistics, 2020: 7871-7880.
25	RAFFEL C, SHAZEER N, ROBERTS A, et al. Exploring the limits of transfer learning with a unified text-to-text transformer[J]. Journal of machine learning research, 2020, 21(140): 1-67.
26	CHUNG H W, HOU L, LONGPRE S, et al. Scaling instruction-finetuned language models[EB/OL]. arXiv, 2022.
27	WORKSHOP B, SCAO T L, FAN A, et al. BLOOM: A 176B-Parameter open-access multilingual language model[EB/OL]. arXiv, 2023.
28	TOUVRON H, MARTIN L, STONE K, et al. Llama 2: Open foundation and fine-tuned chat models[EB/OL]. arXiv, 2023.
29	ZENG A, LIU X, DU Z, et al. GLM-130B: An open bilingual pre-trained model[C]// The Eleventh International Conference on Learning Representations. Piscataway, New Jersey, USA: IEEE, 2022.
30	REZAYI S, LIU Z L, WU Z H, et al. AgriBERT: Knowledge-infused agricultural language models for matching food and nutrition[C]// Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence. Vienna, Austria: International Joint Conferences on Artificial Intelligence Organization, 2022: 5150-5156.
31	YANG X, GAO J, XUE W, et al. PLLaMa: An open-source large language model for plant science[EB/OL]. arXiv, 2024. [2024-01-23].
32	ZHAO B, JIN W Q, DEL SER J, et al. ChatAgri: Exploring potentials of ChatGPT on cross-linguistic agricultural text classification[J]. Neurocomputing, 2023, 557: ID 126708.
33	SILVA B, NUNES L, ESTEVÃO R, et al. GPT-4 as an agronomist assistant? Answering agriculture exams using large language models[EB/OL]. arXiv, 2023.
34	王婷, 王娜, 崔运鹏, 等. 基于人工智能大模型技术的果蔬农技知识智能问答系统[J]. 智慧农业(中英文), 2023, 5(4): 105-116.
	WANG T, WANG N, CUI Y P, et al. Agricultural technology knowledge intelligent question-answering system based on large language model[J]. Smart agriculture, 2023, 5(4): 105-116.
35	BALAGUER A, BENARA V, CUNHA R L DE F, et al. RAG vs fine-tuning: Pipelines, tradeoffs, and a case study on agriculture[EB/OL]. arXiv, 2024.
36	QING J J, DENG X L, LAN Y B, et al. GPT-aided diagnosis on agricultural image based on a new light YOLOPC[J]. Computers and electronics in agriculture, 2023, 213: ID 108168.
37	REDMON J, DIVVALA S, GIRSHICK R, et al. You Only Look Once: Unified, real-time object detection[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New Jersey, USA: IEEE, 2016: 779-788.
38	PENG R L, LIU K, YANG P, et al. Embedding-based retrieval with LLM for effective agriculture information extracting from unstructured data[J]. CoRR, 2023, ID abs/2308.03107.
39	YUAN L, CHEN D, CHEN Y L, et al. Florence: A new foundation model for computer vision[EB/OL]. arXiv, 2021.
40	WILLIAMS D, MACFARLANE F, BRITTEN A. Leaf Only SAM: Asegment anything pipeline for zero-shot automated leaf segmentation[EB/OL]. arXiv, 2023.
41	CARRARO A, SOZZI M, MARINELLO F. The Segment Anything Model (SAM) for accelerating the smart farming revolution[J]. Smart agricultural technology, 2023, 6: ID 100367.
42	LI Y, WANG D, YUAN C, et al. Enhancing agricultural image segmentation with an agricultural segment anything model adapter[J]. Sensors (basel), 2023, 23(18): ID 7884.
43	YANG X, DAI H, WU Z, et al. SAM for poultry science[EB/OL]. arXiv, 2023.
44	XIE E, WANG W, YU Z, et al. SegFormer: Simple and efficient design for semantic segmentation with transformers[EB/OL]. arXiv, 2021.
45	ZHENG S, LU J, ZHAO H, et al. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, New Jersey, USA: IEEE, 2021: 6877-6886.
46	GUI B L, BHARDWAJ A, SAM L. Evaluating the efficacy of segment anything model for delineating agriculture and urban green spaces in multiresolution aerial and spaceborne remote sensing images[J]. Remote sensing, 2024, 16(2): ID 414.
47	GURAV R, PATEL H, SHANG Z C, et al. Can SAM recognize crops? Quantifying the zero-shot performance of a semantic segmentation foundation model on generating crop-type maps using satellite imagery for precision agriculture[EB/OL]. arXiv, 2023: arXiv: 2311.15138.
48	LIU X Y. A SAM-based method for large-scale crop field boundary delineation[C]// 2023 20th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON). Piscataway, New Jersey, USA: IEEE, 2023: 1-6.
49	ZHANG C, MARFATIA P, FARHAN H, et al. Enhancing USDA NASS cropland data layer with segment anything model[C]// 2023 11th International Conference on Agro-Geoinformatics (Agro-Geoinformatics). Piscataway, New Jersey, USA: IEEE, 2023: 1-5.
50	RADFORD A, KIM J W, HALLACY C, et al. Learning transferable visual models from natural language supervision[C]// International Conference on Machine Learning. New York, USA: ACM, 2021.
51	ALAYRAC J B, DONAHUE J, LUC P, et al. Flamingo: A visual language model for few-shot learning[EB/OL]. arXiv, 2022.
52	RAMESH A, PAVLOV M, GOH G, et al. Zero-shot text-to-image generation[EB/OL]. arXiv, 2021. [2024-03-06].
53	KINGMA D P, WELLING M. Auto-encoding variational bayes[EB/OL]. arXiv, 2013.
54	BROOKS T, PEEBLES B, HOLMES C, et al. Video generation models as world simulators[EB/OL]. [2024-03-03].
55	CAO Y Y, CHEN L, YUAN Y, et al. Cucumber disease recognition with small samples using image-text-label-based multi-modal language model[J]. Computers and electronics in agriculture, 2023, 211: ID 107993.
56	MU N, KIRILLOV A, WAGNER D, et al. SLIP: Self-supervision meets language-image pre-training[M]// Lecture Notes in Computer Science. Cham: Springer Nature Switzerland, 2022: 529-544.
57	DETTMERS T, PAGNONI A, HOLTZMAN A, et al. QLoRA: Efficient Finetuning of quantized LLMs[EB/OL]. arXiv, 2023.
58	FRANTAR E, ASHKBOOS S, HOEFLER T, et al. OPTQ: Accurate quantization for generative pre-trained transformers[C]// The Eleventh International Conference on Learning Representations. Piscataway, New Jersey, USA: IEEE, 2022.
59	TZACHOR A, DEVARE M, RICHARDS C, et al. Large language models and agricultural extension services[J]. Nature food, 2023, 4(11): 941-948.
60	STELLA F, DELLA SANTINA C, HUGHES J. How can LLMs transform the robotic design process?[J]. Nature machine intelligence, 2023, 5(6): 561-564.

[1]	李豪, 杜雨秋, 肖星竹, 陈彦羲. 基于深度学习的四川盆地丘陵区县域耕地遥感识别研究[J]. 智慧农业(中英文), 2024, 6(3): 34-45.
[2]	聂刚刚, 饶洪辉, 李泽锋, 刘木华. 基于改进YOLACT的油茶叶片炭疽病感染严重程度分级模型[J]. 智慧农业(中英文), 2024, 6(3): 138-147.
[3]	张京, 赵泽瑄, 赵艳茹, 卜泓超, 吴星宇. 基于Bi-GRU和空-谱信息融合的油菜菌核病侵染区域高光谱图像分割方法[J]. 智慧农业(中英文), 2024, 6(2): 40-48.
[4]	庞春晖, 陈鹏, 夏懿, 章军, 王兵, 邹岩, 陈天娇, 康辰瑞, 梁栋. 用于小麦多生长阶段倒伏边界精准检测的分层交互特征金字塔网络[J]. 智慧农业(中英文), 2024, 6(2): 128-139.
[5]	张玉玉, 邴树营, 纪元浩, 严蓓蓓, 许金普. 基于改进YOLOv8s的玫瑰鲜切花分级方法[J]. 智慧农业(中英文), 2024, 6(2): 118-127.
[6]	沈艳艳, 赵玉涛, 陈庚申, 吕振刚, 赵峰, 杨万能, 孟冉. 玉米典型叶部病害高光谱识别及其烈度分类[J]. 智慧农业(中英文), 2024, 6(2): 28-39.
[7]	张建华, 姚琼, 周国民, 吴雯迪, 修晓杰, 王健. 作物农艺性状与形态结构表型智能识别技术综述[J]. 智慧农业(中英文), 2024, 6(2): 14-27.
[8]	贾文珅, 吕浩林, 张上, 秦英栋, 周巍. 利用便捷式可见-近红外光谱仪和机器学习分辨霉变小麦及霉变程度[J]. 智慧农业(中英文), 2024, 6(1): 89-100.
[9]	王鹤榕, 陈英义, 柴莹倩, 徐玲, 于辉辉. 融合VoVNetv2和置换注意力机制的鱼群摄食图像分割方法[J]. 智慧农业(中英文), 2023, 5(4): 137-149.
[10]	王婷, 王娜, 崔运鹏, 刘娟. 基于人工智能大模型技术的果蔬农技知识智能问答系统[J]. 智慧农业(中英文), 2023, 5(4): 105-116.
[11]	李政凯, 于嘉辉, 潘时佳, 贾泽丰, 牛子杰. 冬季猕猴桃树单木骨架提取与冠层生长预测方法[J]. 智慧农业(中英文), 2023, 5(4): 92-104.
[12]	唐辉, 王铭, 于秋实, 张佳茜, 刘连涛, 王楠. 融合改进UNet和迁移学习的棉花根系图像分割方法[J]. 智慧农业(中英文), 2023, 5(3): 96-109.
[13]	泮玮婷, 孙梦丽, 员琰, 刘平. 基于深度学习ImCascade R-CNN的小麦籽粒表形鉴定方法[J]. 智慧农业(中英文), 2023, 5(3): 110-120.
[14]	管博伦, 张立平, 朱静波, 李闰枚, 孔娟娟, 汪焱, 董伟. 农业病虫害图像数据集构建关键问题及评价方法综述[J]. 智慧农业(中英文), 2023, 5(3): 17-34.
[15]	叶大鹏, 陈晨, 李慧琳, 雷莹晓, 翁海勇, 瞿芳芳. 基于深度卷积生成式对抗网络的菌草丙二醛含量可见/近红外光谱反演[J]. 智慧农业(中英文), 2023, 5(3): 132-141.

农业大模型：关键技术、应用分析与发展方向

Big Models in Agriculture: Key Technologies, Application and Future Directions

在线阅读

知网下载

本地下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 4

参考文献 60

相关文章 15

编辑推荐

Metrics

本文评价