[Objective] Desert vegetation is an indispensable part of desert ecosystems, and its conservation and restoration are crucial. Accurate identification of desert plants is an indispensable task, and is the basis of desert ecological research and conservation. The complex growth environment caused by light, soil, shadow and other vegetation increases the recognition difficulty, and the generalization ability is poor and the recognition accuracy is not guaranteed. The rapid development of modern technology provides new opportunities for plant identification and classification. By using intelligent identification algorithms, field investigators can be effectively assisted in desert plant identification and classification, thus improve efficiency and accuracy, while reduce the associated human and material costs. [Methods] In this research, the following works were carried out for the recognition of desert plant: Firstly, a training dataset of deep learning model of desert plant images in the arid and semi-arid region of Xinjiang was constructed to provide data resources and basic support for the classification and recognition of desert plant images.The desert plant image data was collected in Changji and Tacheng region from the end of September 2021 and July to August 2022, and named DPlants50. The dataset contains 50 plant species in 13 families and 43 genera with a total of 12,507 images, and the number of images for each plant ranges from 183 to 339. Secondly, a migration integration learning-based algorithm for desert plant image recognition was proposed, which could effectively improve the recognition accuracy. Taking the EfficientNet B0-B4 network as the base network, the ImageNet dataset was pre-trained by migration learning, and then an integrated learning strategy was adopted combining Bagging and Stacking, which was divided into two layers. The first layer introduced K-fold cross-validation to divide the dataset and trained K sub-models by borrowing the Stacking method. Considering that the output features of each model were the same in this study, the second layer used Bagging to integrate the output features of the first layer model by voting method, and the difference was that the same sub-models and K sub-models were compared to select the better model, so as to build the integrated model, reduce the model bias and variance, and improve the recognition performance of the model. For 50 types of desert plants, 20% of the data was divided as the test set, and the remaining 5 fold cross validation was used to divide the dataset, then can use DPi(i=1,2,…,5) represents each training or validation set. Based on the pre trained EfficientNet B0-B4 network, training and validation were conducted on 5 data subsets. Finally, the model was integrated using soft voting, hard voting, and weighted voting methods, and tested on the test set. [Results and Discussions] The results showed that the Top-1 accuracy of the single sub-model based on EfficientNet B0 network was 92.26%~93.35%, the accuracy of the Ensemble-Soft model with soft voting, the Ensemble-Hard model with hard voting and the Ensemble-Weight model integrated by weighted voting method were 93.63%, 93.55% and 93.67%, F1 Score and accuracy were comparable, the accuracy and F1 Score of Ensemble-Weight model integrated by weighted voting method were not significantly improved compared with Ensemble-Soft model and Ensemble-hard model, but it showed that the effect of weighted voting method proposed in this study was better than both of them. The three integrated models demonstrate no noteworthy enhancements in accuracy and F1 Score when juxtaposed with the five sub-models. This observation results suggests that the homogeneity among the models constrains the effectiveness of the voting method strategy. Moreover, the recognition effects heavily hinges on the performance of the EfficientNet B0-DP5 model. Therefore, the inclusion of networks with more pronounced differences was considered as sub-models. A single sub-model based on EfficientNet B0-B4 network had the highest Top-1 accuracy of 96.65% and F1 Score of 96.71%, while Ensemble-Soft model, Ensemble-Hard model and Ensemble-Weight model got the accuracy of 99.07%, 98.91% and 99.23%, which further improved the accuracy compared to the single sub-model, and the F1 Score was basically the same as the accuracy rate, and the model performance was significant. The model integrated by the weighted voting method also improved accuracy and F1 Score for both soft and hard voting, with significant model performance and better recognition, again indicating that the weighted voting method was more effective than the other two. Validated on the publicly available dataset Oxford Flowers102, the three integrated models improved the accuracy and F1 Score of the three sub-models compared to the five sub-models by a maximum of 4.56% and 5.05%, and a minimum of 1.94% and 2.29%, which proved that the migration and integration learning strategy proposed in this paper could effectively improve the model performances. [Conclusions] In this study, a method to recognize desert plant images in natural context by integrating migration learning and integration learning was proposed, which could improve the recognition accuracy of desert plants up to 99.23% and provide a solution to the problems of low accuracy, model robustness and weak generalization of plant images in real field environment. After transferring to the server through the cloud, it can realize the accurate recognition of desert plants and serve the scenes of field investigation, teaching science and scientific experiment.
[Objective] Accurate prediction of changes in sugarcane yield in Guangxi can provide important reference for the formulation of relevant policies by the government and provide decision-making basis for farmers to guide sugarcane planting, thereby improving sugarcane yield and quality and promoting the development of the sugarcane industry. This research was conducted to provide scientific data support for sugar factories and related management departments, explore the relationship between sugarcane yield and meteorological factors in the main sugarcane producing areas of Guangxi Zhuang Autonomous Region. [Methods] The study area included five sugarcane planting regions which laid in five different counties in Guangxi, China. The average yields per hectare of each planting regions were provided by Guangxi Sugar Industry Group which controls the sugar refineries of each planting region. The daily meteorological data including 14 meteorological factors from 2002 to 2019 were acquired from National Data Center for Meteorological Sciences to analyze their influences placed on sugarcane yield. Since meteorological factors could pose different influences on sugarcane growth during different time spans, a new kind of factor which includes meteorological factors and time spans was defined, such as the average precipitation in August, the average temperature from February to April, etc. And then the inter-correlation of all the meteorological factors of different time spans and their correlations with yields were analyzed to screen out the key meteorological factors of sensitive time spans. After that, four algorithms of BP neural network (BPNN), support vector machine (SVM), random forest (RF), and long short-term memory (LSTM) were employed to establish sugarcane apparent yield prediction models for each planting region. Their corresponding reference models based on the annual meteorological factors were also built. Additionally, the meteorological yields of every planting region were extracted by HP filtering, and a general meteorological yield prediction model was built based on the data of all the five planting regions by using RF, SVM BPNN, and LSTM, respectively. [Results and Discussions] The correlation analysis showed that different planting regions have different sensitive meteorological factors and key time spans. The highly representative meteorological factors mainly included sunshine hours, precipitation, and atmospheric pressure. According to the results of correlation analysis, in Region 1, the highest negative correlation coefficient with yield was observed at the sunshine hours during October and November, while the highest positive correlation coefficient was found at the minimum relative humidity in November. In Region 2, the maximum positive correlation coefficient with yield was observed at the average vapor pressure during February and March, whereas the maximum negative correlation coefficient was associated with the precipitation in August and September. In Region 3, the maximum positive correlation coefficient with yield was found at the 20‒20 precipitation during August and September, while the maximum negative correlation coefficient was related to sunshine hours in the same period. In Region 4, the maximum positive correlation coefficient with yield was observed at the 20‒20 precipitation from March to December, whereas the maximum negative correlation coefficient was associated with the highest atmospheric pressure from August to December. In Region 5, the maximum positive correlation coefficient with yield was found at the average vapor pressure from June and to August, whereas the maximum negative correlation coefficient as related to the lowest atmospheric pressure in February and March. For each specific planting region, the accuracy of apparent yield prediction model based on sensitive meteorological factors during key time spans was obviously better than that based on the annual average meteorological values. The LSTM model performed significantly better than the widely used classic BPNN, SVM, and RF models for both kinds of meteorological factors (under sensitive time spans or annually). The overall root mean square error (RMSE) and mean absolute percentage error (MAPE) of the LSTM model under key time spans were 10.34 t/ha and 6.85%, respectively, with a coefficient of determination Rv2 of 0.8489 between the predicted values and true values. For the general prediction models of the meteorological yield to multiple the sugarcane planting regions, the RF, SVM, and BPNN models achieved good results, and the best prediction performance went to BPNN model, with an RMSE of 0.98 t/ha, MAPE of 9.59%, and Rv2 of 0.965. The RMSE and MAPE of the LSTM model were 0.25 t/ha and 39.99%, respectively, and the Rv2 was 0.77. [Conclusions] Sensitive meteorological factors under key time spans were found to be more significantly correlated with the yields than the annual average meteorological factors. LSTM model shows better performances on apparent yield prediction for specific planting region than the classic BPNN, SVM, and RF models, but BPNN model showed better results than other models in predicting meteorological yield over multiple sugarcane planting regions.
[Objective] Leaf chlorophyll content (LCC) of flue-cured Tobacco is an important indicator for characterizing the photosynthesis, nutritional status, and growth of the crop. Tobacco is an important economic crop with leaves as the main harvest object, it is crucial to monitor its LCC. Hyperspectral data can be used for the rapid estimation of LCC in flue-cured tobacco leaves, making it of great significance and application value. The purpose of this study was to efficiently and accurately estimate the LCC of flue-cured tobacco during different growth stages. [Methods] Zhongyan 100 was chose as the research object, five nitrogen fertilization levels were set. In each plot, three plants were randomly and destructively sampled, resulting in a total of 45 ground samples for each data collection. After transplanting, the reflectance data of the flue-cured tobacco canopy at six growth stages (32, 48, 61, 75, 89, and 109 d ) were collected using a UAV equipped with a Resonon Pika L hyperspectral. Spectral indices for the LCC estimation model of flue-cured tobacco were screened in two ways: (1) based on 18 published vegetation indices sensitive to LCC of crop leaves; (2) based on random combinations of any two bands in the wavelength range of 400‒1000 nm. The Difference Spectral Index (DSI), Ratio Spectral Index (RSI), and Normalized Spectral Index (NDSI) were calculated and plotted against LCC. The correlations between the three spectral indices and leaf LCC were calculated and plotted using contour maps. Five regression models, unary linear regression (ULR), multivariable linear regression (MLR), partial least squares regression (PLSR), support vector regression (SVR), and random forest regression (RFR), were used to estimate the chlorophyll content. A regression estimate model of LCC based on various combinations of spectral indices was eventually constructed by comparing the prediction accuracies of single spectral index models multiple spectral index models at different growth stages. Results and Discussions] The results showed that the LCC range for six growth stages was 0.52‒2.95 mg/g. The standard deviation and coefficient of variation values demonstrated a high degree of dispersion in LCC, indicating differences in fertility between different treatments at the test site and ensuring the applicability of the estimation model within a certain range. Except for 109 d after transplanting, most vegetation indices were significantly correlated with LCC (p<0.01). Compared with traditional vegetation indices, the newly combined spectral indices significantly improved the correlation with LCC. The sensitive bands at each growth stage were relatively concentrated, and the spectral index combinations got high correlation with LCC were mainly distributed between 780‒940 nm and 520‒710 nm. The sensitive bands for the whole growth stages were relatively dispersed, and there was little difference in the position of sensitive band between different spectral indices. For the univariate LCC estimation model, the highest modeling accuracy was achieved using the newly combined Normalized Spectral Index and Red Light Ratio Spectral Index at 75 d after transplanting. The coefficients of determination (R2 ) and root mean square errors (RMSE) for the modeling and validation sets were 0.822, 0.814, and 0.226, 0.230, respectively. The prediction results of the five resgression models showed that the RFR algorithm based on multivariate data performed best in LCC estimation. The R2 and RMSE of the modeling set using data at 75 d after transplanting were 0.891 and 0.205, while those of the validation set reached 0.919 and 0.146. In addition, the estimation performance of the univariate model based on the whole growth stages dataset was not ideal, with R2 of 0.636 and 0.686, and RMSE of 0.333 and 0.304 for the modeling and validation sets, respectively. However, the estimation accuracy of the model based on multiple spectral parameters was significantly improved in the whole growth stages dataset, with R2 of 0.854 and 0.802, and RMSE of 0.206 and 0.264 for the modeling and validation sets of the LCC-RFR model, respectively. In addition, in the whole growth stages dataset, the estimation accuracy of the LCC-RFR model was better than that of the LCC-MLR, LCC-PLSR, and LCC-SVR models. Compared with the modeling set, R2 increased by 19.06%, 18.62%, and 29.51%, while RMSE decreased by 31.93%, 29.51%, and 28.24%. Compared with the validation set, R2 increased by 8.21%, 12.62%, and 8.17%, while RMSE decreased by 3.76%, 9.33%, and 4.55%. [Conclusions] The sensitivity of vegetation indices (VIs) to LCC is closely connected to the tobacco growth stage, according to the results this study, which examined the reaction patterns of several spectral indices to LCC in flue-cured tobacco. The sensitivity of VIs to LCC at various growth stages is critical for crop parameter assessment using UAV hyperspectral photography. Five estimation models for LCC in flue-cured tobacco leaves were developed, with the LCC-RFR model demonstrating the greatest accuracy and stability. The RFR model is less prone to overfitting and can efficiently decrease outlier and noise interference. This work could provide theoretical and technological references for LCC estimate and flue-cured tobacco growth monitoring.
[Objective] To quickly and accurately assess the situation of crop lodging disasters, it is necessary to promptly obtain information such as the location and area of the lodging occurrences. Currently, there are no corresponding technical standards for identifying crop lodging based on UAV remote sensing, which is not conducive to standardizing the process of obtaining UAV data and proposing solutions to problems. This study aims to explore the impact of different spatial resolution remote sensing images and feature optimization methods on the accuracy of identifying wheat lodging areas. [Methods] Digital orthophoto images (DOM) and digital surface models (DSM) were collected by UAVs with high-resolution sensors at different flight altitudes after wheat lodging. The spatial resolutions of these image data were 1.05, 2.09, and 3.26 cm. A full feature set was constructed by extracting 5 spectral features, 2 height features, 5 vegetation indices, and 40 texture features from the pre-processed data. Then three feature selection methods, ReliefF algorithm, RF-RFE algorithm, and Boruta-Shap algorithm, were used to construct an optimized subset of features at different flight altitudes to select the best feature selection method. The ReliefF algorithm retains features with weights greater than 0.2 by setting a threshold of 0.2; the RF-RFE algorithm quantitatively evaluated the importance of each feature and introduces variables in descending order of importance to determine classification accuracy; the Boruta-Shap algorithm performed feature subset screening on the full feature set and labels a feature as green when its importance score was higher than that of the shaded feature, defining it as an important variable for model construction. Based on the above-mentioned feature subset, an object-oriented classification model on remote sensing images was conducted using eCognition9.0 software. Firstly, after several experiments, the feature parameters for multi-scale segmentation in the object-oriented classification were determined, namely a segmentation scale of 1, a shape factor of 0.1, and a tightness of 0.5. Three object-oriented supervised classification algorithms, support vector machine (SVM), random forest (RF), and K nearest neighbor (KNN), were selected to construct wheat lodging classification models. The Overall classification accuracy and Kappa coefficient were used to evaluate the accuracy of wheat lodging identification. By constructing a wheat lodging classification model, the appropriate classification strategy was clarified and a technical path for lodging classification was established. This technical path can be used for wheat lodging monitoring, providing a scientific basis for agricultural production and improving agricultural production efficiency. [Results and Discussions] The results showed that increasing the altitude of the UAV to 90 m significantly improved flight efficiency of wheat lodging areas. In comparison to flying at 30 m for the same monitoring range, data acquisition time was reduced to approximately 1/6th, and the number of photos needed decreased from 62 to 6. In terms of classification accuracy, the overall classification effect of SVM is better than that of RF and KNN. Additionally, when the image spatial resolution varied from 1.05 to 3.26 cm, the full feature set and all three optimized feature subsets had the highest classification accuracy at a resolution of 1.05 cm, which was better than at resolutions of 2.09 and 3.26 cm. As the image spatial resolution decreased, the overall classification effect gradually deteriorated and the positioning accuracy decreased, resulting in poor spatial consistency of the classification results. Further research has found that the Boruta-Shap feature selection method can reduce data dimensionality and improve computational speed while maintaining high classification accuracy. Among the three tested spatial resolution conditions (1.05, 2.09, and 3.26 cm), the combination of SVM and Boruta-Shap algorithms demonstrated the highest overall classification accuracy. Specifically, the accuracy rates were 95.6%, 94.6%, and 93.9% for the respective spatial resolutions. These results highlighted the superior performance of this combination in accurately classifying the data and adapt to changes in spatial resolution. When the image resolution was 3.26 cm, the overall classification accuracy decreased by 1.81% and 0.75% compared to 1.05 and 2.09 cm; when the image resolution was 2.09 cm, the overall classification accuracy decreased by 1.06% compared to 1.05 cm, showing a relatively small difference in classification accuracy under different flight altitudes. The overall classification accuracy at an altitude of 90 m reached 95.6%, with Kappa coefficient of 0.914, meeting the requirements for classification accuracy. [Conclusions] The study shows that the object-oriented SVM classifier and the Boruta-Shap feature optimization algorithm have strong application extension advantages in identifying lodging areas in remote sensing images at multiple flight altitudes. These methods can achieve high-precision crop lodging area identification and reduce the influence of image spatial resolution on model stability. This helps to increase flight altitude, expand the monitoring range, improve UAV operation efficiency, and reduce flight costs. In practical applications, it is possible to strike a balance between classification accuracy and efficiency based on specific requirements and the actual scenario, thus providing guidance and support for the development of strategies for acquiring crop lodging information and evaluating wheat disasters.
[Objective] To address the problems of low efficiency and high cost of traditional manual identification of pests and diseases, improve the automatic recognition of pests and diseases by introducing advanced technical means, and provide feasible technical solutions for agricultural pest and disease monitoring and prevention and control, a rice image recognition model GE-DenseNet (G-ECA DenseNet) based on improved ECA (Efficient Channel Attention) mechanism with DenseNet201 was proposed. [Methods] The leaf images of three pests and diseases, namely, brownspot, hispa, leafblast and healthy rice were selected as experimental materials. The images were captured at the Zhuanghe Rice Professional Cooperative in Yizheng, Jiangsu Province, and the camera was used to manually take pictures from multiple angles such as the top and side of rice every 2 h, thus acquiring 1250 images of rice leaves under different lighting conditions, different perspectives, and different shading environments. In addition, samples about pests and diseases were collected in the Kaggle database. There were 1488 healthy leaves, 523 images of brownspot, 565 images of hispa, and 779 images of leafblast in the dataset. Since the original features of the pest and disease data were relatively close, firstly, the dataset was divided into a training set and a test set according to the ratio of 9:1, and then data enhancement was performed on the training set. A region of interest (ROI) was randomly selected to achieve a local scale of 1.1 to 1.25 for the sample images of the dataset, thus simulating the situation that only part of the leaves were captured in the actual shooting process due to the different distance of the plants from the camera. In addition, a random rotation of a certain angle was used to crop the image to simulate the different angles of the leaves. Finally, the experimental training set contains 18,018 images and the test set contains 352 images. The GE-DenseNet model firstly introduces the idea of Ghost module on the ECA attention mechanism to constitute the G-ECA Layer structure, which replaces the convolution operation with linear transformation to perform efficient fusion of channel features while avoiding dimensionality reduction when learning channel attention information and effectively enhancing its ability to extract features. Secondly, since the original Dense Block only considered the correlation between different layers and ignores the extraction of important channel information in the image recognition process, introducing G-ECA Layer before the original Dense Block of DenseNet201 gives the model a better channel feature extraction capability and thus improved the recognition accuracy. Due to the small dataset used in the experiment, the weight parameters of DenseNet201 pre-trained on the ImageNet dataset were migrated to GE-DenseNet. During the training process, the BatchSize size was set to 32, the number of iterations (Epoch) was set to 50, and the Focal Loss function was used to solve the problem of unbalanced samples for each classification. Meanwhile, the adaptive moment estimation (Adam) optimizer was used to avoid the problem of drastic gradient changes in back propagation due to random initialization of some weights at the early stage of model training, which weakened the uncertainty of network training to a certain extent. [Results and Discussions] Experimental tests were conducted on a homemade dataset of rice pests and diseases, and the recognition accuracy reached 83.52%. Comparing the accuracy change graphs and loss rate change graphs of GE-DenseNet and DenseNet201, it could be found that the proposed method in this study was effective in training stability, which could accelerate the speed of model convergence and improve the stability of the model, making the network training process more stable. And observing the visualization results of GE-DenseNet and DenseNet201 corresponding feature layers, it could be found that the features were more densely reflected around the pests and diseases after adding the G-ECA Layer structure. From the ablation comparison experiments of the GE-DenseNet model, it could be obtained that the model accuracy increased by 2.27% after the introduction of the Focal Loss function with the G-ECA Layer layer. Comparing the proposed model with the classical NasNet (4@1056), VGG-16 and ResNet50 models, the classification accuracy increased by 6.53%, 4.83% and 3.69%, respectively. Compared with the original DenseNet201, the recognition accuracy of hispa improved 20.32%. [Conclusions] The experimental results showed that the addition of G-ECA Layer structure enables the model to more accurately capture feature information suitable for rice pest recognition, thus enabling the GE-DenseNet model to achieve more accurate recognition of different rice pest images. This provides reliable technical support for timely pest and disease control, reducing crop yield loss and pesticide use. Future research can lighten the model and reduce its size without significantly reducing the recognition accuracy, so that it can be deployed in UAVs, tractors and various distributed image detection edge devices to facilitate farmers to conduct real-time inspection of farmland and further enhance the intelligence of agricultural production.
[Objective] Pineapple is a common tropical fruit, and its ripeness has an important impact on the storage and marketing. It is particularly important to analyze the maturity of pineapple fruit before picking. Deep learning technology can be an effective method to achieve automatic recognition of pineapple maturity. To improve the accuracy and rate of automatic recognition of pineapple maturity, a new network model named MobileNet V3-YOLOv4 was proposed in this study. [Methods] Firstly, pineapple maturity analysis data set was constructed. A total of 1580 images were obtained, with 1264 images selected as the training set, 158 images as the validation set, and 158 images as the test set. Pineapple photos were taken in natural environment. In order to ensure the diversity of the data set and improve the robustness and generalization of the network, pineapple photos were taken under the influence of different factors such as branches and leaves occlusion, uneven lighting, overlapping shadows, etc. and the location, weather and growing environment of the collection were different. Then, according to the maturity index of pineapple, the photos of pineapple with different maturity were marked, and the labels were divided into yellow ripeness and green ripeness. The annotated images were taken as data sets and input into the network for training. Aiming at the problems of the traditional YOLOv4 network, such as large number of parameters, complex network structure and slow reasoning speed, a more optimized lightweight MobileNet V3-YOLOv4 network model was proposed. The model utilizes the benck structure to replace the Resblock in the CSPDarknet backbone network of YOLOv4. Meanwhile, in order to verify the effectiveness of the MobileNet V3-YOLOv4 network, MobileNet V1-YOLOv4 model and MobileNet V2-YOLOv4 model were also trained. Five different single-stage and two-stage network models, including R-CNN, YOLOv3, SSD300, Retinanet and Centernet were compared with each evaluation index to analyze the performance superiority of MobileNet V3-YOLOv4 model. Results and Discussions] MobileNet V3-YOLOv4 was validated for its effectiveness in pineapple maturity detection through experiments comparing model performance, model classification prediction, and accuracy tests in complex pineapple detection environments.The experimental results show that, in terms of model performance comparison, the training time of MobileNet V3-YOLOv4 was 11,924 s, with an average training time of 39.75 s per round, the number of parameters was 53.7 MB, resulting in a 25.59% reduction in the saturation time compared to YOLOv4, and the parameter count accounted for only 22%. The mean average precision (mAP) of the trained MobileNet V3-YOLOv4 in the verification set was 53.7 MB. In order to validate the classification prediction performance of the MobileNet V3-YOLOv4 model, four metrics, including Recall score, F1 Score, Precision, and average precision (AP), were utilized to classify and recognize pineapples of different maturities. The experimental results demonstrate that MobileNet V3-YOLOv4 exhibited significantly higher Precision, AP, and F1 Score the other. For the semi-ripe stage, there was a 4.49% increase in AP, 0.07 improvement in F1 Score, 1% increase in Recall, and 3.34% increase in Precision than YOLOv4. As for the ripe stage, there was a 6.06% increase in AP, 0.13 improvement in F1 Score, 16.55% increase in Recall, and 6.25% increase in Precision. Due to the distinct color features of ripe pineapples and their easy differentiation from the background, the improved network achieved a precision rate of 100.00%. Additionally, the mAP and reasoning speed (Frames Per Second, FPS) of nine algorithms were examined. The results showed that MobileNet V3-YOLOv4 achieved an mAP of 90.92%, which was 5.28% higher than YOLOv4 and 3.67% higher than YOLOv3. The FPS was measured at 80.85 img/s, which was 40.28 img/s higher than YOLOv4 and 8.91 img/s higher than SSD300. The detection results of MobileNet V3-YOLOv4 for pineapples of different maturities in complex environments indicated a 100% success rate for both the semi-ripe and ripe stages, while YOLOv4, MobileNet V1-YOLOv4, and MobileNet V2-YOLOv4 exhibited varying degrees of missed detections. [Conclusions] Based on the above experimental results, it can be concluded that MobileNet V3-YOLOv4 proposed in this study could not only reduce the training speed and parameter number number, but also improve the accuracy and reasoning speed of pineapple maturity recognition, so it has important application prospects in the field of smart orchard. At the same time, the pineapple photo data set collected in this research can also provide valuable data resources for the research and application of related fields.
[Objective] Rapid recognition and automatic positioning of table grapes in the natural environment is the prerequisite for the automatic picking of table grapes by the picking robot. [Methods] An rapid recognition and automatic picking points positioning method based on improved K-means clustering algorithm and contour analysis was proposed. First, euclidean distance was replaced by a weighted gray threshold as the judgment basis of K-means similarity. Then the images of table grapes were rasterized according to the K value, and the initial clustering center was obtained. Next, the average gray value of each cluster and the percentage of pixel points of each cluster in the total pixel points were calculated. And the weighted gray threshold was obtained by the average gray value and percentage of adjacent clusters. Then, the clustering was considered as have ended until the weighted gray threshold remained unchanged. Therefore, the cluster image of table grape was obtained. The improved clustering algorithm not only saved the clustering time, but also ensured that the K value could change adaptively. Moreover, the adaptive Otsu algorithm was used to extract grape cluster information, so that the initial binary image of the table grape was obtained. In order to reduce the interference of redundant noise on recognition accuracy, the morphological algorithms (open operation, close operation, images filling and the maximum connected domain) were used to remove noise, so the accurate binary image of table grapes was obtained. And then, the contours of table grapes were obtained by the Sobel operator. Furthermore, table grape clusters grew perpendicular to the ground due to gravity in the natural environment. Therefore, the extreme point and center of gravity point of the grape cluster were obtained based on contour analysis. In addition, the linear bundle where the extreme point and the center of gravity point located was taken as the carrier, and the similarity of pixel points on both sides of the linear bundle were taken as the judgment basis. The line corresponding to the lowest similarity value was taken as the grape stem, so the stem axis of the grape was located. Moreover, according to the agronomic picking requirements of table grapes, and combined with contour analysis, the region of interest (ROI) in picking points could be obtained. Among them, the intersection of the grapes stem and the contour was regarded as the middle point of the bottom edge of the ROI. And the 0.8 times distance between the left and right extreme points was regarded as the length of the ROI, the 0.25 times distance between the gravity point and the intersection of the grape stem and the contour was regarded as the height of the ROI. After that, the central point of the ROI was captured. Then, the nearest point between the center point of the ROI and the grape stem was determined, and this point on the grape stem was taken as the picking point of the table grapes. Finally, 917 grape images (including Summer Black, Moldova, and Youyong) taken by the rear camera of MI8 mobile phone at Jinniu Mountain Base of Shandong Fruit and Vegetable Research Institute were verified experimentally. Results and Discussions] The results showed that the success rate was 90.51% when the error between the table grape picking points and the optimal points were less than 12 pixels, and the average positioning time was 0.87 s. The method realized the fast and accurate localization of table grape picking points. On top of that, according to the two cultivation modes (hedgerow planting and trellis planting) of table grapes, a simulation test platform based on the Dense mechanical arm and the single-chip computer was set up in the study. 50 simulation tests were carried out for the four conditions respectively, among which the success rate of localization for purple grape picking point of hedgerow planting was 86.00%, and the average localization time was 0.89 s; the success rate of localization for purple grape identification and localization of trellis planting was 92.00%, and the average localization time was 0.67 s; the success rate of localization for green grape picking point of hedgerow planting was 78.00%, and the average localization time was 0.72 s; and the success rate of localization for green grape identification and localization of trellis planting was 80.00%, and the average localization time was 0.71 s. [Conclusions] The experimental results showed that the method proposed in the study can meet the requirements of table grape picking, and can provide technical supports for the development of grape picking robot.
[Objective] Aiming at the problems of low accuracy and incomplete coverage of image recognition of phenological period of apple in natural environment by traditional methods, an improved ResNet50 model was proposed for phenological period recognition of apple. [Methods] With 8 kinds of phenological period images of Red Fuji apple in Sichuan plateau area as the research objects and 3 sets of spherical cameras built in apple orchard as acquisition equipment, the original data set of 9800 images of apple phenological period were obtained, labeled by fruit tree experts. Due to the different duration of each phenological period of apple, there were certain differences in the quantity of collection. In order to avoid the problem of decreasing model accuracy due to the quantity imbalance, data set was enhanced by random cropping, random rotation, horizontal flip and brightness adjustment, and the original data set was expanded to 32,000 images. It was divided into training set (25,600 images), verification set (3200 images) and test set (3200 images) in a ratio of 8:1:1. Based on the ResNet50 model, the SE (Squeeze and Excitation Network) channel attention mechanism and Adam optimizer were integrated. SE channel attention was introduced at the end of each residual module in the benchmark model to improve the model's feature extraction ability for plateau apple tree images. In order to achieve fast convergence of the model, the Adam optimizer was combined with the cosine annealing attenuation learning rate, and ImageNet was selected as the pre-training model to realize intelligent recognition of plateau Red Fuji apple phenological period under natural environment. A "Intelligent Monitoring and Production Management Platform for Fruit Tree Growth Period" has been developed using the identification model of apple tree phenology. In order to reduce the probability of model misjudgment, improve the accuracy of model recognition, and ensure precise control of the platform over the apple orchard, three sets of cameras deployed in the apple orchard were set to capture motion trajectories, and images were collected at three time a day: early, middle, and late, a total of 27 images per day were collected. The model calculated the recognition results of 27 images and takes the category with the highest number of recognition as the output result to correct the recognition rate and improve the reliability of the platform. [Results and Discussions] Experiments were carried out on 32,000 apple tree images. The results showed that when the initial learning rate of Adam optimizer was set as 0.0001, the accuracy of the test model tended to the optimal, and the loss value curve converged the fastest. When the initial learning rate was set to 0.0001 and the iteration rounds are set to 30, 50 and 70, the accuracies of the optimal verification set obtained by the model was 0.9354, 0.9635 and 0.9528, respectively. Therefore, the improved ResNet50 model selects the learning rate of 0.0001 and iteration rounds of 50 as the training parameters of the Adam optimizer. Ablation experiments showed that the accuracy of validation set and test set were increased by 0.8% and 2.99% in the ResNet50 model with increased SE attention mechanism, respectively. The validation set accuracy and test set accuracy of the ResNet50 model increased by 2.19% and 1.42%, respectively, when Adam optimizer was added. The accuracy of validation set and test set was 2.33% and 3.65%, respectively. The accuracy of validation set was 96.35%, the accuracy of test set was 91.94%, and the average detection time was 2.19 ms.Compared with the AlexNet, VGG16, ResNet18, ResNet34, and ResNet101 models, the improved ResNet50 model improved the accuracy of the optimal validation set by 9.63%, 5.07%, 5.81%, 4.55%, and 0.96%, respectively. The accuracy of the test set increased by 12.31%, 6.88%, 8.53%, 8.67%, and 5.58%, respectively. The confusion matrix experiment result showed that the overall recognition rate of the improved ResNet50 model for the phenological period of apple tree images was more than 90%, of which the accuracy rate of bud stage and dormancy stage was the lowest, and the probability of mutual misjudgment was high, and the test accuracy rates were 89.50% and 87.44% respectively. There were also a few misjudgments during the young fruit stage, fruit enlargement stage, and fruit coloring stage due to the similarity in characteristics between adjacent stages. The external characteristics of the Red Fuji apple tree were more obvious during the flowering and fruit ripening stages, and the model had the highest recognition rate for the flowering and fruit ripening stages, with test accuracy reaching 97.50% and 97.49%, respectively. [Conclusions] The improved ResNet50 can effectively identify apple phenology, and the research results can provide reference for the identification of orchard phenological period. After integration into the intelligent monitoring production management platform of fruit tree growth period, intelligent management and control of apple orchard can be realized.
[Objective] Conducting research on the recognition of yak meat parts can help avoid confusion and substandard parts during the production and sales of yak meat, improve the transparency and traceability of the yak meat industry, and ensure food safety. To achieve fast and accurate recognition of different parts of yak meat, this study proposed an improved residual network model and developed a smartphone based yak meat part recognition software. [Methods] Firstly, the original data set of 1960 yak tenderloin, high rib, shank and brisket were expanded by 8 different data enhancement methods, including horizontal flip, vertical flip, random direction rotation 30°, random direction rotation 120°, random direction rotation 300°, contrast adjustment, saturation adjustment and hue adjustment. After expansion, 17,640 yak meat images of different parts were obtained. The expanded yak meat images of different parts were divided according to the 4:1 ratio, resulting in 14,112 yak meat sample images in the training set and 3528 yak meat sample images in the test set. Secondly, the convolutional block attention module (CBAM) was integrated into each residual block of the original network model to enhance the extraction of key detail features of yak images in different parts. At the same time, introducing this mechanism into the network model could achieve greater accuracy improvement with less computational overhead and fewer parameters. In addition, in the original network model, the full connection layer was directly added after all residual blocks instead of global average pooling and global maximum pooling, which could improve the accuracy of the network model, prevent overfitting, reduce the number of connections in subsequent network layers, accelerate the execution speed of the network model, and reduce the computing time when the mobile phone recognized images. Thirdly, different learning rates, weight attenuation coefficients and optimizers were used to verify the influence of the improved ResNet18_CBAM network model on convergence speed and accuracy. According to the experiments, the stochastic gradient descent (SGD) algorithm was adopted as the optimizer, and when the learning rate was 0.001 and the weight attenuation coefficient was 0, the improved ReaNet18_CBAM network model had the fastest convergence speed and the highest recognition accuracy on different parts of yak data sets. Finally, the PyTorch Mobile module in PyTorch deep learning framework was used to convert the trained ResNet18_CBAM network model into TorchScript model and saved it in *.ptl. Then, the yak part recognition App was developed using the Android Studio development environment, which included two parts: Front-end interface and back-end processing. The front-end of the App uses *.xml for a variety of price control layout, and the back-end used Java language development. Then TorchScript model in *.ptl was used to identify different parts of yak meat. Results and Discussions] In this study, CBAM, SENet, NAM and SKNet, four popular attentional mechanism modules, were integrated into the original ResNet18 network model and compared by ablation experiments. Their recognition accuracy on different parts of yak meat dataset were 96.31%, 94.12%, 92.51% and 93.85%, respectively. The results showed that among CBAM, SENet, NAM and SKNet, the recognition accuracy of ResNet18 CBAM network model was significantly higher than that of the other three attention mechanism modules. Therefore, the CBAM attention mechanism module was chosen as the improvement module of the original network model. The accuracy of the improved ResNet18_CBAM network model in the test set of 4 different parts of yak tenderloin, high rib, shank and brisket was 96.31%, which was 2.88% higher than the original network model. The recognition accuracy of the improved ResNet18_CBAM network model was compared with AlexNet, VGG11, ResNet34 and ResNet18 network models on different parts of yak test set. The improved ResNet18_CBAM network model had the highest accuracy. In order to verify the actual results of the improved ResNet18_CBAM network model on mobile phones, the test conducted in Xining beef and mutton wholesale market. In the actual scenario testing on the mobile end, a total of 54, 59, 51, and 57 yak tenderloin, high rib, shank and brisket samples were collected, respectively. The number of correctly identified samples and the number of incorrectly identified samples were counted respectively. Finally, the recognition accuracy of tenderloin, high rib, shank and brisket of yak reached 96.30%, 94.92%, 98.04% and 96.49%, respectively. The results showed that the improved ResNet18_CBAM network model could be used in practical applications for identifying different parts of yak meat and has achieved good results. [Conclusions] The research results can help ensure the food quality and safety of the yak industry, improve the quality and safety level of the yak industry, improve the yak trade efficiency, reduce the cost, and provide technical support for the intelligent development of the yak industry in the Qinghai-Tibet Plateau region.
[Objective] Saposhnikovia divaricata (Turcz.) Schischk is a kind of traditional Chinese medicine. Currently, the methods of identifying the origin and quality of Saposhnikovia divaricata (Turcz.) Schischk are mainly based on their physical or chemical characteristics, which is impossible to make an accurate measurement of Groundness identification. With the continuous development of deep learning, its advantages of no manual extraction and high classification accuracy are widely used in different fields, and an attention-embedded ShuffleNet V2-based model was proposed in this study to address the problems of large computation and low accuracy of most convolutional neural network models in the identification of Chinese herbal medicine Saposhnikovia divaricata (Turcz.) Schischk. [Methods] The model architecture was adjusted to reduce the number of model parameters and computation without degrading the network performance, and the traditional residual network was replaced by the Hourglass residual network, while the SE attention mechanism was introduced to embed the hourglass residual network with additional channel attention into ShuffleNet V2. The important features were enhanced and the unimportant features were weakened by controlling the size of the channel ratio to make the extracted features more directional by SE attention. The SiLU activation function was used to replace the ReLU activation function to enhance the generalization ability of the model Enriching local feature learning. Therefore, a lightweight Shuffle-Hourglass SE model was proposed. The samples of Saposhnikovia divaricata (Turcz.) Schischk used in this research were samples from the main production areas, including more than 1000 samples from five production areas in Heilongjiang, Jilin, Hebei, Gansu and Inner Mongolia. A total of 5234 images of Saposhnikovia divaricata (Turcz.) Schischk were obtained by using cell phone photography indoors under white daylight, fully taking into account the geographical distribution differences of different Saposhnikovia divaricata (Turcz.) Schischk. The data set of Saposhnikovia divaricata (Turcz.) Schischk images was expanded to 10,120 by using random flip, random crop, brightness and contrast enhancement processes. In order to verify the effectiveness of the model proposed, four classical network models, VGG16, MobileNet V2, ShuffleNet V2 and SqueezeNet V2, were selected for comparison experiments, ECA ( Efficient Channel Attention ) attention mechanism, CBAM ( Convolutional Block Attention Module ) attention mechanism and CA attention mechanism were chosen to compare with SE. All attention mechanisms were introduced into the same position in the ShuffleNet V2 model, and ReLU, H-swish and ELU activation functions were selected for contrast experiments under the condition in which other parameters unchanged. In order to explore the performance improvement of ShuffleNet V2 model by using the attention mechanism of SE module, Hourglass residual block and activation function, Shuffle-Hourglass SE model ablation experiment was carried out. Finally, loss, accuracy, precision, recall and F1 score in test set and training set were used as evaluation indexes of model performances. [Results and Discussions] The results showed that the Shuffle-Hourglass SE model proposed achieved the best performances. An accuracy of 95.32%, recall of 95.28%, and F1 score of 95.27% were obtained in the test set, which was 2.09%, 2.1 %, and 2.19 % higher than the ShuffleNet V2 model, respectively. The test duration and model size were 246.34 ms and 3.23 M, respectively, which were not only optimal among Traditional CNN such as VGG and Desnet,but had great advantages among lightweight networks such as MobileNet V2、SqueezeNet V2 and ShufffleNet V2. Compared with the classical convolutional network VGG, 7.41% of the accuracy was improved, 71.89% of the test duration was reduced, and 96.76% of the model size was reduced by the Shuffle-Hourglass SE model proposed in this study. Although the test duration of ShuffleNet V2 and MobileNet V2 were similar, the accuracy and speed of the Shuffle-Hourglass SE model improved, which proved its better performance. Compared with MobileNet V2, the test duration was reduced by 69.31 ms, the model size was reduced by 1.98 M, and the accuracy was increased by 10.5 %. In terms of classification accuracy, the improved network maintains higher recognition accuracy and better classification performance. [Conclusions] The model proposed in this research is able to identify the Saposhnikovia divaricata (Turcz.) Schischk originality well while maintaining high identification accuracy and consuming less storage space, which is helpful for realizing real-time identification of Saposhnikovia divaricata (Turcz.) Schischk originality in the future low performance terminals.
[Objective] The fruit load estimation of fruit tree is essential for horticulture management. Traditional estimation method by manual sampling is not only labor-intensive and time-consuming but also prone to errors. Most existing models can not apply to edge computing equipment with limited computing resources because of their high model complexity. This study aims to develop a lightweight model for edge computing equipment to estimate fruit load automatically in the orchard. [Methods] The experimental data were captured using the smartphone in the citrus orchard in Jiangnan district, Nanning city, Guangxi province. In the dataset, 30 videos were randomly selected for model training and other 10 for testing. The general idea of the proposed algorithm was divided into two parts: Detecting fruits and extracting ReID features of fruits in each image from the video, then tracking fruit and estimating the fruit load. Specifically, the CSPDarknet53 network was used as the backbone of the model to achieve feature extraction as it consumes less hardware computing resources, which was suitable for edge computing equipment. The path aggregation feature pyramid network PAFPN was introduced as the neck part for the feature fusion via the jump connection between the low-level and high-level features. The fused features from the PAFPN were fed into two parallel branches. One was the fruit detection branch and another was the identity embedding branch. The fruit detection branch consisted of three prediction heads, each of which performed 3×3 convolution and 1×1 convolution on the feature map output by the PAFPN to predict the fruit's keypoint heat map, local offset and bounding box size, respectively. The identity embedding branch distinguished between different fruit identity features. In the fruit tracking stage, the byte mechanism from the ByteTrack algorithm was introduced to improve the data association of the FairMOT method, enhancing the performance of fruit load estimation in the video. The Byte algorithm considered both high-score and low-score detection boxes to associate the fruit motion trajectory, then matches the identity features' similarity of fruits between frames. The number of fruit IDs whose tracking duration longer than five frames was counted as the amount of citrus fruit in the video. [Results and Discussions] All experiments were conducted on edge computing equipment. The fruit detection experiment was conducted under the same test dataset containing 211 citrus tree images. The experimental results showed that applying CSPDarkNet53+PAFPN structure in the proposed model achieved a precision of 83.6%, recall of 89.2% and F1 score of 86.3%, respectively, which were superior to the same indexes of FairMOT (ResNet34) model, FairMOT (HRNet18) model and Faster RCNN model. The CSPDarkNet53+PAFPN structure adopted in the proposed model could better detect the fruits in the images, laying a foundation for estimating the amount of citrus fruit on trees. The model complexity experimental results showed that the number of parameters, FLOPs (Floating Point Operations) and size of the proposed model were 5.01 M, 36.44 G and 70.2 MB, respectively. The number of parameters for the proposed model was 20.19% of FairMOT (ResNet34) model's and 41.51% of FairMOT (HRNet18) model's. The FLOPs for the proposed model was 78.31% less than FairMOT (ResNet34) model's and 87.63% less than FairMOT (HRNet18) model's. The model size for the proposed model was 23.96% of FairMOT (ResNet34) model's and 45.00% of FairMOT (HRNet18) model's. Compared with the Faster RCNN, the model built in this study showed advantages in the number of parameters, FLOPs and model size. The low complexity proved that the proposed model was more friendly to edge computing equipment. Compared with the lightweight backbone network EfficientNet-Lite, the CSPDarkNet53 applied in the proposed model's backbone performed better fruit detection and model complexity. For fruit load estimation, the improved tracking strategy that integrated the Byte algorithm into the FairMOT positively boosted the estimation accuracy of fruit load. The experimental results on the test videos showed that the AEP (Average Estimating Precision) and FPS (Frames Per Second) of the proposed model reached 91.61% and 14.76 f/s, which indicated that the proposed model could maintain high estimation accuracy while the FPS was 2.4 times and 4.7 times of the comparison models, respectively. The RMSE (Root Mean Square Error) of the proposed model was 4.1713, which was 47.61% less than FairMOT (ResNet34) model's and 22.94% less than FairMOT (HRNet18) model's. The R2 of the determination coefficient between the algorithm-measured value and the manual counted value was 0.9858, which was superior to other comparison models. The proposed model revealed better performance in estimating fruit load and lower model complexity than other comparatives. [Conclusions] The experimental results proved the validity of the proposed model for fruit load estimation on edge computing equipment. This research could provide technical references for the automatic monitoring and analysis of orchard productivity. Future research will continue to enrich the data resources, further improve the model's performance, and explore more efficient methods to serve more fruit tree varieties.
