Abstract:

[Objective] Aiming at the problems of low accuracy and incomplete coverage of image recognition of phenological period of apple in natural environment by traditional methods, an improved ResNet50 model was proposed for phenological period recognition of apple. [Methods] With 8 kinds of phenological period images of Red Fuji apple in Sichuan plateau area as the research objects and 3 sets of spherical cameras built in apple orchard as acquisition equipment, the original data set of 9800 images of apple phenological period were obtained, labeled by fruit tree experts. Due to the different duration of each phenological period of apple, there were certain differences in the quantity of collection. In order to avoid the problem of decreasing model accuracy due to the quantity imbalance, data set was enhanced by random cropping, random rotation, horizontal flip and brightness adjustment, and the original data set was expanded to 32,000 images. It was divided into training set (25,600 images), verification set (3200 images) and test set (3200 images) in a ratio of 8:1:1. Based on the ResNet50 model, the SE (Squeeze and Excitation Network) channel attention mechanism and Adam optimizer were integrated. SE channel attention was introduced at the end of each residual module in the benchmark model to improve the model's feature extraction ability for plateau apple tree images. In order to achieve fast convergence of the model, the Adam optimizer was combined with the cosine annealing attenuation learning rate, and ImageNet was selected as the pre-training model to realize intelligent recognition of plateau Red Fuji apple phenological period under natural environment. A "Intelligent Monitoring and Production Management Platform for Fruit Tree Growth Period" has been developed using the identification model of apple tree phenology. In order to reduce the probability of model misjudgment, improve the accuracy of model recognition, and ensure precise control of the platform over the apple orchard, three sets of cameras deployed in the apple orchard were set to capture motion trajectories, and images were collected at three time a day: early, middle, and late, a total of 27 images per day were collected. The model calculated the recognition results of 27 images and takes the category with the highest number of recognition as the output result to correct the recognition rate and improve the reliability of the platform. [Results and Discussions] Experiments were carried out on 32,000 apple tree images. The results showed that when the initial learning rate of Adam optimizer was set as 0.0001, the accuracy of the test model tended to the optimal, and the loss value curve converged the fastest. When the initial learning rate was set to 0.0001 and the iteration rounds are set to 30, 50 and 70, the accuracies of the optimal verification set obtained by the model was 0.9354, 0.9635 and 0.9528, respectively. Therefore, the improved ResNet50 model selects the learning rate of 0.0001 and iteration rounds of 50 as the training parameters of the Adam optimizer. Ablation experiments showed that the accuracy of validation set and test set were increased by 0.8% and 2.99% in the ResNet50 model with increased SE attention mechanism, respectively. The validation set accuracy and test set accuracy of the ResNet50 model increased by 2.19% and 1.42%, respectively, when Adam optimizer was added. The accuracy of validation set and test set was 2.33% and 3.65%, respectively. The accuracy of validation set was 96.35%, the accuracy of test set was 91.94%, and the average detection time was 2.19 ms.Compared with the AlexNet, VGG16, ResNet18, ResNet34, and ResNet101 models, the improved ResNet50 model improved the accuracy of the optimal validation set by 9.63%, 5.07%, 5.81%, 4.55%, and 0.96%, respectively. The accuracy of the test set increased by 12.31%, 6.88%, 8.53%, 8.67%, and 5.58%, respectively. The confusion matrix experiment result showed that the overall recognition rate of the improved ResNet50 model for the phenological period of apple tree images was more than 90%, of which the accuracy rate of bud stage and dormancy stage was the lowest, and the probability of mutual misjudgment was high, and the test accuracy rates were 89.50% and 87.44% respectively. There were also a few misjudgments during the young fruit stage, fruit enlargement stage, and fruit coloring stage due to the similarity in characteristics between adjacent stages. The external characteristics of the Red Fuji apple tree were more obvious during the flowering and fruit ripening stages, and the model had the highest recognition rate for the flowering and fruit ripening stages, with test accuracy reaching 97.50% and 97.49%, respectively. [Conclusions] The improved ResNet50 can effectively identify apple phenology, and the research results can provide reference for the identification of orchard phenological period. After integration into the intelligent monitoring production management platform of fruit tree growth period, intelligent management and control of apple orchard can be realized.

Key words: apple, residual network, ResNet50, phenological period recognition, smart orchard