基于EfficientNet与FPN多尺度特征融合的田间害虫智能检测(原标题:Enhancing Pest Detection in Agriculture: A Multi-Scale Feature Fusion Approach with YOLOv3)

1. Introduction

Sustainable production of staple cereal (e.g., wheat and maize) and oilseed crops (e.g., soybean and canola) underpins global agricultural stability and catalyzes the advancement of precision and ecologically sustainable farming systems [1,2]. However, pest infestation remains the predominant biotic constraint on crop productivity, accounting for global yield losses ranging from 10% to 28%, with even more severe losses reported in developing countries [3]. Additionally, pests are indirectly responsible for major losses in few plants, such as cankers and shoot blights in nuts, since many insects serve as a vector for pathogens [4]. Beyond significant yield reduction, pest infestation often compels farmers to increase pesticide application, hinders the development of green and sustainable agriculture [5,6]. These adverse effects further disrupt routine field management, reduce crop productivity, and hinder the implementation of precision agriculture, forming a core bottleneck limiting the stability and sustainability of agricultural production systems. With the global transition toward digital and precision agriculture, mitigating pest-induced hazards is essential for advancing agronomic development and optimizing crop cultivation and management [7]. Traditional pest monitoring relies heavily on manual field surveys, which are characterized by low operational efficiency, delayed feedback loops, and limited spatial coverage [8]. In the absence of robust, reliable monitoring systems, large-scale pest outbreaks can emerge readily—compromising agroecosystem integrity and significantly reducing crop productivity [9].

In recent years, advances in deep learning and computer vision have built a new intelligent agricultural risk management paradigm [10]. Vision-based pest detection has become a mainstream research direction, outperforming manual surveys in field practicality [11]. Convolutional neural network (CNN)-based pest identification methods automatically extract hierarchical visual features to enable fast, high-accuracy pest classification [12,13]. The hierarchical features extracted by the VGG network are shown in Figure 1a. Compared with traditional methods, deep learning detection techniques have superior performance in real-time capability, scalability and accuracy, relying on standardized unified frameworks. As illustrated in Figure 1b, mainstream object detectors adopt a four-stage structure: input, backbone, neck and head (Figure 1b). These detectors are broadly categorized into single-stage (dense prediction) and two-stage (sparse prediction) paradigms. Each paradigm further subdivides into anchor-based and anchor-free approaches. Representative dense-prediction models include the anchor-based YOLO and RetinaNet, as well as the anchor-free CenterNet and FCOS. In contrast, sparse-prediction models encompass the anchor-based Faster R-CNN and the anchor-free RepPoints. Among these algorithms, the YOLO series models have attracted considerable attention due to their excellent balance between detection speed and accuracy [14,15].

Figure 1. Schematic diagram of convolutional neural network feature extraction and object detection structure. (a) Schematic diagram of feature extraction in convolutional neural networks; (b) Schematic diagram of the target detection structure.

Although YOLO-based architectures have demonstrated robust performance in agricultural pest monitoring, several critical methodological limitations persist in practical field-scale crop pest detection tasks. First, the original Darknet-53 backbone of YOLOv3 lacks attention modules, failing to capture subtle features of tiny pests in complex field backgrounds [16,17], leading to misclassification of similar pest species and frequent missed detections [18]. Second, the primitive simple upsampling feature fusion module of original YOLOv3 fails to construct complete top-down multi-scale feature transmission, weakening the model’s ability to distinguish small-size pests against cluttered leaf [19,20]. Existing feature fusion frameworks rarely construct bidirectional feature circulation, restricting miniature pest recognition [21,22]. Third, most detection models adopt generic anchor boxes pre-trained on public natural image datasets. The size distribution of these anchor boxes may not match miniature field pests, leading to frequent missed localization of small pest individuals. Dataset-adaptive anchor clustering effectively alleviates this issue [23,24]. Fourth, existing YOLO variants struggle to balance detection precision and computational overhead, forming an unaddressed trade-off dilemma [25]. Deep heavy backbones deliver favorable recognition accuracy but incur excessive inference latency, which cannot be deployed on low-power portable edge devices for field real-time monitoring [26]. While lightweight YOLO-tiny models cut computational cost by simplifying network layers, yet the compressed feature extraction capacity drastically reduces detection accuracy for small and overlapping pests [27]. Furthermore, few datasets have been specially annotated for wheat, corn, soybean and canola pests, restricting the targeted optimization and benchmark evaluation of detection algorithms for dominant field crops [28,29]. To address the above shortcomings, this study proposes an optimized YOLOv3-based intelligent pest detection framework with three targeted optimizations: replacing Darknet-53 with lightweight attention-equipped EfficientNet as the feature extraction backbone, introducing a complete Feature Pyramid Network (FPN) for comprehensive multi-scale feature fusion, and comparing the COCO anchor boxes with the anchor boxes that are adaptively generated from the dataset through K-means clustering, thereby selecting the scale features that are more suitable for field pests. Additionally, a manually annotated multi-class pest dataset covering four major field crops is constructed to validate the performance of the optimized detection model. The proposed model demonstrates enhanced pest detection performance and delivers a computationally efficient, lightweight detection solution, thereby providing technical support for intelligent and precision monitoring of crop pests in agricultural fields.

2. Material and Methods

2.1. Dataset

The experimental image data in this paper is sourced from the Anhui Academy of Agricultural Sciences [30,31], covering 4285 images of 37 types of pests found in several common field crops, including corn, soybean, wheat, and canola. All images were captured by the use of digital cameras (e.g., Canon, Nikon, Tokyo, Japan, and mobile devices). Among them, 3162 images (73.8% of the entire dataset) contained only a single pest individual; the remaining 1123 images (26.2% of the total) contained two or more overlapping pest instances. Furthermore, Following the standard pixel-area-based threshold definition used in YOLO-based detection tasks, we classified annotated pest objects into three size categories and computed their respective proportions: small targets (pixel area < 32 2) constitute 47.2%, medium targets (32 2 ≤ pixel area < 96 2) account for 39.6%, and large targets (pixel area ≥ 96 2) represent only 13.2%. The predominance of small-scale pests further underscores the necessity of our multi-scale feature fusion architecture. To eliminate the potential negative effects of illumination variability, all sample images were preprocessed with uniform illumination settings in crop field situations, as was done by Wen and Guyer (2012) [32]. The various types of pest examples included in the dataset are shown in Figure 2, and the names and quantities of each type of pest are listed in Table 1.

Figure 2. Sample images from the pest dataset, along with their corresponding identifiers, which match those listed in Table 1.

Table 1. Pest infestation dataset: image names and quantities.

2.2. Data Processing

To train the model, it is necessary to label the pests in the dataset first. LabelImg is a tool for marking the position and name of the target in the image in deep learning. It is written in Python 3.12 and uses PyQt as its graphical interface. During the annotation process, the annotations are saved as XML files in the PASCAL VOC format. The images of the labeled pests and the generated XML files after labeling are shown in Figure 3.

Figure 3. Sample images and annotated pest files.

Secondly, the dataset constitutes the foundational input for training deep neural network models, and its quality critically determines the performance and generalizability of the resulting models. Prior to model construction, it is essential to design an input data processing strategy aligned with the chosen network architecture commonly referred to as dataset preprocessing. This preprocessing stage typically comprises two key components: image rescaling and image augmentation.

All pest images in this dataset have an original resolution of 200 × 200 pixels. However, the YOLOv3 network model expects input images of size 416 × 416 pixels by default. Thus, all images are first resized to this dimension. This upsampling operation inevitably introduces interpolation artifacts, which smooth fine high-frequency spatial gradients and attenuate subtle edge-texture features, potentially degrading the model’s capability in feature extraction and precise localization of tiny pest targets. To mitigate the adverse effects of such gradient distortion, we implement a suite of coordinated optimization strategies: (i) the multi-scale feature fusion architecture inherent to YOLO-series networks preserves delicate local gradient information through shallow feature branches, specifically enhancing sensitivity to small objects; (ii) dataset-adaptive for anchor box generation ensures anchor dimensions closely match the actual pixel scales of miniature pests, thereby reducing localization bias arising from blurred target boundaries; and (iii) L2 regularization and gradient accumulation configured via the hyperparameters jointly stabilize gradient updates that would otherwise fluctuate due to resizing-induced artifacts.

A unified baseline augmentation pipeline is implemented identically across all four models to guarantee fair cross-architecture comparison. All augmentation operations are applied on-the-fly during training rather than generating a static offline expanded dataset; the raw dataset retains a fixed total of 4285 original images, with unique random distortions generated dynamically for every training iteration. The shared augmentation suite includes standardized geometric and photometric transformations, with full parameter constraints defined as follows: (1) Isotropic random scaling: Each image and its corresponding bounding boxes are scaled synchronously with a randomly sampled scaling factor ranging from 0.5 to 1.5, which avoids extreme shrinkage or magnification that would obscure pest morphological features; (2) Random translation: Horizontal and vertical translation offsets are restricted within ±15% of the total image width and height to prevent partial cropping of pest bodies. (3) Horizontal flipping and color jittering: Random horizontal mirroring is performed with a 50% probability, while mild brightness, contrast and saturation jitter are applied to enrich photometric diversity without distorting pest texture identities. Meanwhile, to prevent morphological distortion that could lead to inter-species pest misidentification, strict amplitude constraints are imposed on all geometric transformations. Concurrently, shallow feature extraction branches retain fine-grained texture gradients, while COCO anchor boxes are more suitable for the natural size distribution of the target pests in this research dataset. These measures collectively enhancing the model’s ability to discriminate visually similar pest individuals, even under augmentation-induced distortions. Since all YOLO variants are bounding-box–based object detectors, ground-truth bounding box coordinates are transformed in strict synchrony with each augmentation operation, thereby preserving consistent spatial alignment between input images and their corresponding annotation labels.

Beyond the shared baseline preprocessing, YOLOv4 and YOLOv4-tiny incorporate an exclusive additional data augmentation technique named Mosaic augmentation, a refined variant derived from CutMix [33], while Mosaic augmentation is not activated during the training of YOLOv3 and YOLOv3-tiny. The detailed operating procedure of Mosaic augmentation is as follows. Firstly, the Mosaic data augmentation technique combines four different input images (each with its corresponding bounding box) into a single composite image. Following this integration, a new image and its corresponding bounding boxes are generated and subsequently fed into the neural network for training and optimization. Specifically, each of the four constituent images is first independently subjected to a series of geometric and photometric transformations, including horizontal flipping, isotropic scaling, and color-space jittering. The transformed images are then spatially arranged in the top-left, bottom-left, bottom-right, and top-right quadrants of the output canvas, respectively. Finally, fixed subregions from each quadrant are extracted via matrix indexing and seamlessly stitched together to form the final mosaic image, whose merged bounding boxes are fed into the network for training and optimization.

2.3. The Framework Structure of the YOLO Model

YOLOv3 is developed based on YOLOv1 and YOLOv2. As shown in Figure 4, it is a fully convolutional network with a total of 252 layers, using Darknet53 as the backbone feature extraction network [34]. To enhance the detection ability for small objects, the network adopts the methods of feature map upsampling and multi-scale feature fusion, conducting detection on feature maps of multiple scales. The input is the image of pests. The backbone feature extraction network adopts the 52 convolutional layers of Darknet53 after removing the last fully connected layer, which consists of 5 groups of ResNet residual blocks. Each group of residual blocks is repeated 1, 2, 8, 8, and 4 times, respectively. During each repetition process, a 1 × 1 convolution is executed first, followed by a 3 × 3 convolution, and finally the filter is halved and then restored. YOLO takes the entire insect damage image as the input of the network. After the backbone network extracts the feature map of a certain size (S × S), the entire insect damage image is divided into S × S grids. Each grid predicts three bounding boxes. If the center of a certain insect damage target falls within a certain grid, then this grid is responsible for predicting this insect damage target. The output layer outputs the position of the bounding box and the category to which the insect damage target belongs. Each bounding box predicts four positions (x, y, w, h) and one confidence, totaling five values. Each grid also predicts one category information, with a total of C categories (there are 37 types of insect damage in the dataset of this article). For one anchor box, the total number of predicted channels equals 5 + 37 = 42; since each grid predicts three boxes, the total channel count per grid is 3 × 42 = 126, and ultimately the output tensor of each detection branch takes the shape of S × S × 126, where S × S stands for the spatial grid partition of the feature map, channels 1–4 correspond to the bounding box regression parameters (x, y, w, h) of the first anchor box, channel 5 represents the objectness confidence score of the first anchor box, channels 6–42 denote the class probability distribution of the first anchor box, and this identical channel layout is repeated sequentially for the second and third anchor boxes within every grid cell.

Figure 4. Schematic diagram of the YOLOv3 structure.

Figure 4. Schematic diagram of the YOLOv3 structure.

YOLOv3-tiny is a compressed version of the YOLOv3 network, which has removed the residual layers and only uses two YOLO output layers of different scales. The network has a total of 23 layers, including 13 convolutional layers, 6 pooling layers, 1 routing layer, 1 upsampling layer, and 2 output layers. Except for the convolutional layer before the YOLO output layer, which uses a linear activation function, each convolutional layer is followed by a BN layer, and each convolutional layer is followed by a convolutional layer with a leaky rectified linear unit (Leaky ReLU) activation function [35]. The output dimensions of the two output layers are