[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"topic-Sensors":3},{"name":4,"kind":5,"tokens":6,"total":7,"page":8,"page_size":9,"items":10},"Sensors","tag",[4],2,1,100,[11,84],{"id":12,"title":13,"url":14,"summary":15,"summary_zh":16,"content":17,"source_name":18,"source_url":16,"published_at":19,"category":20,"cover_url":16,"hotness":21,"is_selected":22,"score":23,"score_detail":24,"sources":30,"tags":32,"primary_topic":36,"secondary_topics":42,"classification":54,"search_phrases":78,"slug":81,"view_count":82,"doi":16,"paper":16,"created_at":83},379,"TMAF-YOLO:温室番茄成熟度与缺陷果实轻量化实时检测模型","https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950","中国农业大学黄晨晓、何林然、黄文涛、张晓绚等于Sensors发表TMAF-YOLO模型——基于YOLOv8n的轻量化番茄成熟度与缺陷果实检测模型,隶属中国农业大学120周年校庆专刊。该模型以LGhostConv替换选定卷积结构减少冗余计算;引入番茄成熟度感知聚合融合模块TMAF增强检测分支的颜色、纹理和局部结构特征表示能力;使用MA-CB Focal Loss改善类别不平衡和难样本条件下的学习。实验结果显示,Precision、Recall、mAP50和mAP50-95分别达到0.889、0.873、0.954和0.776,模型仅含2.647M参数,推理速度209.030 FPS,优于YOLOv11n和YOLOv12n,同时保持实时推理性能。",null,"## 1. Introduction\n\nTomato is an important vegetable crop in protected agriculture. Its maturity directly affects harvesting time, transport management, postharvest ripening, quality grading, and commercial value. In greenhouse environments, tomato fruits are often affected by leaf occlusion, fruit overlap, uneven illumination, and scale variation. Manual maturity assessment is inefficient and easily influenced by subjective experience. Therefore, a rapid, stable, and non-contact detection method is needed to support automated harvesting, grading management, and robotic visual perception in protected agriculture [[1](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B1-sensors-26-04950),[2](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B2-sensors-26-04950),[3](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B3-sensors-26-04950)].\n\nAmong deep learning detection methods, region-based convolutional neural network (R-CNN) models and their variants were among the earlier methods applied to fruit and vegetable detection and counting tasks. An improved Faster R-CNN model, MatDet, was proposed for multi-object tomato maturity detection in complex scenarios [[1](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B1-sensors-26-04950)]. The model used a 50-layer residual network (ResNet-50), region-of-interest alignment (RoIAlign), and a path aggregation network (PANet) to improve detection under occlusion, overlap, and illumination variation, achieving a mean average precision (mAP) of 96.14%. Faster R-CNN has also been used for tomato-related detection tasks. Highly occluded immature tomatoes on plants were detected using Faster R-CNN, with an average precision of 87.83% [[4](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B4-sensors-26-04950)]. Greenhouse tomato flowers were detected and counted using Faster R-CNN, achieving an average precision of 96.02% [[5](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B5-sensors-26-04950)]. These methods have strong localization ability and can provide stable target region information. However, their model structures are relatively complex, and their inference speed and lightweight deployment ability remain limited.\n\nWith the development of Transformer architectures, end-to-end detectors such as Detection Transformer (DETR), DETR with Improved deNoising anchOr boxes (DINO), and Real-Time Detection Transformer (RT-DETR) have been introduced into object detection tasks. DETR formulates object detection as a set prediction problem [[6](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B6-sensors-26-04950)]. DINO improves the convergence and detection performance of DETR-like models through denoising training and query optimization [[7](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B7-sensors-26-04950)]. RT-DETR further improves the Transformer detection structure for real-time detection requirements [[8](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B8-sensors-26-04950)]. In fruit and vegetable maturity recognition, PDSI-RTDETR was proposed for tomato maturity detection. Compared with the original RT-DETR, its mAP50 and mAP50-95 increased by 3.9 and 4.1 percentage points, respectively [[9](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B9-sensors-26-04950)]. An improved RT-DETR was also applied to fruit ripeness detection, indicating the potential of Transformer detectors in global feature modeling [[10](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B10-sensors-26-04950)]. However, these models usually have higher training costs, larger parameter sizes, and greater hardware requirements.\n\nIn contrast, You Only Look Once (YOLO) series models provide a better balance between detection accuracy and inference speed. They have become common methods for fruit and vegetable detection and maturity recognition. A lightweight improved YOLOv5 model was proposed for real-time tomato detection and mobile deployment [[2](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B2-sensors-26-04950)]. Compared with the original YOLOv5s, the number of parameters and floating-point operations (FLOPs) were reduced by 78% and 84.15%, respectively, while maintaining high detection accuracy. RGB-D information was combined with YOLOv5 to detect and count greenhouse tomato clusters [[3](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B3-sensors-26-04950)]. MTD-YOLO was proposed for multitask detection of cherry tomato fruits, clusters, and maturity [[11](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B11-sensors-26-04950)]. The model achieved an overall detection score of 86.6% and an average inference time of 4.9 ms. An improved YOLOX model was proposed for tomato ripeness and stem recognition, achieving an mAP of 92.17% and improving recognition under sample imbalance and scale variation [[12](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B12-sensors-26-04950)]. YOLOv8 was combined with a lightweight Swin Transformer for strawberry ripeness detection [[13](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B13-sensors-26-04950)]. Compared with YOLOv5s, CenterNet, and the Single Shot MultiBox Detector (SSD), the mAP50 increased by 1.6%, 33.5%, and 3.4%, respectively. These results indicate that improved YOLO models are suitable for fine-grained fruit maturity recognition. In addition, recent YOLO-based studies on tomato detection and ripening recognition have improved detection performance using attention mechanisms, lightweight structures, multiscale feature fusion, and postprocessing optimization [[14](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B14-sensors-26-04950),[15](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B15-sensors-26-04950),[16](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B16-sensors-26-04950),[17](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B17-sensors-26-04950)].\n\nExisting studies have verified the effectiveness of deep learning methods in fruit and vegetable detection and maturity recognition. However, further improvement is still needed for in situ greenhouse tomato detection. First, the color transition between semi-mature and mature tomatoes is continuous, making fine-grained classification difficult. Second, leaf occlusion, fruit overlap, and illumination variation weaken maturity-related feature representation. Third, defective samples are relatively limited, and conventional classification losses may underemphasize minority and hard samples. Previous studies have shown that Focal Loss and Class-Balanced Loss can adjust the loss weights of hard and imbalanced samples, thereby improving the learning ability for long-tailed classes and difficult samples [[18](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B18-sensors-26-04950),[19](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B19-sensors-26-04950)]. To address these issues, this study proposes TMAF-YOLO based on YOLOv8n.\n\nThe main contributions of this study are as follows.\n\n(1)\nA lightweight tomato maturity detection model was constructed for complex greenhouse scenarios. Inspired by the low-cost feature generation strategy in GhostNet, an LGhostConv module was designed and embedded in key positions of the Backbone and Neck [[20](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#B20-sensors-26-04950)]. This design reduces the number of parameters and computational cost while maintaining detection performance.\n\n(2)\nA tomato maturity-aware aggregation fusion module, termed TMAF, was proposed. This module was designed to address continuous color transitions between adjacent maturity stages, weak local texture differences, and insufficient feature representation caused by occlusion and overlap. By integrating local texture enhancement and channel-spatial recalibration, TMAF strengthens maturity-related feature representation.\n\n(3)\nMA-CB Focal Loss was developed to improve learning under class imbalance and hard-sample conditions. By combining class-balanced weights with the Focal Loss modulation term, the loss enhances learning for minority and difficult samples, including defective fruits and adjacent maturity stages.\n\n## 2. Materials and Methods\n\n### 2.1. Overall Method Design\n\nThis study focuses on in situ tomato maturity detection before greenhouse harvesting. An overall application workflow was constructed, including greenhouse image acquisition, fruit detection and category recognition, harvesting decision-making, and postharvest grading management. As shown in [Figure 1](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#fig_body_display_sensors-26-04950-f001), tomato images were first acquired under natural growth conditions in the greenhouse. Then, the developed detection model was used to identify fruit locations, maturity stages, and defective fruits. Finally, the detection results were used to support harvesting time determination, defective fruit screening, and postharvest grading and quality management. It should be noted that this study mainly focuses on greenhouse tomato maturity and defective fruit detection. The harvesting, grading, and management processes in the figure are used to illustrate the potential application scenarios of the proposed model.\n\n**Figure 1.** Overall workflow of greenhouse tomato maturity detection.\n\n### 2.2. Dataset Construction and Annotation\n\nThe self-collected images used in this study were acquired in a tomato greenhouse at the Tianjin Academy of Agricultural Sciences, China, from 10 to 17 January 2026. Image acquisition was conducted between 9:00 a.m. and 12:00 p.m. using a HUAWEI Mate 60 smartphone (Huawei Device Co., Ltd., Dongguan, China) in automatic shooting mode. Exposure, focus, ISO sensitivity, and shutter speed were adjusted automatically by the built-in camera system. The shooting distance ranged from approximately 0.5 to 2.0 m, and the image resolution was 1280 × 760 pixels. Images were captured under naturally varying greenhouse illumination; illuminance was not measured instrumentally.\n\nTo represent practical greenhouse conditions, images were acquired from different viewing angles and distances. The collected scenes included fruits at different maturity stages and scales, together with leaf and branch occlusion, fruit overlap, dense fruit distribution, small targets, complex backgrounds, and local image blur. Representative examples of self-collected images are shown in [Figure 2](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#fig_body_display_sensors-26-04950-f002).\n\n**Figure 2.** Representative tomato images acquired under typical greenhouse conditions: (**a**) mature fruits; (**b**) adjacent fruits at different maturity stages; (**c**) fruit overlap and foliage interference; (**d**) leaf and branch occlusion; (**e**) small targets in a relatively long-range view; and (**f**) local image blur.\n\nA total of 596 valid images were collected from the greenhouse. Because defective fruits were relatively scarce, 33 additional defective-fruit images were selected from the Tomato Fruit Diseases Dataset available on Kaggle ([https:\u002F\u002Fwww.kaggle.com\u002Fdatasets\u002Fprofnourasemary\u002Ftomato-dataset](https:\u002F\u002Fwww.kaggle.com\u002Fdatasets\u002Fprofnourasemary\u002Ftomato-dataset), accessed on 1 July 2026). The additional images mainly consisted of tomato fruits exhibiting visible disease symptoms, together with a smaller number showing physical damage, decay, cracking, or other abnormal surface characteristics. The selected public images were annotated according to the same criteria as the self-collected images.\n\nThe 596 self-collected images were divided at the original-image level into training, validation, and test subsets containing 470, 63, and 63 images, respectively. The 33 public images were added exclusively to the training set to increase the number and diversity of defective-fruit samples. Consequently, the final training, validation, and test sets contained 503, 63, and 63 images, respectively. The validation and test sets therefore consisted entirely of self-collected greenhouse images.\n\nThe annotated tomato instances were divided into four categories: immature, semi-mature, mature, and defective. The first three categories represent successive normal maturity stages based primarily on fruit color and visible appearance. The defective category includes fruits with visible disease spots, physical damage, decay, cracking, or other abnormal surface characteristics and was treated as an independent appearance category rather than as a normal maturity stage.\n\nAll tomato instances were manually annotated with bounding boxes using LabelMe v6.0.0. The original annotations were stored in JSON format and subsequently converted into the format required by the YOLO detection framework. Partially occluded or locally blurred fruits were retained when their categories and principal visible boundaries could be reliably determined. Instances were excluded only when severe occlusion, insufficient visible information, or ambiguous appearance prevented consistent annotation. All annotations were reviewed according to uniform class definitions to ensure labeling consistency.\n\nThe final dataset contained 629 images and 2505 annotated tomato instances, including 1170 immature, 788 semi-mature, 304 mature, and 243 defective instances. Data augmentation was performed after dataset partitioning and was applied only to the training set. The validation and test sets were not augmented. The training set was used for model parameter optimization, the validation set for training monitoring and model selection, and the test set exclusively for final performance evaluation. The image and instance distributions before data augmentation are summarized in [Table 1](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#table_body_display_sensors-26-04950-t001).\n\n**Table 1.** Distribution of images and annotated tomato instances before data augmentation.\n\n### 2.3. Construction of the TMAF-YOLO Model\n\n#### 2.3.1. TMAF-YOLO: An Improved YOLOv8n Model\n\nThe proposed TMAF-YOLO uses YOLOv8n as the baseline model. It is improved from three aspects: lightweight feature extraction, maturity-related feature fusion, and classification loss optimization. The overall architecture is shown in [Figure 3](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#fig_body_display_sensors-26-04950-f003). After the original image is input into the network, multiscale features are first extracted by the Backbone, in which selected convolutional structures are replaced by LGhostConv. Then, features at different scales are fused in the Neck, and TMAF modules are added to the P3\u002F8 and P4\u002F16 branches. Finally, the model outputs prediction results through three detection heads at P3\u002F8, P4\u002F16, and P5\u002F32 scales. During training, the model is optimized using a joint loss function composed of WIoU, DFL, and MA-CB Focal Loss. The specific designs of LGhostConv, TMAF, and MA-CB Focal Loss are introduced in the following sections.\n\n**Figure 3.** Overall architecture of TMAF-YOLO.\n\n#### 2.3.2. Lightweight Feature Extraction Module: LGhostConv\n\nTo reduce redundant computation during feature extraction and multiscale feature fusion, a lightweight Ghost convolution module, termed LGhostConv, was introduced into selected downsampling layers of the backbone and neck. Similar to GhostConv, LGhostConv generates output features through primary and complementary feature branches. However, it replaces the dense spatial convolution in the primary branch with a depthwise-separable operation and employs directional depthwise convolutions in the complementary branch.\n\nGiven an input feature map $X$ with $C_{i n}$ channels, LGhostConv first applies a $k \\times k$ depthwise convolution with stride s for spatial feature extraction and downsampling, followed by a $1 \\times 1$ pointwise convolution for channel interaction. This process generates approximately half of the required output channels as the primary features. The remaining channels are generated by dividing the primary features into two groups and applying $1 \\times 3$ and $3 \\times 1$ depthwise convolutions to capture horizontal and vertical information, respectively. The directional features are then fused with the primary features through a parameter-free residual connection and concatenated to form the final output.\n\nIgnoring bias terms, batch-normalization parameters, and activation operations, the number of parameters in LGhostConv is expressed as follows:\n\n$$\nP_{L G h o s t C o n v} = k^{2} C_{i n} + C_{i n} m + 3 n\n$$\n\n(1)\n\nwhere $m$ = ⌈$C_{o u t}$\u002F2⌉ and $n$ = $C_{o u t}$ − $m$ denote the numbers of primary and complementary feature channels, respectively. In comparison, a standard $k \\times k$ convolution requires $k^{2} C_{i n} C_{o u t}$ parameters. Therefore, LGhostConv reduces model complexity by replacing dense spatial channel interactions with depthwise spatial filtering, pointwise channel projection, and low-cost directional feature generation. Its effects on detection accuracy, model size, computational complexity, and inference speed are evaluated in [Section 3.3](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#sec3dot3-sensors-26-04950).\n\n#### 2.3.3. Tomato Maturity-Aware Aggregation Fusion Module: TMAF\n\nTo strengthen the representation of maturity- and defect-related cues in the fused neck features, a tomato maturity-aware aggregation fusion module, termed TMAF, was designed. TMAF is embedded in the P3 and P4 feature fusion paths of the YOLOv8n neck, where relatively high-resolution features are retained for fine-grained maturity and defect detection. The original P5 path is preserved to limit the additional computational cost. As illustrated in [Figure 4](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4950#fig_body_display_sensors-26-04950-f004), TMAF comprises a lightweight texture aggregation branch and a TMCAttention submodule.\n\n**Figure 4.** Structure of the TMAF module.\n\nLet the input feature be $X$, with a size of $X \\in R^{B \\times C \\times H \\times W}$, where $B$, $C$, $H$, and $W$ denote the batch size, number of channels, feature map height, and feature map width, respectively. The input feature first passes through a $1 \\times 1$ convolution for channel mapping. It is then split along the channel dimension into a retained branch and a texture branch:\n\n$$\nX_{1} , X_{2} = S p l i t \\left(\\right. C o n v_{1 \\times 1} \\left(\\right. X \\left.\\right) \\left.\\right)\n$$\n\n(2)\n\nwhere $X_{1}$ is the retained branch, which maintains the direct transmission of input information. $X_{2}$ is the texture branch, which passes through three LTMBlocks in sequence to obtain texture-enhanced features at different levels, denoted as $T_{1}$, $T_{2}$, and $T_{3}$. Each LTMBlock consists of a $1 \\times 1$ convolution, a $3 \\times 3$ depthwise convolution, a $5 \\times 5$ depthwise convolution, and a $1 \\times 1$ convolution. The depthwise convolutions are used to supplement local texture and edge details. The $1 \\times 1$ convolutions are used for channel adjustment and feature fusion.\n\nThen, the retained branch $X_{1}$, and the texture branch outputs $T_{1}$, $T_{2}$, and $T_{3}$ are concatenated along the channel dimension. A $1 \\times 1$ convolution is then used to complete feature fusion and obtain the fused feature $F$:\n\n$$\n= C o n v_{1 \\times 1} \\left(\\right. C o n c a t \\left(\\right. X_{1} , T_{1} , T_{2} , T_{3} \\left.\\right) \\left.\\right)\n$$\n\n(3)\n\nTo further adjust maturity-related feature responses, TMCAttention is applied after feature fusion. This structure contains three branches: channel attention, spatial attention, and residual connection. Channel attention is used to adjust the importance of different feature channels. It helps the model focus on color, texture, and defect responses related to maturity discrimination. Spatial attention is used to highlight fruit region responses and reduce interference from irrelevant background features. The outputs of the two attention branches are denoted as $F_{c}$ and $F_{s}$, respectively.\n\nFinally, the outputs of the channel attention branch and the spatial attention branch are fused and added to the residual branch. The output feature of the TMAF module is obtained as follows:\n\n$$\nY = F + C o n v_{1 \\times 1} \\left(\\right. F_{c} + F_{s} \\left.\\righ","Sensors 2026-08-05","2026-08-05T00:00:00Z","论文",10,false,62,{"impact":25,"substance":26,"depth":27,"authority":25,"freshness":28,"relevant":8,"comment":29},12,18,16,4,"针对温室番茄成熟度与缺陷检测的轻量化模型研究，方法有创新，但影响范围限于细分领域，时效性一般。",[31],{"name":18,"url":14},[33,4,34,35],"中国农业大学","YOLOv8","番茄成熟度",{"code":37,"slug":38,"name":39,"path":40,"parent_code":41},"B1","vegetables","蔬菜","\u002Ftopic\u002Fhorticulture\u002Fvegetables","B",[43,48],{"code":44,"slug":45,"name":46,"path":47,"parent_code":41},"B7","protected-horticulture","设施园艺与植物工厂","\u002Ftopic\u002Fhorticulture\u002Fprotected-horticulture",{"code":49,"slug":50,"name":51,"path":52,"parent_code":53},"I2","agricultural-ai","农业AI与大模型","\u002Ftopic\u002Fdigital-agriculture\u002Fagricultural-ai","I",{"status":55,"locked":22,"version":56,"method":57,"evidence":58},"classified","2026-10-22.2","rules",{"candidates":59,"agricultural_evidence":61},[60,66,72],{"code":49,"score":25,"supported":61,"title":62,"abstract":64,"tags":65},true,[63],"YOLO",[63],[34],{"code":37,"score":67,"supported":61,"title":68,"abstract":70,"tags":71},11,[69],"番茄",[69],[],{"code":44,"score":73,"supported":61,"title":74,"abstract":76,"tags":77},8,[75],"温室",[],[],[79,80],"中国农业大学 番茄成熟度 Sensors YOLOv8","中国农业大学 番茄成熟度","中国农业大学番茄成熟度SensorsYOLOv8-379",3,"2026-08-10T23:54:46.339395Z",{"id":85,"title":86,"url":87,"summary":88,"summary_zh":16,"content":89,"source_name":90,"source_url":16,"published_at":91,"category":20,"cover_url":16,"hotness":21,"is_selected":22,"score":92,"score_detail":93,"sources":97,"tags":99,"primary_topic":103,"secondary_topics":109,"classification":110,"search_phrases":127,"slug":130,"view_count":82,"doi":16,"paper":16,"created_at":131},373,"基于多模态时空对齐与因果利润建模的农业投入优化(MAION)","https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994","中国农业大学付正捷、张誉恒等于Sensors发表针对智慧农业投入优化的多模态网络MAION。该框架构建统一的'地块-时间窗'农业状态表示,通过跨模态时空对齐模块解决田间图像、气象环境、土壤状态和田间管理记录之间采样频率不一致、语义尺度异构问题;通过投入-产出因果利润建模模块评估不同投入行为对产量、成本和净利润的潜在影响;并以利润驱动的强化学习生成面向长期净收益最大化的投入策略。在产量、成本和净利润预测上RMSE分别为0.587、0.531和0.648,R2达0.914\u002F0.891\u002F0.883;经济决策中净收益改善17.68%,投入产出比1.71,显著优于随机森林、XGBoost、LSTM、GRU和Transformer等基线。","## 1. Introduction\n\nWith the continuous advancement of smart agriculture, agricultural production is shifting from experience-dependent judgment toward integrated management supported by data, models, and intelligent decision-making [[1](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B1-sensors-26-04994)]. Internet of Things sensors, unmanned aerial vehicle remote sensing, field image acquisition devices, meteorological monitoring systems, and agricultural management platforms have been increasingly deployed in farmland and orchard production scenarios, enabling continuous recording of soil moisture, temperature and humidity, light intensity, rainfall, crop growth status, pest and disease conditions, irrigation and fertilization records, and pesticide application information [[2](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B2-sensors-26-04994)]. These data provide a fine-grained perception basis for precision agriculture and allow crop growth processes and farmland environmental changes to be understood more comprehensively [[3](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B3-sensors-26-04994)]. However, data availability does not necessarily mean that collected data can be directly transformed into effective decisions [[4](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B4-sensors-26-04994)]. In practical production, managers are usually concerned not only with the accurate prediction of a single indicator, but also with how fertilization, irrigation, and pesticide application should be arranged under current crop status, meteorological conditions, and soil background so that input costs can be controlled while yield and quality are maintained [[5](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B5-sensors-26-04994)]. With increasing production costs, tightening resource constraints, and intensifying climate variability, simply pursuing yield maximization is no longer sufficient for modern agricultural management. A more reasonable balance among yield, cost, resource utilization, and net profit has therefore become an important issue in smart agricultural decision-making research [[6](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B6-sensors-26-04994)].\n\nTraditional agricultural management usually relies on farmer experience, expert rules, and statistical analysis models [[7](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B7-sensors-26-04994)]. Experience-driven methods are widely used because of their direct operation and low implementation cost, but their effectiveness depends heavily on personal experience and shows limited adaptability to climate variation, plot heterogeneity, and crop growth-stage changes [[8](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B8-sensors-26-04994)]. Expert systems can provide recommendations for irrigation, fertilization, and pest or disease control according to predefined rules; however, their rule structures are usually fixed and may lack flexibility when complex coupling relationships exist among environmental variables, crop states, and input behaviors. Statistical models and production function models can analyze relationships between agricultural inputs and outputs, but they often rely on linear or static assumptions and are insufficient for describing nonlinear responses, time-lag effects, and stage-dependent differences in agricultural systems [[9](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B9-sensors-26-04994)]. Moreover, many traditional methods still focus on a single production indicator, such as yield, disease incidence, or soil moisture, while the joint relationships among crop status, environmental conditions, input behavior, and economic return are rarely considered. Therefore, traditional methods remain inadequate for multi-source data fusion, dynamic input optimization, and profit-oriented intelligent decision-making [[10](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B10-sensors-26-04994)].\n\nDeep learning has provided a new technical pathway for smart agriculture and has been widely applied to crop recognition, pest and disease detection, yield prediction, quality assessment, and environmental monitoring [[11](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B11-sensors-26-04994)]. Convolutional neural networks can extract crop growth, canopy structure, and disease features from field images and unmanned aerial vehicle remote sensing images, while recurrent neural networks, long short-term memory networks, and Transformer models can process meteorological, soil, and crop growth time-series data. Multimodal learning methods have further attempted to integrate images, sensor data, and management records to improve agricultural state perception and yield prediction [[12](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B12-sensors-26-04994)]. These studies have promoted the transition from manual observation to automated analysis, but limitations remain in input optimization. Several recent studies have explored causal modeling, reinforcement learning, and multimodal learning for agricultural or related complex decision-making tasks. Zhang et al. [[13](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B13-sensors-26-04994)] proposed the CMRL causal meta-reinforcement learning framework, integrating reinforcement learning feedback, causal intervention, and meta-learning optimization for multimodal remote sensing classification, with overall accuracies of\n\n$86.27 \\%$\n,\n\n$95.51 \\%$\n, and\n\n$99.41 \\%$\non the MUUFL, Houston, and Trento datasets, respectively. Liu et al. [[14](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B14-sensors-26-04994)] proposed the ASGO-PAPIS framework, which combined deep learning and causal inference for multidimensional dairy production cost analysis and improved Precision, NDCG, and MAP by up to approximately 5 percentage points. Zhang et al. [[15](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B15-sensors-26-04994)] developed a cross-modal attention and PPO reinforcement learning framework for pesticide application optimization by fusing unmanned aerial vehicle hyperspectral data, soil meteorological data, and pest monitoring data, achieving an RMSE of\n\n$0.162$\n, a spraying accuracy of\n\n$88.3 \\%$\n, an\n\n$18.3 \\%$\nreduction in pesticide usage, and a pest suppression effect of\n\n$77.9 \\%$\n.\n\nAlthough multimodal learning, causal inference, and reinforcement learning have each demonstrated considerable potential for intelligent agricultural management, they address different stages of the decision-making process and therefore exhibit complementary strengths as well as inherent limitations. Multimodal fusion effectively integrates heterogeneous information from crop images, environmental observations, soil conditions, and management records, enabling comprehensive agricultural state perception [[16](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B16-sensors-26-04994)]. However, it primarily models statistical correlations and cannot reliably determine how changes in irrigation, fertilization, or pesticide application causally affect agricultural productivity and economic returns. Causal inference alleviates confounding effects caused by weather variability, soil heterogeneity, and management differences, providing more reliable estimates of intervention effects. Nevertheless, existing causal approaches mainly focus on effect estimation rather than generating optimal sequential management strategies throughout an entire growing season [[17](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B17-sensors-26-04994)]. Reinforcement learning is naturally suitable for optimizing long-term agricultural decisions, yet its performance strongly depends on accurate state representations and reliable reward signals. Inaccurate perception or biased intervention estimation may consequently result in unstable or economically suboptimal policies. Therefore, these three paradigms should be regarded as complementary rather than competing approaches. A unified framework integrating multimodal perception, causal reasoning, and reinforcement learning can simultaneously achieve comprehensive agricultural state representation, reliable intervention effect estimation, and long-term profit-oriented decision optimization.\n\nAccordingly, a multimodal causal reinforcement learning framework for agricultural input–profit optimization, namely the Multimodal Agricultural Input–Output Optimization Network, abbreviated as MAION, is proposed in this study. Visual data, environmental data, soil data, and agricultural management records are used as inputs, while yield, cost, net profit, and optimal input strategies are taken as outputs. An end-to-end modeling method from multi-source agricultural perception to economically optimal decision-making is thereby constructed. Unlike models that focus only on yield prediction or state recognition, MAION incorporates agricultural input behavior, production cost, and economic return into a unified learning objective. Through cross-modal spatiotemporal alignment, causal input–profit modeling, and profit-driven reinforcement learning decision-making, agricultural intelligent perception is connected with practical production decisions, thereby providing an executable modeling strategy for precision input, cost control, and net profit improvement. The main contributions of this study are summarized as follows:\n\n*   To address the lack of unified agricultural state representation caused by heterogeneous data sources with inconsistent temporal and semantic characteristics, a unified multimodal agricultural input–output modeling framework is developed to support profit-oriented decision making.\n\n*   To address the spatiotemporal inconsistency among crop images, environmental observations, soil measurements, and management records, a cross-modal spatiotemporal alignment strategy is introduced to establish consistent multimodal agricultural representations.\n\n*   To address the inability of existing predictive models to distinguish causal intervention effects from statistical correlations, a causal-aware input–profit modeling mechanism is developed to provide more reliable estimation of the economic effects of agricultural management actions.\n\n*   To address the limitation of conventional optimization methods in achieving long-term economic management, a profit-driven reinforcement learning strategy is designed to optimize sequential agricultural input decisions under long-term profit objectives.\n\n*   To address the challenge of robust deployment in practical agricultural environments, comprehensive cross-plot, cross-year, and missing-modality experiments are conducted to validate the generalization capability and practical effectiveness of the proposed framework.\n\n## 2. Related Work\n\n### 2.1. Multimodal Perception Methods in Smart Agriculture\n\nMultimodal perception in smart agriculture refers to the continuous observation and integrated representation of agricultural production systems using heterogeneous data sources, including images, sensors, remote sensing, and management records [[18](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B18-sensors-26-04994)]. Although these data differ in acquisition frequency, spatial coverage, and semantic level, they jointly provide important information for agricultural state perception [[19](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B19-sensors-26-04994)]. Therefore, multimodal perception is valuable not only because more data types are introduced, but also because complementary information from different sources can be used to identify crop growth status, environmental changes, and production risks more reliably [[20](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B20-sensors-26-04994)]. Image-based agricultural perception methods have been widely studied [[21](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B21-sensors-26-04994)]. Convolutional neural networks can automatically extract texture, color, edge, and morphological features from crop images, and have been applied to disease recognition, fruit detection, weed identification, and quality grading [[22](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B22-sensors-26-04994)]. Vision Transformers further improve the representation of long-range regional dependencies and complex background information, thereby enhancing adaptability under occlusion, illumination variation, and scale differences in field environments [[23](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B23-sensors-26-04994)]. Sensor-based time-series data are also important for greenhouse control, soil moisture monitoring, and crop growth trend analysis, where continuous environmental variables need to be modeled dynamically [[24](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B24-sensors-26-04994)]. Long short term memory (LSTM), gated recurrent unit (GRU), and Transformer models have been adopted to capture long-term dependencies among meteorological variables, soil variables, and growth indicators [[25](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B25-sensors-26-04994)]. Remote sensing methods are more commonly used for regional growth estimation, drought monitoring, and yield assessment through spectral features, vegetation indices, and spatial textures [[26](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B26-sensors-26-04994)]. Despite these advances, most existing studies remain centered on a single modality or a single task [[27](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B27-sensors-26-04994)]. Image models usually focus on disease categories, fruit numbers, or phenotypic traits; sensor models are mainly used for environmental trend prediction or soil state estimation; and remote sensing models emphasize regional crop growth variation [[28](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B28-sensors-26-04994)]. Such fragmented strategies are effective for specific tasks, but they are insufficient for describing the coupled relationships among crops, environments, soils, and management behaviors [[29](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B29-sensors-26-04994)]. For agricultural input–profit optimization, crop growth status or environmental risk alone is not enough; it is also necessary to understand how these states influence fertilization, irrigation, and pesticide application, and how such behaviors are ultimately reflected in yield, cost, and net profit [[30](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B30-sensors-26-04994)]. Accordingly, limitations remain in cross-modal spatiotemporal alignment, unified state representation, and feature fusion oriented toward economic decision-making [[31](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B31-sensors-26-04994),[32](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B32-sensors-26-04994)].\n\n### 2.2. Agricultural Yield Prediction and Input Modeling Methods\n\nAgricultural yield prediction and input modeling aim to estimate crop production by modeling the relationships among environmental conditions, soil properties, management practices, and historical crop growth states [[33](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B33-sensors-26-04994)]. Early studies mainly relied on statistical regression, crop growth models, and production function models to quantify the effects of climate, soil, irrigation, and fertilization on yield [[34](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B34-sensors-26-04994),[35](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B35-sensors-26-04994)]. Although these methods are interpretable, they have limited ability to capture the nonlinear and dynamic interactions among meteorological conditions, soil characteristics, crop varieties, pests, and management practices that jointly determine agricultural production [[36](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B36-sensors-26-04994)]. Recent studies have introduced machine learning and deep learning methods to improve yield modeling [[37](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B37-sensors-26-04994)]. Tree-based models such as Random Forest and XGBoost, together with support vector machines, have demonstrated robust performance on structured agricultural data [[38](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B38-sensors-26-04994)], while sequential models including LSTM, GRU, and Transformer further improve the modeling of temporal dependencies and long-term environmental effects from multi-source observations [[39](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B39-sensors-26-04994),[40](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B40-sensors-26-04994)]. These advances have substantially improved prediction accuracy for agricultural production. However, existing studies remain primarily prediction-oriented. Most methods optimize yield without explicitly considering production costs or economic returns, despite the fact that maximizing yield does not necessarily maximize profit [[41](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B41-sensors-26-04994),[42](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B42-sensors-26-04994)]. In addition, current models generally learn statistical correlations between agricultural inputs and outputs rather than identifying the underlying causal relationships, making them vulnerable to confounding factors such as weather, soil conditions, and management differences [[43](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B43-sensors-26-04994)]. Consequently, they provide limited support for reliable input optimization [[44](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B44-sensors-26-04994)]. To address these limitations, the proposed causal-aware input–profit modeling module treats fertilization, irrigation, and pesticide application as intervenable variables, jointly estimating their effects on yield, cost, and profit from multimodal agricultural state representations, thereby extending agricultural modeling from prediction toward interpretable decision-oriented optimization [[45](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B45-sensors-26-04994)].\n\n### 2.3. Agricultural Economic Optimization and Intelligent Decision-Making Methods\n\nAgricultural economic optimization aims to allocate land, water, fertilizer, pesticides, labor, and other production resources to balance yield, quality, cost, and profit under resource constraints [[46](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B46-sensors-26-04994)]. Traditional approaches mainly rely on production functions, cost–benefit analysis, and input–output efficiency evaluation to quantify the economic effects of agricultural inputs and support production planning [[47](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B47-sensors-26-04994),[48](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B48-sensors-26-04994)]. Although these methods provide clear economic interpretations, they generally assume relatively stable environmental conditions and fixed input–output relationships, making them difficult to apply to real agricultural systems where weather, market conditions, and crop responses vary continuously across growth stages [[49](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B49-sensors-26-04994)]. Consequently, agricultural input optimization is increasingly regarded as a dynamic sequential decision-making problem rather than a static optimization task [[50](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B50-sensors-26-04994)]. Reinforcement learning (RL) provides an effective framework for learning long-term agricultural management strategies by optimizing cumulative rewards through continuous interactions with the environment [[51](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B51-sensors-26-04994),[52](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B52-sensors-26-04994)]. Existing studies formulate crop status, soil conditions, meteorological observations, and historical management records as states [[53](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B53-sensors-26-04994)], while irrigation, fertilization, and pesticide application are treated as actions, enabling optimization of long-term objectives such as yield, quality, resource consumption, and economic return [[54](https:\u002F\u002Fwww.mdpi.com\u002F1424-8220\u002F26\u002F15\u002F4994#B54-sensors-26-04994)]. Compared with prediction-based approaches, RL is better suited to sta","Sensors 2026-08-06","2026-08-06T00:00:00Z",67,{"impact":25,"substance":94,"depth":26,"authority":25,"freshness":95,"relevant":8,"comment":96},20,5,"论文提出多模态时空对齐与因果利润建模的农业投入优化方法，具有创新性，但影响范围限于学术研究，时效性一般。",[98],{"name":90,"url":87},[100,33,101,102,4],"智慧农业","多模态","强化学习",{"code":104,"slug":105,"name":106,"path":107,"parent_code":108},"H2","soil","土壤健康与改良","\u002Ftopic\u002Fenvironment\u002Fsoil","H",[],{"status":55,"locked":22,"version":56,"method":57,"evidence":111},{"candidates":112,"agricultural_evidence":61},[113,118,123],{"code":114,"score":28,"supported":61,"title":115,"abstract":116,"tags":117},"I1",[],[100],[100],{"code":104,"score":82,"supported":61,"title":119,"abstract":120,"tags":122},[],[121],"土壤",[],{"code":49,"score":7,"supported":22,"title":124,"abstract":125,"tags":126},[],[],[101,102],[128,129],"中国农业大学 强化学习 智慧农业 多模态","中国农业大学 强化学习","中国农业大学强化学习智慧农业多模态-373","2026-08-10T23:54:46.189166Z"]