A new multi-scale deep-learning framework combines high-frequency medium-resolution drone imagery with sparse high-resolution images to identify rice growth stages across hundreds of genetically diverse varieties. By introducing a Missing-Aware Gated Fusion (MAGF) mechanism, the model handles irregular sampling and phenological asynchrony, where different cultivars reach the same stage on different dates. Validated on more than 100,000 images covering roughly 500 rice cultivars over two growing seasons, the approach achieved an overall accuracy (OA) and F1-score of 0.873, with a Kappa coefficient of 0.84, while sharply reducing the drone flight time needed for high-resolution monitoring.
Crop phenology—the timing of stages such as jointing, heading, flowering, and maturity—is central to yield prediction and germplasm evaluation. In large rice breeding trials, hundreds of cultivars with diverse genetic backgrounds often develop asynchronously, making single-date observations insufficient. Manual surveys are labor-intensive and lack temporal continuity, while crop growth models need variety-specific calibration that is impractical at population scale. Drone remote sensing offers centimeter-level resolution, but high-resolution imagery is temporally sparse and costly, whereas medium-resolution imagery provides frequent coverage with less spatial detail. Because of these challenges, there is a pressing need for a robust, scalable modeling framework that preserves temporal continuity while retaining fine-scale spatial discrimination in large-scale rice germplasm trials.
Researchers from Nanjing Agricultural University, together with collaborators at The University of Tokyo and the Henan Academy of Agricultural Sciences, have developed a multi-scale spatial-temporal deep-learning framework for rice phenology identification. The study was published (DOI: 10.1016/j.plaph.2026.100222) in Plant Phenomics . The framework uses high-frequency medium-resolution (MR) images as a temporal anchor and sparse high-resolution (HR) images as spatial enhancement, integrating them through a Missing-Aware Gated Fusion (MAGF) mechanism and a long short-term memory (LSTM) network for temporal modeling.
The framework was tested on over 100,000 plot-level images covering approximately 500 rice cultivars across the 2023 and 2024 growing seasons. It classified five stages: Jointing–Booting Stage (JBS), Heading–Flowering Stage (HFS), Milky Stage (MKS), Dough Stage (DS), and Maturity Stage (MS). MobileNetV3 was chosen as the backbone after comparisons with VGG16, ResNet50, and GoogLeNet, delivering the best balance of accuracy and efficiency, with only 5.48 M parameters and 0.23 G FLOPs. The MAGF mechanism treats MR features as a stable anchor and HR features as conditional refinement, using a binary availability mask so missing HR data does not contaminate the feature space. When HR data is absent, the fusion term vanishes and the model relies on the MR anchor; when present, HR features are adaptively weighted and added. The full asynchronous MR + HR + LSTM model reached an OA of 0.873 and an F1-score of 0.873 with a Kappa coefficient of 0.84, while a dense HR benchmark reached 0.887 at far higher cost. Temporal modeling improved F1-scores by 1.7% to 4.7% during ambiguous transition stages, with the largest gains at JBS, HFS, and MKS. A hybrid sampling strategy—daily MR imaging plus weekly HR imaging—cut weekly flight time from about 28 hours to roughly 6 hours while maintaining high accuracy.
The authors said the framework was designed to reflect how phenology actually unfolds in the field. They said that frequent medium-resolution observations maintain continuous situational awareness, while sparse high-resolution flights act as periodic calibration at critical stages. They added that even when high-resolution acquisition was reduced to once every 14 days, fusion performance remained clearly superior to daily medium-resolution monitoring alone. They said this matters for breeding programs that cannot afford daily high-resolution flights across thousands of plots.
The approach offers a flexible, scalable, and cost-effective tool for high-throughput phenotyping in precision breeding. When a model trained on 2024 data was applied directly to the independent 2023 dataset, it maintained an OA of 0.774 and an F1-score of 0.738 under a 3-day error tolerance, with recall for maturity consistently exceeding 0.96. The authors suggest that a sampling interval of 5 to 7 days for high-resolution imagery is a practical optimum, keeping accuracy loss within about 1.5%. Future work will explore integrating meteorological covariates such as growing degree days (GDD) and photosynthetically active radiation (PAR) into the temporal model, and linking deep features with genotypes to build a genotype–phenotype–environment (G × P × E) evaluation framework.
