A new study published in Big Earth Data systematically evaluates popular Uncertainty Quantification (UQ) methods and metrics for AI/ML-based geospatial applications, addressing challenges of predictive uncertainty, trustworthiness, and real-time integration in complex Earth system modelling. Using a PM2.5 calibration case study, it demonstrates that Deep Ensembles and Bayesian Neural Networks implemented in TensorFlow achieved the best reliability and calibration performance, while also highlighting framework-specific differences between TensorFlow and PyTorch in geospatial UQ applications.
Citation
Malarvizhi, A. S., Smith, K., & Yang, C. (2026). Uncertainty quantification in geospatial AI/ML applications: methods, metrics, and open-source support with an air quality use case. Big Earth Data, 1–34. https://doi.org/10.1080/20964471.2026.2629680
Abstract
The adoption of AI/ML models in geospatial applications faces challenges related to trustworthiness and reliability due to uncertainties in model processing and datasets. Such uncertainties are exacerbated by the complexity of Earth's systems and constraints, such as data quality and measurement accuracy. Despite growing interest, the field lacks a systematic comparison of Uncertainty Quantification (UQ) methods tailored to the unique challenges of geospatial modelling, particularly in real-time settings. These challenges should be addressed by integrating UQ into geospatial workflows, enabling stakeholders and users to understand model uncertainties and make decisions with confidence. This paper addresses these gaps by systematically evaluating three popular UQ methods, focusing on their suitability for geospatial applications and their ability to address predictive uncertainties. Additionally, we analyzed various UQ metrics, highlighting their role in assessing predictive capacity and calibration. However, improvements are needed to establish unified frameworks and streamline integration for real-time and large-scale applications. A case study on PM2.5 calibration demonstrated the practical utility of UQ in enhancing air quality (AQ) data reliability. Results highlight that Deep Ensembles (DE) in TensorFlow achieved the strongest performance, followed by BNN in TensorFlow, which offered dependable calibration and accuracy for the case study. MCD in TensorFlow came next, delivering solid performance but with less adaptable uncertainty at extreme PM2.5 values. MCD in PyTorch performed the poorest, with lower accuracy and unreliable calibration. These results demonstrate that TensorFlow consistently outperformed PyTorch across methods, underscoring framework-specific differences in implementing UQ. Overall, this study conducted a comprehensive review and experimental study of UQ methods and frameworks, offering critical insights into their theoretical foundations, practical implementation, and performance in geospatial applications using PM2.5 calibration as an example.
#geoscience #remote sensing #earth observation #GIS #data analysis #Big Data #visualization
Big Earth Data is an interdisciplinary Open Access journal which aims to provide an efficient and high-quality platform for promoting the sharing, processing and analyses of Earth-related big data, thereby revolutionizing the cognition of the Earth's systems. The journal publishes a wide range of content, including Research Articles, Review Articles, Data Notes, Technical Notes, and Perspectives. It is now included in ESCI (IF=3.8, Q1), Scopus (CiteScore=9.0, Q1), Ei Compendex, GEOBASE, and Inspec. Starting from 2023, Big Earth Data has announced a new award series for authors: Best and Outstanding Paper Awards.
