Artificial Intelligence (AI) plays a growing role in remote sensing. In particular, during the last decade there has been an exponentially increasing interest in deep learning research for analysis of optical satellite images, hyperspectral images, and radar images. The main reasons for this interest is the increased availability of a wealthy stream of data coming from different Earth observation instruments and that AI techniques enable a learning-based "data model" in remote sensing. In order to promote research in this area, we have organized a special focus on Artificial Intelligence Innovation in Remote Sensing in SCIENCE CHINA Information Sciences(Vol.66, Issue.4, 2023). Eight papers are included in this special focus as detailed below.
Multimodal remote sensing imagery interpretation (MRSII) is an emerging direction in the communities of Earth Observation and Computer Vision. In the contribution entitled "From single- to multi-modal remote sensing imagery interpretation: a survey and taxonomy", Sun et al. provide a comprehensive overview on the developments of this field. Importantly, in the paper, an easily understandable hierarchical taxonomy is developed for the categorization of MRSII, further providing a systematic discussion on the recent advances and guidance to researchers in many realistic MRSII problems.
Hyperspectral imaging enables the integration of 2D plane imaging and spectroscopy to capture the spectral diagram/signatures and spatial distribution of the objects in the region of interest. However, ground objects and the reflectance received by the imaging instruments may be degraded, owing to environmental disturbances, atmospheric effects and hardware limitations of sensors. HSI restoration aims at reconstructing a high-quality clean hyperspectral image from a degraded one. In the contribution entitled "A survey on hyperspectral image restoration: from the view of low-rank tensor approximation", Liu et al. present a cutting-edge and comprehensive technical survey of low-rank tensor approximation toward HSI restoration, with a specific focus on denoising, fusion, restriping, inpainting, deblurring and super-resolution, along with their state-of-the-art methods, and quantitative and visual performance assessment.
Recently, hyperspectral and multispectral image fusion (aimed at generating images with both high spectral and spatial resolutions) has been a popular topic. However, it remains a challenging and underdetermined problem. In the contribution entitled "Learning the external and internal priors for multispectral and hyperspectral image fusion", Li et al. propose two kinds of priors, i.e., external priors and internal priors, to regularize the fusion problem. The external prior represents the general image characteristics and is learned from abundant sample data by using a Gaussian denoising convolutional neural network trained with additional grayscale images. On the other hand, the internal prior represents the unique characteristics of the hyperspectral and multispectral images to be fused. Experiments on simulated and real datasets demonstrate the superiority of the proposed method. The source code for this paper is available at https://github.com/renweidian.
Wide-beam autofocus processing is essential for high-precision imaging of airborne synthetic aperture radar (SAR) data, due to the absence of inertial navigation system/global positioning system (INS/GPS) data or insufficient accuracy. In the contribution entitled "Wide-beam SAR autofocus based on blind resampling", Chen and Yu propose a full-aperture autofocus method for wide-beam SAR based on blind resampling. The proposed method does not require INS/GPS data as baseline methods, which can significantly improve the overall image quality. The measured data processing results of the wide-beam SAR verify the effectiveness of the newly proposed algorithm in this contribution.
Remote sensing image (RSI) semantic segmentation has attracted increased research interest during the last few years. However, RSI is difficult in holistic processing for currently available graphics processing units cards on account of large field-of-views (FOVs) of the imagery. Furthermore, prevailing practices such as image down sampling and cropping inevitably decrease the quality of semantic segmentation. In the contribution entitled "MFVNet: a deep adaptive fusion network with multiple field-of-views for remote sensing image semantic segmentation", Li et al. propose a new deep adaptive fusion network with multiple FOVs (MFVNet) for RSI semantic segmentation, surpassing the previous state-of-the-art models on three typical RSI datasets. Codes and pre-trained models for this paper are publicly available https://github.com/weichenrs/MFVNet.
Change detection of buildings, given two registered aerial images captured at different times, aims to detect and localize image regions where buildings have been added or torn down between flyovers is challenging. The main challenges are the mismatch of the nearby buildings and the semantic ambiguity of the building facades. In the contribution entitled "Detecting building changes with off-nadir aerial images", Pang et al. present a multi-task guided change detection network model, named as MTGCD-Net, providing indispensable and complementary building parsing and matching information, along with extensive comparisons to existing methods. More importantly, a new benchmark dataset, named BANDON, were created fin this research and it is available at https://github.com/fitzpchao/BANDON.
Photovoltaic devices, a typical new energy source, have progressed rapidly and become among the main sources of power generation in the world. In the contribution "AIR-PV: a benchmark dataset for photovoltaic panel extraction in optical remote sensing imagery", Yan et al. propose a large-scale benchmark dataset, namely AIR-PV, for photovoltaic panel extraction in RS imagery. The main features of this benchmark dataset are: (1) large-scale with wide distribution across five provinces of western China to cover a wide range of geographical styles and background diversity, covering more than 3 million square kilometers with more than 300,000 photovoltaic panels; (2) one of the earliest publicly available datasets (https://github.com/AICyberTeam) for photovoltaic panel extraction, providing a standard data foundation for applying advanced deep learning technology to photovoltaic panel extraction in remote sensing, thereby promoting various social applications related to photovoltaic power.
In the last contribution, "Multi-layer composite autoencoders for semi-supervised change detection in heterogeneous remote sensing images", Shi et al. develop concise multi-layer composite autoencoders for change detection in heterogeneous remote sensing images, which avoid complex alignment or transformations in the traditional change detection frameworks, which only require 0.1% of true labels (approaching the cost of unsupervised models).
Please find below details of this Special Topic: Artificial Intelligence Innovation in Remote Sensing.
Sun X, Tian Y, Lu W X, et al. From single- to multi-modal remote sensing imagery interpretation: a survey and taxonomy. Sci China Inf Sci, 2023, 66(4): 140301
https://link.springer.com/article/10.1007/s11432-022-3588-0
Liu N, Li W, Wang Y J, et al. A survey on hyperspectral image restoration: from the view of low-rank tensor approximation. Sci China Inf Sci, 2023, 66(4): 140302
https://link.springer.com/article/10.1007/s11432-022-3609-4
Li S T, Dian R W, Liu H B. Learning the external and internal priors for multispectral and hyperspectral image fusion. Sci China Inf Sci, 2023, 66(4): 140303
https://link.springer.com/article/10.1007/s11432-022-3610-5
Chen J L, Yu H W. Wide-beam SAR autofocus based on blind resampling. Sci China Inf Sci, 2023, 66(4): 140304
https://link.springer.com/article/10.1007/s11432-022-3574-7
Li Y S, Chen W, Huang X, et al. MFVNet: a deep adaptive fusion network with multiple field-of-views for remote sensing image semantic segmentation. Sci China Inf Sci, 2023, 66(4): 140305
https://link.springer.com/article/10.1007/s11432-022-3599-y
Pang C, Wu J, Ding J, et al. Detecting building changes with off-nadir aerial images. Sci China Inf Sci, 2023, 66(4): 140306
https://link.springer.com/article/10.1007/s11432-022-3691-4
Yan Z Y, Wang P J, Xu F, et al. AIR-PV: a benchmark dataset for photovoltaic panel extraction in optical remote sensing imagery. Sci China Inf Sci, 2023, 66(4): 140307
https://link.springer.com/article/10.1007/s11432-022-3663-1
Shi J, Wu T C, Yu H W, et al. Multi-layer composite autoencoders for semi-supervised change detection in heterogeneous remote sensing images. Sci China Inf Sci, 2023, 66(4): 140308
https://link.springer.com/article/10.1007/s11432-022-3693-0
