More and more plastic litter ends up in oceans every day. Satellite images can help detect accumulations of litter along shores and at sea, so that it could be taken out. A research team has developed a new artificial intelligence model that recognises floating plastics much more accurately in satellite images than before. Even when the images are partly covered by clouds, or weather conditions are hazy.
Our society relies heavily on plastic products and the amount of plastic waste is expected to increase in the future. If not properly discarded or recycled, much of it accumulates in rivers and lakes. Eventually it will flow into the oceans, where it can form aggregations of marine debris together with natural materials like driftwood and algae. A new study from Wageningen University and EPFL researchers, recently published in Cell iScience, has developed an artificial intelligence-based detector that estimates the probability of marine debris shown in satellite images. This could help to systematically remove plastic litter from the oceans with ships.
Searching through satellite images with AI
Accumulations of marine debris are visible in freely available Sentinel-2 satellite images that capture coastal areas every 2-5 days worldwide on land masses and coastal areas. Because these amount to terabytes of data, the data needs to be analysed automatically through artificial intelligence models like deep neural networks. Marc Rußwurm, Assistant Professor at Wageningen University: "These models learn from examples provided by oceanographers and remote sensing specialists, who visually identified several thousand instances of marine debris in satellite images on locations across the globe. In this way they 'trained' the model to recognise plastic debris."
Improved detection in challenging conditions
The researchers developed an AI-based marine debris detector that estimates the probability of marine debris present for every pixel in Sentinel-2 satellite images. The detector is trained following data-centric AI principles that aim to make best use of the limited training data that is available for this problem. One example is the design of a computer vision algorithm that snaps manual annotations from experts precisely to the debris visible in the images. With this tool, oceanographers and remote sensing experts can provide more training data examples by being less precise in the manual clicking of outlines.
Overall, this training method combined with the refinement algorithm teaches the deep artificial intelligence detection model to better predict marine debris objects than previous approaches. Rußwurm: "The detector remains accurate even in more challenging conditions. For example when cloud cover and atmospheric haze make it difficult for existing models to identify marine debris precisely."
Following plastic debris after the Durban Easter floods 2019
Detecting plastics in marine debris under difficult atmospheric conditions with clouds and haze is particularly important, as often plastics are washed into open waters after rain and flood events. This is shown by the Durban Easter floods in South Africa: In 2019 a long period of rain led to overflowing rivers, resulting in much more litter being washed away than normally. It was taken along through the Durban harbour (left picture) into the open Indian Ocean (right picture). In satellite images, such objects floating between clouds are hard to distinguish when using common red-green-blue colour 'channels'. They can be visualised by switching to other spectral channels, including near-infrared light.
Double view reveals drift directions
Apart from more accurate prediction of marine debris aggregations, the detection model will also notice debris in daily-accessible PlanetScope images. "Combining weekly Sentinel-2 with daily PlanetScope acquisitions can close the gap towards continuous daily monitoring", explained Rußwurm. "Also, PlanetScope and Sentinel-2 sometimes capture the same patch of marine debris at the same day only a few minutes apart. This double view of the same object at two locations reveals the drift direction due to wind and ocean currents on the water. This information can be used to improve drift estimation models for marine debris."
Further explorations
These directions will be explored further in the research of Prof. Marc Rußwurm at Wageningen University together with Prof. Tim van Emmerik, who is an expert in river plastics, and partners in the Netherlands, such as the Ocean Cleanup, who collect plastics on open oceans with dedicated ships. It originates from work in the AI for Detection of Plastics with Tracking (ADOPT) project in collaboration with the Swiss Data Science Center (a joint venture between ETH Zurich and EPFL) and EPFL by Prof. Devis Tuia, Dr. Emanuele Dalsasso, and Prof. Marc Rußwurm that investigates the continuous tracking of marine debris in between satellite imagery further.
