Artificial intelligence has exploded in popularity in recent years, and many proponents are excited about its potential uses in medicine: for example, processing samples quickly or identifying markers of disease that may be missed by the human eye. However, is applying AI always the best option?
Researchers found that while an AI method called virtual staining can improve the use of medical images in certain cases, in other situations it may actually decrease the ability to get useful information from those images. In general, they urge caution when deciding whether to apply AI to a given workflow, to ensure that it actually improves accuracy compared to other methods.
"The general conclusion is that AI can be a great tool — it does help in some cases — but you have to be a little bit cautious," said Sourya Sengupta, a graduate student at the Beckman Institute for Advanced Science and Technology and this study's lead author.
This study was conducted by researchers in the Center for Label-free Imaging and Multiscale Biophotonics , which aims to improve imaging technologies for clinical and research applications by developing new imaging methods and algorithms. In addition to Sengupta, CLIMB researchers Phuong Nguyen, Frank Brooks , Yang Liu and Mark Anastasio all collaborated on this project.
Most of us have had a medical image taken during a doctor's appointment, such as an ultrasound, MRI or X-ray. These essential tools help researchers and clinicians diagnose diseases, test new treatments and monitor patient health. Another common class of medical images are microscopy images, which allow clinicians to get a closer look at magnified tissue and cell samples.
To improve a microscopy image's contrast — for example, to make a certain part of a cell stand out so that clinicians can analyze its features — the tissue or cell samples are often stained using dyes or other chemicals. While widely used, staining can be time-intensive and may damage the cells.
Label-free imaging is an alternative to staining in which chemicals are not added to the sample. Instead, researchers use natural properties of biological materials to make observations and create images. For example, measuring the different ways light passes through transparent objects like cells gives us information about cell density and growth.
However, this method also has drawbacks. Label-free images still tend to have less contrast than stained images, which can make it difficult to identify key features. To improve the usefulness and reliability of label-free images, there has been recent interest in a new method called virtual staining.
In the virtual staining process, a computational model analyzes a label-free image and predicts what the image would look like stained. Ideally, this would result in an image with the high contrast of a stained image, but produced much more quickly and without the potential for chemically damaging the sample. However, it is important to confirm these virtually stained images are truly accurate and useful for biological discoveries and clinical applications.
"In medicine or in drug discovery, taking images is not the end goal," Sengupta said. "In biomedical imaging, we always think in terms of a task: a biological or clinical application that the images are meant to serve. So we started asking: these computationally generated images may look real, but do they actually help with the real task?"
One of the biggest challenges in answering this kind of question is simply having enough data. Researchers often need large sets of paired images — one from label-free imaging and the other from fluorescent staining — to train and test various AI models. Fortunately, Liu's team recently developed the Omni-Mesoscope , a powerful high-throughput imaging system that can capture tens of thousands of cells at different states within minutes, creating large, high-quality datasets. These datasets provided the foundation for testing how virtually stained images perform in real-world analytical tasks.
The researchers tested how virtually stained images performed compared to label-free images and stained images in two tasks. First, the images were used in a segmentation task: a process in which a neural network identifies individual cell nuclei and crops them to each be their own picture. Like cropping a photo, this allows researchers and clinicians to hone in on the most important parts of the image.
Secondly, researchers used the images in a cell classification task, in which the network identified what stages different cells were in after a drug treatment. This task has applications for monitoring drug effectiveness in research and in disease treatment.
A comparison of label-free (first column), virtually stained (middle columns) and fluorescent stained (last column) images of two cells. The cell in the bottom row was treated with a drug, while the cell in the top row was not.
For both tasks, researchers assessed the performance of different networks when using each type of image. The researchers wanted to know if the relative success of each image type would change depending on the properties of the network being used, so they repeated the tasks using five different networks.
Many networks perform similar tasks, but some networks may be better than others at representing complex functions or relationships depending on how the network is programmed to learn. These networks are called high-capacity networks. The networks used in this study had a range of capacities, so the researchers could see whether capacity impacted how the networks used the virtually stained images.
When processed by low-capacity networks, the virtually stained images performed much better than the label-free images. However, with high-capacity networks, this was not the case. When applied to the segmentation task, the virtually stained and label-free images performed about the same when processed by high-capacity networks. When applied to the cell classification task, however, the virtually stained images performed substantially worse. In other words, when using a high-capacity network to analyze your images, you would be more likely to get accurate information if you used label-free images rather than virtually stained ones.
This result is consistent with a concept called the data processing inequality, which states that processing any image (such as by virtual staining) cannot increase the information contained in that image, Sengupta said. This is similar to touching up a family photograph: you can blur the background to make the people stand out, but no amount of edits will open the eyes of someone who was blinking when the shutter clicked.
Low-capacity networks are likely helped by virtually stained images because processing can emphasize important information. In contrast, high-capacity networks, which can already pick out complex relationships from the label-free images, are not helped by virtual staining. The virtual staining process may even remove information that is crucial for certain tasks, which may explain why virtually stained images performed worse than label-free images in the cell classification task.
While AI has potential applications in many areas of healthcare, Sengupta reminds clinicians, researchers and members of the public interested in this technology to be cognizant of its limitations. If AI is being used for a specific task, it is important to verify that it will actually be beneficial in that situation.
"Even if AI is a buzzword now, you have to be a little bit cautious when applying it in sensitive domains like biomedical imaging and healthcare," Sengupta said. "In a lot of cases, AI is very useful, but it might not always be."
