© ANNE WESTON, FRANCIS CRICK INSTITUTE / SCIENCE PHOTO LIBRARY
Artificial intelligence (AI) is quickly changing the way medical diagnoses and prognoses are made. For example, machine learning systems can already diagnose Alzheimer's disease from pathology images with greater accuracy than human experts.
But AI suffers from a major limitation: the reasons it makes a prediction are so complicated that it is effectively a black box to human experts. The AI deep learning systems used in medical research often rely on algorithms in which correlations are reinforced (backpropagation), forming complex networks to provide suggested prognoses or diagnoses. But it can be a bit like looking up the answer to a math problem at the back of a text book-it might give you the right answer, but if you don't delve into the process of how it was derived, you're none the wiser when confronted with a different problem.
This issue was acknowledged by the guidelines drawn up by the G20 summit held in Osaka in June 2019, which call for AI to be developed in a fair and accountable manner with transparent decision-making processes and appropriate safeguards.
Opening the black box
Now, Yoichiro Yamamoto of the RIKEN Center for Advanced Intelligence Project and his co-workers in Japan are lifting the lid on this black box.
Yamamoto's team has developed a deep-learning system that can surpass human pathology experts in accurately predicting the probability that prostate cancer will reoccur within one year1.
They trained their system by using a portion of more than 13,000 pathological images of whole prostates acquired from a hospital in Tokyo. The researchers then evaluated its performance by letting it loose on the remaining images and seeing how well its predictions matched actual outcomes.
In the study, human pathologists scored an accuracy, as AUC (area under the curve), of 0.744 on a scale between 0 and 1 (where a score of 1 indicates perfect accuracy), while the machine achieved an impressive 0.820. In the clinic, this could help doctors decide whether a patient needs additional treatments or merely to be monitored.
But Yamamoto's system goes further and shows what image features the AI has correlated positively or negatively with recurrence, so that pathologists can try to better understand why.
One example of a novel insight that the team's deep-learning technology has uncovered is that it sometimes highlights images that don't include any cancer cells. "AI says these images are very important for predicting cancer recurrence, but you won't find them in pathology books produced by humans," says Yamamoto. "This is new knowledge." More research is required to find out what is important in these images.
"Most AI techniques have only been used for classification, so they essentially mimic human pathologists," explains Yamamoto. "But our deep-learning technology goes beyond that-it can glean new insights."
The technology is therefore more than just a tool for helping pathologists to enhance the accuracy of their prognoses; it is also a powerful research assistant that can uncover new areas for human researchers to explore. "This is breaking new ground for AI," notes Yamamoto.
Better working and learning together
While the deep-learning system was more accurate than human pathologists, the highest accuracy was achieved when both AI and humans cooperated in making prognoses (a score of 0.842). Yamamoto thinks this is because humans and machines complement each other. "Humans and AI have different strengths when evaluating pathology images: humans tend to be better at focusing on the small areas that contain cancer cells, whereas AI is better at getting an overall view based on the entire image." So Yamamoto doesn't anticipate that AI will replace human pathologists any time soon. Indeed, he sees a two-way interaction in which humans will learn from machines and machines from humans.
Robust across hospitals
Another critical point is that the deep-learning system had similar accuracies when it was used in two other hospitals in different prefectures in Japan. "This is a very significant result," says Yamamoto. He says it reveals the potential for very broad use of their system.
This robustness in different settings is the result of the use of our unsupervised learning-based algorithm, says Yamamoto. In this type of learning, a computer is fed unlabeled images and the outcome and has to figure out the relationship between the two on its own. In contrast, supervised deep learning involves spoon-feeding the machine with data that has been labeled with various output parameters. This makes it easier to implement, but its accuracy sometimes reduces markedly when it is used in environments different from the one in which it was trained, in which labeling or imaging might be slightly different. In contrast, unsupervised learning was much more stable in their dataset.
Yamamoto thinks that unsupervised learning should be highlighted in medicine more often and he says the technique is not restricted to prostate cancer. Yamamoto and his team are now applying it to breast cancer and other rare cancers. They may also extend it to other, non-medical fields that involve large amounts of images.
Reference
- 1. Yamamoto, Y., Tsuzuki, T., Akatsuka, J., Ueki, M., Morikawa, H., Numata, Y., Takahara, T., Tsuyuki, T., Tsutsumi, K., Nakazawa, R., Shimizu, A., Maeda, I., Tsuchiya, S, Kanno, H., Kondo, Y., Fukumoto, M., Tamiya, G., Ueda, N. & Kimura, G. Automated acquisition of explainable knowledge from unannotated histopathology images. Nature Communications 10, 5642 (2019). doi: 10.1038/s41467-019-13647-8
About the researcher
Yoichiro Yamamoto received his Doctor of Medicine and PhD degrees from Tohoku University. His PhD thesis focused on the cell dynamics in the carcinogenesis process. From 2012 onwards, he focused on mathematical modeling of cancer cell dynamics at the Mayo Clinic and Harvard University. This led to his current interest in artificial intelligence (AI). In 2014-2015, his research at Heidelberg University in Germany focused on computer science, and he succeeded in developing a new way to combine pathology and machine learning. In 2017, he moved to the RIKEN Center for Advanced Intelligence Project (AIP) to start his own lab. His lab's mission is to discover disease mechanisms and new therapies through state-of-the-art AI technologies and medical big data.
