AI Transforms Medical Image Analysis

CSIRO

Key points

  • AI is now a prominent feature of the healthcare landscape.
  • One type of AI called visual language models is being used to "read" X-rays and generate reports.
  • The technology will not replace human analysis but provide a tool to support radiologists.

One in two Australians regularly use artificial intelligence (AI), with that number expected to grow. AI is showing up in our lives more prominently than ever, with the arrival of ChatGPT and other chatbots.

Researchers at CSIRO's Australian e-Health Research Centre (AEHRC) are exploring how AI – including the systems that underpin chatbots – can be leveraged for a more altruistic endeavour: to revolutionise healthcare.

Earlier versions of ChatGPT were built on an AI system called a large language model (LLM) and were entirely text-based. You would 'talk' to it by entering text.

The latest version of ChatGPT, for instance, incorporates visual-language models (VLM) which add visual understanding on top of the LLM's language skills. This allows it to 'see', describe what it 'sees' and connect it to language.

AEHRC researchers are now using VLMs to help interpret medical images such as X-rays.

It's complicated technology, but the aim is straightforward: to support radiologists and reduce the burden on them.

X-ray of the human body, showing ribs, spine
This work enables automated reporting of X-rays

Visual language models are transforming X-ray analysis

Dr Aaron Nicolson, Research Scientist at AEHRC, is one of the researchers working on the project.

He said any kind of image can be used with VLMs, but his team is focusing on chest X-rays.

Chest X-rays are used for many important reasons, including to diagnose heart and respiratory conditions, screen for lung cancers and to check the positioning of medical devices such as pacemakers.

Typically, trained specialists – radiologists – are required to interpret the complex images and produce a diagnostic report.

But in Australia, radiologists are overburdened.

"There are too few radiologists for the mountain of work that needs to be completed," Aaron said.

The problem will likely get worse with the number of patients and chest X-rays taken set to keep increasing, especially as the population ages.

That's why Aaron is developing a model that uses a VLM to produce radiology reports from chest X-rays.

"The goal is to create technology that can integrate into radiologists' workflow and provide assistance," he said.

Man typing at computer at desk, with smile on face
Aaron Nicolson working on his model for automated X-ray reporting

Practice makes (almost) perfect

Training the VLM involves lots of data. The more information a model has, the better it can make predictions.

The VLM is given the same information that a radiologist would receive – X-ray images and the patient's referral, Aaron explained.

"Then we give the model the matching radiology report written by a radiologist. The model learns to produce a report based on the images and information it is given," he said.

Like humans, AI models improve by practising.

"We train the model using hundreds and thousands of X-rays. As the model trains on more data, it can produce more accurate reports," said Aaron.

At this stage of his research, Aaron was looking to improve the accuracy of the reports even further – so he decided to try something new.

"We gave model the patient's records from the emergency department as well," he said.

"That means information like the patient's chief complaint when triaged, their vital signs over the course of the stay, the medications they usually take and the medications administered during the patient's stay."

Just as he had hoped, giving the model this extra information improved the accuracy of the radiology reports.

"We are trying to get the technology to a point where it can be considered for prospective trials. This is a big step in that direction," he said.

Infographic describing the workflow of a multimodal large language AI model
Workflow of the large language model.

Ethical and applicable AI

As well as generating diagnostic reports from chest X-ray images, AEHRC is exploring other applications of VLMs.

Dr Arvin Zhuang, at post-doc at AEHRC is using VLMs to retrieve information from images of medical documents. Processing the documents as an image rather than text enables the information to be retrieved more efficiently.

It's an exciting time for Aaron and Arvin, but ethical and safety considerations are always at the front of their minds.

"We want to make sure that the model is effective for all populations. To do that, we have to consider and manage issues like demographic biases in the data we train our models on," Aaron said.

He also notes that the technology is not designed to replace medical specialists.

"The technology will not be making clinical decisions by itself. There will always be a radiologist in the loop," Aaron said.

Aaron and his team are currently conducting a trial of the technology in collaboration with the Princess Alexandra Hospital in Brisbane, assessing how the AI-generated reports compare with those produced by human radiologists.

They are also actively seeking additional clinical sites to participate in further trials, to evaluate the technology's effectiveness across a broader range of settings.

/Public Release. This material from the originating organization/author(s) might be of the point-in-time nature, and edited for clarity, style and length. Mirage.News does not take institutional positions or sides, and all views, positions, and conclusions expressed herein are solely those of the author(s).View in full here.