Lancaster University researchers have identified gaps in AI security research following a government-commissioned thematic review.
The research team were invited by the Department for Science, Innovation and Technology (DSIT) to review existing global research into AI security and identify blind spots to help guide where research, policy, and investment into AI security is required.
The team performed an extensive review into the peer-reviewed AI security literature since 2020, using data-driven techniques to identify more than 9,100 sources, extract key themes, and identify and quantify the gaps.
Their review reveals five key areas where AI security currently needs further research:
1) Data integrity: Research is needed to develop methods to formally guarantee, or verify, the integrity (safety, authenticity and security) of data used by AI systems, such as data used to train AI models.
2) Security of AI system infrastructure: More research attention needs to be given to the infrastructure underpinning generative or agentic-AI systems to develop a better understanding of how emerging AI systems, such as large language models, change what is needed to be secure-by-design.
3) Securing against end-user risks: More consideration needs to be given to how end-users can pose a risk to, or be at risk from, AI systems by relying on 'hallucinated' or malicious AI results.
4) Data and model provenance: There is an increased need to understand how third-party AI models are developed and trained and their associated security implications.
5) Model disposal: More attention needs to be given about how to dispose of, or decommission, AI systems safely at the end of their lifespan.
The research highlighted that agentic-AI systems are an emerging threat with significant gaps in security research, with the review authors expecting research in this area to increase as these AI models become more prevalent.
Lead author Dr Edward Austin, a research fellow in Lancaster University's School of Computing and Communications, said: "The review findings highlight how the cybersecurity of AI goes beyond the models themselves, and that more research is needed to understand and secure these systems across their lifecycle. With AI adoption across society increasing, this is crucial for the safe use of this technology."
The review was led by Professor Nick Race, Associate Dean for Research in Lancaster University's Faculty of Science and Technology, and Professor of Networked Systems.
He said: "We were delighted to be invited by DSIT to lead a review into the AI security research landscape, supporting the government in their work to enhance the cyber resilience of the UK."
The 'Thematic review and gap analysis on AI security' was also authored by Dr Tessa Wilkie, of Lancaster University's School of Computing and Communications.
The review is available at the government website.
