New work explaining the inner workings of artificial intelligence could provide a way around the threat of AI 'Model Collapse', potentially averting growing numbers of AI hallucinations in the future.
First coined in 2024, 'Model Collapse' refers to a scenario where an AI model trained on AI produced data ceases to provide accurate results, instead producing inaccurate "gibberish" because of the poor quality of its training data.
Some have warned that high quality text data to train systems like Large Language Models (LLMs) is set to run out as early as this year , and so data produced by models themselves has taken a larger training role – inviting the threat of model collapse.
Through analysis of a simple yet powerful set of statistical models called Exponential Families, the team of researchers, from King's College London, the Norwegian University of Science and Technology, and the Abdus Salam International Centre for Theoretical Physics, found that it took as little as one datapoint from the outside world integrated into their training to prevent this in all cases.
While much simpler than the LLMs, the Exponential Family models are some of the most powerful models used for modelling data. The team hope that by shining a light on closed-loop learning in such a simple yet powerful setting, they can establish principles for how to potentially avoid model collapse in more commonly used LLMs.
Professor Yasser Roudi, Professor of Disordered Systems in the Department of Mathematics at King's, explains "Previous work undertaken on model collapse primarily looks at large, complicated LLMs, where it's not clear how these models work and if results are repeatable – it is why you get unexplained hallucinations, where you can't explain why an AI has generated a wrong answer.
"By focusing on a simple model, we can establish why adding just one data point prevents them from generating gibberish from an objective, statistical standpoint. From this foundation, we can establish principles that will be vital in future AI construction. As larger models are deployed in areas touching our lives, from ChatGPT to self-driving cars, and synthetic data takes on a larger share of AI training, computer scientists will have the tools to prevent this potentially disastrous scenario."
Published in Physical Review Letters, the study lays out how standard training of Exponential Families (called Maximum Likelihood) in a closed-loop scenario, where a model is trained only on data it produces, will always lead to model collapse.
However, the work shows that introducing a single datapoint from outside the closed loop, or by incorporating a prior belief during training e.g. from previously acquired knowledge prevents model collapse. Surprisingly this effect of a single datapoint from the outside world is present even when the amount of machine-generated datapoints is infinitely larger.
The authors also provide evidence that similar phenomenon is observed in another class of models, Restricted Boltzmann Machines, suggesting that their results are likely not restricted only to Exponential Families. In the future, the group hope to test these first principles against larger and more complex models like neural networks to validate
