Researchers at Aalto University, together with international partners, have developed the most accurate model yet of how humans read. The new model uses reinforcement learning, a type of AI used in robotics, to explain—and recreate—the choices readers make as they move through text.
'For the first time we've used AI methods to understand—not just mimic—how people read,' says Professor Antti Oulasvirta from Aalto University. In a study to be published on Monday, August 10, in Nature Human Behaviour, researchers say the model could power smarter Augmented Reality (AR) displays and tailor complex texts to different readers and everyday situations.
Earlier models learned from large datasets pairing text snippets with eye tracking data, then mimicked human behaviour, but they lacked true understanding of the content and didn't generalise well across languages or contexts, explains Oulasvirta. In contrast, the new model follows the psychological mechanisms readers use to direct attention, revealing how understanding is built as the eyes move through words, sentences and paragraphs.
Understanding how human memory serves reading is the key to unlocking enormous potential for customisable apps, services or products, according to Oulasvirta.
'We read all the time, yet throughout written history we have read texts that have been produced for mass use and not for an individual person and a specific situation,' he says. 'Now we are in a position to change that.'
How it works
The new model is guided by resource rationality—the idea that while reading, we constantly decide where to look next to improve our understanding as much as possible within the time available. Decisions about gaze allocation are made at three levels: word, sentence and text. They are influenced by factors such as a reader's language, memory capacity and their vision and eye speed. For example, a fast reader with a good memory may jump briskly from one paragraph to the next, whereas a reader with a poorer memory is more likely to loop back.
'Reading feels effortless, but your brain is constantly deciding where to look, what to skip, and when to backtrack—spending attention like a budget to maximize understanding,' says Professor Shengdong Zhao from City University of Hong Kong.
The researchers added reader characteristics as parameters so that each could be adjusted, then let the model learn for itself the best strategy for directing attention.
'We placed the model in a world with millions of texts. Then, using AI-based reinforcement learning, we trained it to optimise eye movements so that it truly understands what it reads,' Oulasvirta explains.
As it reads, the model forms a condensed description of the text's content. When a crucial word or clause is missing, the gaze can be directed to gather that information. The model's understanding can be tested by asking what it retained from the text within the given time and constraints.
When the researchers compared the model's attention-allocation decisions with real human eye-tracking data they found that its decisions mirrored readers' behaviour. In practice, they had succeeded in building a model of an average reader that can be tailored to different reader profiles.
What's next?
The development paves the way to new reading support tools and personalised text design. For example, the model could be used to enable smart glasses that pace and lay out on-screen text to fit the situation and the user's needs, or to customise texts to suit users.
'We could take the same source text—say, a convoluted piece of legal writing—and with little effort produce versions that are more comprehensible for different readers,' Oulasvirta says.
The next step for the team will be to evaluate how the model can be used to help individuals suffering from dyslexia and low language proficiency.
'We want to help users in real-time situations, for example, by designing text that helps drivers without distracting them,' says Oulasvirta. 'Now we have this new understanding of something that's so central to our lives, it's just a matter of exploring all the possibilities.'
In addition to Aalto University, the study involved researchers from The Hong Kong University of Science and Technology, City University of Hong Kong, and the National University of Singapore.
