Artificial intelligence (AI) is profoundly reshaping education worldwide. While AI tools increasingly support students in brainstorming, drafting and problem-solving, assessment practices often remain narrowly focused on final outputs. This raises a critical question: how can educators truly understand whether students are learning, rather than simply producing polished answers with AI assistance?
A new international study led by The Education University of Hong Kong (EdUHK) provides fresh insights into this challenge. The research reveals that although students frequently engage in idea sharing and social interaction, they devote significantly less time to deeper stages, such as integrating ideas and developing final solutions. The findings underscore the importance of assessing the learning process, not just the product, to cultivate judgment, creativity, and genuine understanding in the AI era.
The study was led by Dr Shen Ba, Assistant Professor in the Department of Curriculum and Instruction at EdUHK, in collaboration with researchers from Monash University, the University of Wisconsin–Madison, and Northwest Normal University. It was recently published in Computers & Education.
To address gaps in existing research, Dr Ba and his team developed Movement Analysis (MOVA), a novel method that traces how learners progress through stages of inquiry, collaboration and problem-solving over time.
Unlike traditional assessment, which often focuses on the final essay, report or presentation, MOVA captures the dynamic journey of learning, providing evidence of how students think, collaborate, respond to feedback, and develop ideas over time. It helps researchers observe how engaged they are during group learning. By tracking "movements" between inquiry stages, MOVA reveals whether groups are actively exploring and refining ideas or remaining in static discussion patterns.
Drawing on online discussion data from a university course involving 108 students in 16 groups, with a total of 1,617 messages, MOVA identified distinct inquiry states such as questioning, exploration, social connection, integration and resolution.
The study examined the integration of AI chatbots powered by OpenAI's GPT‑4 model:
- Groups using the chatbot demonstrated richer, more dynamic interactions.
- They showed more frequent transitions between inquiry states, greater engagement in higher-order thinking, and enhanced ability to move beyond idea generation to deeper integration and application.
- In contrast, groups without the chatbot tended to remain in static discussion states, focusing on social cohesion or basic exploration without advancing toward integration or resolution.
MOVA also highlighted variations in group engagement. Some groups showed longer, more dynamic movements, reflecting sustained activity and deeper thinking, while others remained confined to basic exploration. This cyclical, non-linear view of inquiry pinpointed bottlenecks and revealed where groups struggled to move from idea generation to higher-order thinking.
The study suggests that assessment should pay greater attention to process, not just product. By tracing students' movements through inquiry, MOVA provides evidence of how they think, collaborate, respond to feedback and develop ideas over time. This visibility can help teachers offer timely support and guide students toward becoming reflective, critical and collaborative learners.
Dr Ba noted: "As AI changes how students learn and complete academic work, assessment also needs to change. We need to understand the process behind the product. MOVA provides a way to examine how students learn with peers, teachers and AI."
The study contributes to the fields of learning analytics, collaborative learning and AI-supported education. It marks a step toward rethinking assessment in the AI era—moving beyond the question "What did students produce?" to also asking "How did students learn?"
This research provides concrete evidence that AI, such as chatbots, can play a significant role in enhancing collaborative inquiry and group learning in education. By supporting cognitive, social, and instructional processes, AI tools can help students engage more deeply, navigate complex phases of inquiry, and achieve better learning outcomes.
