As artificial intelligence becomes part of university life, UNSW Sydney researchers are rethinking assessment to protect something technology can't replace: human judgement.
Academics from UNSW's Faculty of Arts, Design & Architecture are redesigning assessment to ensure students continue doing the intellectual work that drives learning.
Dr John Carr, Dr Tania Leimbach and Professor Tema Milstein say universities must prepare students to use AI responsibly while ensuring they still develop the ability to think critically, question assumptions and make ethical decisions.
In a new article published in Humanities and Social Sciences Communications, the researchers warn that the use of generative AI can allow students to bypass work that is integral to learning, including discussing and testing ideas, weighing arguments, revising their thinking and forming their own judgements.
Their argument is not that AI should be removed from education. Instead, educators need to rethink assessment to continue to support the development of "human intelligence" in the face of AI's growing influence.
"If AI undermines students' ability to undergo the sometimes challenging process of learning, then those students and - at a broader level - society loses fundamental capacities to engage in complex analysis, engage in judgement and tie decisions to a robust vision of what a thriving society could be like," says lead author Dr Carr.
The researchers teach UNSW's Master of Environmental Management (MEM) and undergraduate Environmental Humanities, where they have developed assessments designed to strengthen four key human capabilities: the abilities to Discern, Engage, Evaluate and Produce - or DEEP skills.
"These skills help students assess the quality of information and evidence, engage with other people and their ideas, evaluate complex situations and ethical consequences, and produce original, rigorous and persuasive work," explains Dr Leimbach.
The process is learning
The researchers say the growing use of generative AI is exposing a misconception about what learning is.
"University learning is not simply a final essay, presentation or other polished product. The process of producing that work - including reading widely, discussing ideas, questioning evidence and making connections - is where much of the learning takes place. The process is the learning," says Dr Carr.
"Just as the body loses muscle and bone density when it is not placed under load, people can lose the capacity for complex thinking and decision-making if they no longer practise those skills.
"University is a little bit like a gym for our minds. It's a gym for ethical judgement. It's a gym for decision making. It's a gym for envisioning futures that we want."
If students outsource the difficult parts of learning to AI, they may submit a coherent answer without developing the ability to judge whether it is accurate, ethical or appropriate, say the researchers.
"When students substitute AI-generated responses for their own thinking, they may forgo iterative practices of reflection, synthesis and ethical deliberation," the researchers argue.
"The issue is particularly important in fields such as environmental management, planning, government, policy and advocacy, where graduates will need to make decisions involving uncertainty, competing interests and consequences for people and the environment," says Dr Leimbach.
AI cannot make judgements for us
AI tools can process large amounts of information and generate plausible responses. But, the researchers say, AI cannot be counted on to decide what a good or responsible outcome should be.
"The reason AI can't substitute for these judgements, this discernment, is that it has no ethical framework," Dr Carr says.
AI does not have an inherent vision of what a good life, society or planet should look like. Those judgements remain our responsibilities.
"AI just isn't built for that. It's built for predicting the next word in a series of words," Dr Carr says.
Dr Leimbach says universities also cultivate curiosity and the capacity to keep questioning what appears to be established knowledge.
"In the sciences, uncertainty is so foundational," she says. "The capacity to continue to question is something that we teach at universities. We do not want to give that away."
The researchers describe the challenge as a double bind. Students need to develop the skills to work in a future shaped by AI, but relying on AI to complete their assessments may prevent them from developing those skills in the first place.
Rethinking university assessment
The researchers suggest assessment should give students more opportunities to personally demonstrate their thinking, rather than relying on unsupervised written work.
One approach is to use more in-person learning and assessment, including fieldwork, supervised exams, in-class writing, workshopped projects and collaborative activities, such as exhibitions or events.
The MEM has three field courses, including a course that takes students to Yuin Country on the South Coast to learn from Elders on Country. These experiences give students the opportunity to learn through direct engagement with place, people and context.
"Field-based learning and the embodiment that comes with being off campus and out in the world creates opportunities for students to engage directly with complex real-world situations, apply their judgement and reflect on their own lived experiences," Dr Leimbach says.
Oral assessment can provide an additional way for students to explain and apply their ideas. In the MEM, students may be asked to defend a written assessment in a conversation with the course convenor, answering questions and responding in real time.
Prof. Milstein says oral assessment can be challenging to implement, particularly in large classes, but it creates an opportunity for students to take ownership of their work, demonstrate their understanding, and dig deeper in one-on-one conversation with their educator.
The researchers also recommend connecting assessment more closely to students' experiences and real-world concerns, in what they describe as an "Inside-Out Classroom" approach.
This approach helps students link course concepts with their personal concerns, experiences and challenges and apply their learning as positive changemakers in the wider world.
"Inside-Out pedagogy adds engagement to the learning along with making it more difficult to rely on a generic response as students are encouraged to reflect and act on how ideas relate to their own lives and future work," says Prof. Milstein.
Finally, the researchers say assessment should place greater value on how students develop their ideas, not only on the final product they submit.
Discussion, reflection, questioning, collaboration and revision can help students experience and demonstrate how their thinking evolves.
Together, these approaches are designed to strengthen the four DEEP capabilities: discerning information, engaging with different perspectives, evaluating complexity and ethical consequences, and producing original and rigorous work.
Preparing students for an AI-shaped future
The researchers say AI should be treated as a tool that supports human capabilities, not as a substitute for developing them.
"The human has to be there," Dr Carr says. "Whether it's a supervised exam, whether it's by creating an artefact to illustrate the concepts they're dealing with, whether it's discussion in the field, whether it's having a student talk about their personal attachment to concepts, about their personal life stories."
Dr Leimbach says AI can make it tempting to priortise efficiency over the slower process of developing understanding, judgement and expertise.
"Learning is not always easy. It's a struggle sometimes," she says.
For Dr Carr, this is what makes the university experience distinctive. Universities give students time and space to develop human abilities that are difficult to cultivate amid the pressures of everyday professional life.
"The university is unique in that it really is about giving people that capacity and time and focus to develop their human abilities," he says.
As AI reshapes education and employment, the researchers say each discipline will need to decide what students must still be able to learn and do for themselves.
"The question is not simply what AI can produce, but whether students have developed the judgement to understand, question and use its outputs responsibly," says Dr Carr.
