UNIVERSITY PARK, Pa. — A coaching tool built into artificial intelligence (AI)-powered systems may raise user awareness of bias in AI algorithms and help individuals better prompt generative AI tools to produce more inclusive content, according to researchers at Penn State and Oregon State University.
The researchers developed a new text-to-image generative AI application intended to provide immediate media literacy interventions — methods designed to make users pause and reflect on the inclusiveness of their prompt design before image generation. As users enter prompts into the application, the "inclusive prompt coaching" tool issues warnings about biases in generative AI systems and offers suggestions for making their prompts more inclusive. The team presented their research today (April 16) at the 2026 Association of Computing Machinery Computer-Human Interaction Conference on Human Factors in Computing Systems in Barcelona, Spain. The paper received an honorable mention from the conference's awards committee.
In the study, the researchers found that the inclusive prompt coaching intervention increased users' awareness of algorithmic bias, or its tendency to produce stereotypical content. It also boosted their confidence in writing inclusive prompts to produce less biased outputs. The intervention also increased users' perceived trust calibration, or their capability to adjust their trust levels to better reflect the systems' actual trustworthiness. But the intervention led to a less satisfactory user experience, according to the researchers.
"Oftentimes, media literacy interventions like those for social media occur outside of the medium, informing or warning users about the dangers of social media before or after they've interacted with it," said study co-author S. Shyam Sundar , Evan Pugh University Professor and the James P. Jimirro Professor of Media Effects at Penn State. "Here we are using the medium itself — AI text-to-image generators — to educate users about how to better use the medium while they're interacting with it. It's a newer twist on the media literacy approach to address the problem of lack of inclusiveness in generative AI."
To see if prompt coaching can serve as an effective media literacy intervention, the researchers recruited 344 study participants from an online survey platform. They randomly assigned the participants to one of three study conditions: an inclusive prompt coaching condition; a detailed prompt coaching condition; and no coaching condition. The latter two served as control conditions. The researchers asked participants to use the system to generate an image of any character and then answer questions about their experience using the AI system, such as how much control they felt they had over the tool, their awareness of algorithmic bias and their confidence in their ability to craft effective prompts.
Participants in the inclusive prompt coaching condition received feedback on their prompts as soon as they wrote them. If a participant asked the tool to generate an image of beautiful girls in the forest, it would draw their attention to potential bias by explaining that the prompt reinforces the bias that female beauty is primarily defined by physical appearance, running the risk of objectifying the characters. It would then suggest a more inclusive wording, such as "enchanting individuals in a forest."
Those who went through this intervention reported higher awareness of algorithmic bias compared to those in the no coaching condition. They also reported a higher perception of being able to craft effective prompts compared to those in the other two conditions. Yet participants in the inclusive and detailed prompt coaching conditions reported a more frustrating user experience compared to those in the no coaching condition.
"We found a positive effect of this new approach on improving peoples' awareness of algorithmic bias and increasing their confidence in creating effective prompts to reduce bias in AI images," said first author Cheng "Chris" Chen, assistant professor of emerging media and technology at Oregon State University who completed her doctorate with Sundar at Penn State. "The downside of the current version is that participants perceived it as less helpful and more frustrating compared to the control conditions, but we can address this in future design iterations."
Participant feedback suggested that there was resentment among users that the AI system was giving them "a slap on the wrist" for not being inclusive, or that it was identifying potential biases in prompts but then generating images with biased components, the researchers explained. They pointed to one example where the system issued a warning and offered a suggestion for an innocent prompt asking for an image of "a cute toad."
"To address these complaints, we can make the system more context aware and more specifically tailor it to user prompts, because some prompts may be more innocent than others," Chen said. "More tailored interventions may be able to reduce negative perceptions regarding the user experience, reduce frustrations with the design and improve perceived helpfulness."
Giving users the option of toggling the system on and off could also address the user experience issues, added Sundar, who is also the director of the Penn State Center for Socially Responsible Artificial Intelligence ( CSRAI ).
"When you're asking an AI system to generate an image of a toad, the system should not bother trying to automatically correct your lack of inclusiveness," he said. "But when you're dealing with a topic much more in the world of human affairs, the system should realize that you might need help, and that you might appreciate assistance with regard to prompt coaching for inclusiveness."
The prompt coaching approach could help technology companies make their AI tools more ethical and responsible, which could promote appropriate trust among their users, Chen said.
"For everyday users, the inclusive prompt coaching intervention could provide a moment to pause and reflect on how inclusive their prompt is to elicit the best output from AI," she said. "We found that the increased thinking, or elaboration, in users' prompt design led to greater trust and improved perceptions of trust calibration."
In addition to Sundar and Chen, other study co-authors were Mengqi Liao, assistant professor at the University of Georgia who received her doctorate from Penn State; Penn State master's students Aditya Anand Phadnis and Yao Li; Andrew High, professor of communication arts and sciences at Penn State; and Saeed Abdullah, associate professor of information sciences and technology at Penn State.
