As the world is increasingly flooded with deepfakes, the European Union is taking action to help people distinguish what is real and what is not.
Authors
- Rory Mulcahy
Associate Professor of Marketing, University of the Sunshine Coast
- Aimee Riedel
Senior Lecturer in Marketing, Griffith University
- Lucas Whittaker
Lecturer in Marketing, Swinburne University of Technology
- Renee Barnes
Associate Professor of Journalism, University of the Sunshine Coast
It recently started enforcing transparency obligations for tech companies which require them to clearly label any photos, videos and audio which are created using their artificial intelligence (AI) systems and designed to look authentic.
This is a step up from the approach of other countries, such as Australia, which only recommend AI developers mark AI-generated content to help people identify it. This matters because research suggests that when confronted with sophisticated AI-generated content, our ability to distinguish fake from real can be little better than a coin-toss.
But will the EU's approach actually work? And could requiring the labelling of AI-generated content have unintended consequences? Could it potentially make us more vulnerable to the very content designed to deceive us?
A set of icons
Article 50 of the EU AI Act took effect from August 2. It introduces various transparency obligations for companies who provide or otherwise deploy AI systems or content.
Among these obligations, AI-generated or manipulated images, audio, or videos which resemble existing persons, objects, places, entities or events must now be disclosed as such.
AI-generated or manipulated text on matters of public interest, which is not subject to human review or editorial control, must also be disclosed.
The EU has developed a set of icons which authentic-looking AI content must be labelled with.
For audio deepfakes, a short audible disclaimer should be placed at the commencement of the content.
Icons should be clearly visible when users first encounter the content. They should also be placed without obstructing overlays, and embedded into the content so it remains visible when shared or downloaded. Using these EU icons is optional, however disclosure of such content in accordance with the act remains mandatory for companies.
Interestingly, for evidently artistic, creative, satirical, or fictional AI-generated content, disclosure is only required in a way that informs audiences without disrupting the presentation or enjoyment of the work.
The EU's new transparency obligations also require AI companies to include invisible, machine-readable watermarks on any content that is generated or even processed with their systems. To comply with the act, leading AI lab Anthropic said it will ensure all Claude chatbot models launched in the EU after August 2 will support machine-readable marking. The company is also working to add watermarks to earlier Claude models.
The act applies to companies even outside the EU who create AI-generated content visible to those within the EU. Non-compliant companies may face fines up to €15 million (A$24.5 million), or up to 3% of total worldwide annual turnover.
However, the act does not apply to individuals when they are creating content in a personal or non-professional capacity. Therefore, the act has limitations in enforcing the disclosure of some AI-generated content.
Labels may create unintended consequences
Even when labels are applied as intended, we need to understand how people will respond.
AI labels are designed to help us identify synthetic content. But they could have an unintended " boomerang effect ". This is where an intervention produces the opposite of what was intended.
If we become accustomed to seeing AI-generated content with a label, we may start assuming that content without a label must be real. This is problematic because malicious actors who deliberately create deceptive content are unlikely to disclose AI use.
Government efforts to protect citizens through disclosure might in turn be encouraging less critical evaluation of wider digital content. Reliance on labels may therefore make citizens increasingly susceptible to undisclosed and malicious AI use.
Our research has shown that disclosure is more complicated than simply adding an "AI-generated" label.
The presence and timing of AI disclosure can influence how people respond to the content. This means the effectiveness of disclosure depends not only on the presence of a label, but also on how people interpret and use it.
Importantly, labels are only one of many cues people use when deciding whether to trust online content. Our research has also shown that when content begins to go viral or is accompanied by comments that reinforce its message, people can become more susceptible to misinformation.
These social cues may compete with, or even override, information indicating whether content is authentic or AI-generated.
Literacy, not just labels
People cannot solely rely on labelling to help them identify AI-generated content. They also need to be alert to AI cues such as inconsistencies in facial movements, expressions, speech or voice.
However, as AI becomes more sophisticated, these cues are becoming harder and harder to spot. So consideration must also be given to who created or shared the content, whether the source is credible, whether the content makes sense, and whether it can be verified elsewhere.
Cultivating AI literacy such as this is vital.
AI labels can help. But they must not be a replacement for the critical evaluation of online content. As the boundary between synthetic and authentic content becomes increasingly blurred, governments and tech companies must do more to support AI literacy efforts to help people make informed judgements about what they see online.
Rory Mulcahy is the recipient of an Office of National Intelligence funded - National Intelligence Discovery Grant funded by the Australian Government.
Aimee Riedel is the recipient of an Office of National Intelligence funded - National Intelligence Discovery Grant funded by the Australian Government.
Lucas Whittaker is the recipient of an Office of National Intelligence funded - National Intelligence Discovery Grant funded by the Australian Government.
Renee Barnes is the recipient of an Office of National Intelligence funded - National Intelligence Discovery Grant funded by the Australian Government.
