AI Gives Predators New Ways to Target Children, UN Warns

The United Nations

Artificial intelligence (AI) is handing predators and criminal groups new ways to groom, extort and exploit children, while efforts to protect them are failing to keep pace, senior UN officials warned on Thursday.

"We lost the battle on social media because we were lagging behind, and we are now losing the battle on artificial intelligence because we are still lagging behind," said Dr. Najat Maalla M'jid , the UN Special Representative of the Secretary-General on Violence against Children.

She was speaking during a high-level event in the General Assembly on the risks and opportunities of AI to help end the scourge.

In the past year alone, at least 1.2 million children in 11 countries had their images manipulated into sexually explicit deepfakes, UN Deputy Secretary-General Amina Mohammed told the event at UN Headquarters in New York, citing a recent study by the UN Children's Fund ( UNICEF ).

Reports of child sexual abuse material linked to generative AI rose by 1,325 per cent between 2023 and 2024, according to the US National Center for Missing and Exploited Children, said Dr. M'jid, noting that most AI-facilitated crimes against children remain unreported and under prosecuted.

'Weaponising technology'

Chatbots are being used to gain children's trust, gather personal information and coerce them into sharing explicit images, she said, while deepfakes made with "nudification" apps are used to humiliate, threaten and exploit them.

Profiling and predictive targeting allow predators, traffickers and criminal groups to reach vulnerable children more easily.

John Trenary who has worked on child exploitation investigations for more than 25 years, said predators have always sought ways of "weaponizing technology to exploit our kids".

But because AI can scale so efficiently - automate and create wholly synthetic content - its weaponisation is "self-perpetuating", he warned. As soon as investigators get a handle on one way predators misuse it, "something else comes out".

Kay Chau of the Tech Coalition - a non-profit alliance focused on child protection - said perpetrators may be using chatbots to facilitate abusive role-play or generate grooming scripts, while agentic AI (systems that can carry out tasks independently) is introducing new risks.

'Lucia is our agent'

Mariana Rozo-Paz of the Datasphere Initiative recalled working with teenage girls on a digital literacy programme in rural Colombia last year. Asked to draw what the internet looked like, the girls drew a girl named Lucia - their AI agent.

"We talk to Lucia every day," they told her. "Whenever adults don't want to answer specific questions, we ask Lucia. She's always there for us. She has intimate mode, best friend mode."

'Proven safe'

"We would never accept a children's toy being put on the market without proper safety checks, but AI systems are reaching children without the same scrutiny," Ms. Mohammed said. "We cannot leave their safety to the goodwill of technology companies."

In July, Secretary-General António Guterres launched the AI Child Safety Pledge. Its aims include making AI systems certifiably safe before they reach children, and zero tolerance of AI-generated child sexual abuse material.

Voluntary approaches by the tech sector are insufficient, Dr. M'jid said, calling for global standards at the core of AI governance, including mandatory child safety and privacy by design.

AI can also help detect abuse material, identify victims and spot early signs of grooming, added the Deputy Secretary-General.

In a World Childhood Foundation study, a young person in Sweden put it this way: "The technology is really cool. But you can tell that the people in charge of controlling it cannot keep up."

"Children should never have to tell us that the adults responsible for their safety cannot keep up," said the foundation's Secretary General, Paula Guillet de Monthoux.

"It is our responsibility to prove that we can."

/UN News Release. This material from the originating organization/author(s) might be of the point-in-time nature, and edited for clarity, style and length. Mirage.News does not take institutional positions or sides, and all views, positions, and conclusions expressed herein are solely those of the author(s).View in full here.