Online platforms often fail to act on reports of non-consensual intimate images submitted through safety or abuse systems-but remove the same material far more quickly when it is framed as a copyright violation, according to new University of Michigan research.
Beginning May 19, online platforms will be legally required under the TAKE IT DOWN Act to establish systems for removing non-consensual intimate imagery, including AI-generated deepfakes, within 48 hours of a valid report. TAKE IT DOWN is enforced by the FTC, and a lack of compliance could result in civil penalties of $53,088 per violation.
Why it matters: The research is important due to the rapid rise of AI-generated intimate imagery and the need for safer methods to study how platforms respond to abuse.
The new study, co-authored by School of Information researchers, finds that X (formerly Twitter) removed AI-generated nude images far more quickly when reports were filed under copyright law than under the platform's own non-consensual nudity policy.
To conduct the audit, researchers uploaded 50 AI-generated nude images to X and reported half through the platform's "non-consensual nudity" reporting mechanism and the other half through copyright infringement claims under the Digital Millennium Copyright Act (DMCA).
All 25 images reported through copyright claims were removed within 25 hours. None of the images reported through X's internal non-consensual nudity reporting system were removed during the three-week study period.
"The contrast is stark," said Qiwei Li, UMSI doctoral candidate and study's lead author. "Reports filed through a law got fast action, and reports through the internal channel got nothing. X responds to legal pressure, not to their own stated policies. Legal pressure works; platform goodwill doesn't."
Rather than using real human content, the researchers generated realistic synthetic nude images using AI tools and reported the content while posing as the depicted, but fictional, individuals.
"The idea came from watching AI-generated nude imagery enter mainstream public consciousness and recognizing it could let us study a real, urgent problem without risking survivor content," Li said. "Our audit suggests that platforms will remove when there's legal pressure behind a report. That's the part of TAKE IT DOWN that could work. The more challenging question is what else might come down along with it. A law that mandates fast removals on the internet risks creating collateral damage."
Li also points to a problem that the law leaves untouched entirely: the burden it places on survivors doing the reporting.
"The law still leaves survivors doing the work, searching for their own abusive content, documenting it, submitting reports, monitoring after submission and doing it all again when the content reappears on a new website," she said.
Li has studied survivor experiences with reporting non-consensual intimate imagery online in an interview study published at this year's Association of Computing Machinery Conference on Human Factors in Computing Systems.
"TAKE IT DOWN might better address the platform-side failure our audit documented. But it does not fix the survivor-side burden, and that gap is where policy attention needs to go next,"Li said. TAKE IT DOWN stands for Tools to Address Known Exploitation by Immobilizing Technological Deepfakes on Websites and Networks Act.
