Understanding Deepfake Nudes and Other AI-Generated Child Sexual Abuse Material

As technology continues to advance, we’re facing an emerging threat to children’s safety online: artificial intelligence tools being misused to create sexually exploitative content of minors. This disturbing trend requires our immediate attention and action as a community.

A particularly concerning development is the rise of “deepfake nudes” – AI-generated or manipulated images that sexually exploit children. According to recent data from Thorn’s Youth Monitoring report, approximately one in ten minors reported knowing friends or classmates who have used AI tools to generate nude images of other kids. This statistic is not just alarming – it represents a real crisis requiring urgent attention.

AI-generated child sexual abuse material creates real harm

While these images may be artificially created, the harm they can cause is very real. AI-generated child sexual abuse material impacts children in many ways:

• Children who are targeted experience trauma and psychological harm
• Law enforcement face increased challenges in identifying abuse victims
• These images can normalize the sexual exploitation of children
• Predators may use this content for grooming, blackmail, or harassment

Whether an image is completely AI-generated or is a manipulated version of a real photo, the emotional and psychological impact on victims remains devastating.

Taking action to protect children

As we confront this challenge, there are several important steps we can all take to help protect children:

• Educate yourself and others about digital safety
• Have open conversations with children about online risks
• Know how to report suspicious content to appropriate authorities
• Support organizations working to combat online child exploitation
• Stay informed about evolving online risks to children
• For parents and caregivers looking for guidance on discussing this sensitive topic with their children, Thorn has created a comprehensive resource: “Navigating Deepfake Nudes: A Guide to Talking to Your Child About Digital Safety.” This guide provides practical advice and strategies for having these crucial conversations.

It’s important that AI leaders building the technology also do their part to stop the misuse of generative AI to harm children sexually. Child safety can and should be part of their products. Our Safety by Design for Generative AI initiative has led the charge to drive the adoption of principles and mitigations that address this growing problem while we still have the chance. 

Every child deserves to grow up safe from sexual abuse. As generative AI technology continues to evolve, we must work together to protect our children from these new forms of exploitation. By staying informed and taking action, we can help create a safer digital world for all children.