New method aims to keep kids safe from illegal AI-generated content

Source Domain: news.mit.edu

The rapid growth of generative artificial intelligence has led to the development and distribution of numerous open-source models that can be adapted for various tasks, including generating product renderings in specific artistic styles.
Unfortunately, these same models can also be misused to produce illegal content such as hate speech and child sexual abuse material (CSAM), which has become a significant issue.
In 2025, the National Center for Missing and Exploited Children received over 1.5 million reports of AI-generated CSAM, a sharp increase from 67,000 in 2024, exemplifying the growing problem.
Traditional AI safety testing methods that rely on inspecting harmful outputs can’t be applied to CSAM due to its illegality for any purpose.
To address this, a team from MIT, led by graduate student Vinith Suriyakumar, in collaboration with researchers from Thorn, a child safety nonprofit, developed a new auditing approach.
Their method examines the internal modifications made by the fine-tuning process without generating or inspecting any illegal outputs.
They used a technique called Gaussian probing to analyze how the model’s internal layers manipulate random data points, inferring whether it can produce CSAM based on these manipulations.
Their approach was 100 percent accurate in identifying model variations adapted to generate CSAM, offering a scalable solution to detect and prevent the harmful use of open-source AI generative models.

New method aims to keep kids safe from illegal AI-generated content | MIT News