New method aims to keep kids safe from illegal AI-generated content
Researchers developed an auditing technique to test generative AI models for malicious capabilities, without prompting them for illegal outputs. Researchers developed an evaluation procedure that tests generative AI models for harmful capabilities without generating outputs. This could enable auditors to identify open-source models that have been adapted to produce illegal content, like child sexual abuse material (CSAM).
Key Takeaways
- Adam Zewe | MIT News Publication Date : July 13, 2026 Press Inquiries Press Contact : Abby Abazorius Email: abbya@mit.edu MIT News Office Media Download ↓ Download Image Caption : Researchers developed an evaluation procedure that tests generative AI models for harmful capabilities without generating outputs.
This could enable auditors to identify open-source models that have been adapted to produce illegal content.
- A credit line must be used when reproducing images; if one is not provided below, credit the images to "MIT."
- Engineers usually test AI for harmful capabilities by prompting the model and inspecting its outputs, but this is impossible for CSAM, since it is illegal in the U.S to generate such content, regardless of intent.
- When tested, the auditing procedure identified model variations that had been specialized to generate CSAM with 100 percent accuracy.
A hosting platform could use this technique to flag unsafe models and quickly remove them or prevent them from being uploaded in the first place.
- Now, we can address an AI safety problem that is having severe negative impacts," says Vinith Suriyakumar, an MIT electrical engineering and computer science (EECS) graduate student and lead author of a paper on this technique.
Stats & Key Facts
- #Adam Zewe | MIT News Publication Date : July 13, 2026 Press Inquiries Press Contact : Abby Abazorius Email: abbya@mit.edu MIT News Office Media Download ↓ Download Image Caption : Researchers developed an evaluation procedure that tests generative AI models for harmful capabilities without generating outputs.
- #This is a growing problem - the National Center for Missing and Exploited Children received more than 1.5 million reports of AI-generated CSAM in 2025, an increase from 67,000 in 2024.
- #When tested, the auditing procedure identified model variations that had been specialized to generate CSAM with 100 percent accuracy.
Researchers developed an evaluation procedure that tests generative AI models for harmful capabilities without generating outputs. This could enable auditors to identify open-source models that have been adapted to produce illegal content, like child sexual abuse material (CSAM). Researchers developed an auditing technique to test generative AI models for malicious capabilities, without prompting them for illegal outputs.
Adam Zewe | MIT News Publication Date : July 13, 2026 Press Inquiries Press Contact : Abby Abazorius Email: abbya@mit.edu MIT News Office Media Download ↓ Download Image Caption : Researchers developed an evaluation procedure that tests generative AI models for harmful capabilities without generating outputs. This could enable auditors to identify open-source models that have been adapted to produce illegal content. Credits : Credit: Christine Daniloff, MIT; iStock *Terms of Use: Images for download on the MIT News office website are made available to non-commercial entities, press and the general public under a Creative Commons Attribution Non-Commercial No Derivatives license .
You may not alter the images provided, other than to crop them to size. A credit line must be used when reproducing images; if one is not provided below, credit the images to "MIT." : Researchers developed an evaluation procedure that tests generative AI models for harmful capabilities without generating outputs.
For more details please read the original article at MIT News AI.
Continue Learning
Comments
Comments appear only after moderation. Your email identifies your submission to the moderator and is never displayed here.
No approved comments yet.