New method aims to keep kids safe from illegal AI-generated content
Researchers developed an auditing technique to test generative AI models for malicious capabilities, without prompting them for illegal outputs. Researchers developed an evaluation procedure that tests generative AI models for harmful capabilities without generating outputs. This could enable auditors to identify open-source models that have been adapted to produce illegal content, like child sexual abuse material (CSAM).
Key Takeaways
- Adam Zewe | MIT News Publication Date : July 13, 2026 Press Inquiries Press Contact : Abby Abazorius Email: abbya@mit.
edu MIT News Office Media Download ↓ Download Image Caption : Researchers developed an evaluation procedure that tests generative AI models for harmful capabilities without generating outputs.
- A credit line must be used when reproducing images; if one is not provided below, credit the images to "MIT.
This could enable auditors to identify open-source models that have been adapted to produce illegal content.
- Engineers usually test AI for harmful capabilities by prompting the model and inspecting its outputs, but this is impossible for CSAM, since it is illegal in the U.
- By examining hidden representations, it can reliably infer whether a model has been specialized to produce harmful imagery.
When tested, the auditing procedure identified model variations that had been specialized to generate CSAM with 100 percent accuracy.
- Now, we can address an AI safety problem that is having severe negative impacts," says Vinith Suriyakumar, an MIT electrical engineering and computer science (EECS) graduate student and lead author of a paper on this technique.
Stats & Key Facts
- #5 million reports of AI-generated CSAM in 2025, an increase from 67,000 in 2024.
- #When tested, the auditing procedure identified model variations that had been specialized to generate CSAM with 100 percent accuracy.
Researchers developed an evaluation procedure that tests generative AI models for harmful capabilities without generating outputs. This could enable auditors to identify open-source models that have been adapted to produce illegal content, like child sexual abuse material (CSAM). Researchers developed an auditing technique to test generative AI models for malicious capabilities, without prompting them for illegal outputs.
Adam Zewe | MIT News Publication Date : July 13, 2026 Press Inquiries Press Contact : Abby Abazorius Email: abbya@mit. edu MIT News Office Media Download ↓ Download Image Caption : Researchers developed an evaluation procedure that tests generative AI models for harmful capabilities without generating outputs. This could enable auditors to identify open-source models that have been adapted to produce illegal content.
Credits : Credit: Christine Daniloff, MIT; iStock *Terms of Use: Images for download on the MIT News office website are made available to non-commercial entities, press and the general public under a Creative Commons Attribution Non-Commercial No Derivatives license . You may not alter the images provided, other than to crop them to size. A credit line must be used when reproducing images; if one is not provided below, credit the images to "MIT.
For more details please read the original article at MIT News AI.
Continue Learning
Comments
Sign in to join the conversation