🔗 Share this article

Technology companies and child safety organizations will be granted authority to evaluate whether AI systems can generate child exploitation images under new UK legislation.

Substantial Increase in AI-Generated Illegal Material

The announcement coincided with findings from a safety monitoring body showing that reports of AI-generated CSAM have increased dramatically in the last twelve months, rising from 199 in 2024 to 426 in 2025.

Updated Legal Framework

Under the amendments, the government will allow designated AI companies and child safety groups to examine AI models – the foundational technology for chatbots and image generators – and ensure they have sufficient protective measures to prevent them from creating images of child exploitation.

"Ultimately about preventing exploitation before it happens," declared the minister for AI and online safety, adding: "Experts, under strict conditions, can now detect the risk in AI systems early."

Addressing Regulatory Obstacles

The changes have been introduced because it is illegal to create and own CSAM, meaning that AI developers and other parties cannot generate such images as part of a testing regime. Until now, authorities had to delay action until AI-generated CSAM was uploaded online before addressing it.

This law is designed to averting that problem by helping to stop the creation of those materials at source.

Legal Structure

The changes are being added by the authorities as modifications to the crime and policing bill, which is also establishing a prohibition on owning, creating or distributing AI systems designed to generate child sexual abuse material.

Real-World Consequences

This week, the minister visited the London base of Childline and listened to a mock-up conversation to advisors featuring a report of AI-based abuse. The call portrayed a adolescent seeking help after facing extortion using a sexualised deepfake of himself, constructed using AI.

"When I learn about young people facing blackmail online, it is a cause of intense anger in me and rightful anger amongst parents," he stated.

Concerning Statistics

A leading online safety foundation reported that instances of AI-generated abuse content – such as webpages that may include multiple files – had significantly increased so far this year.

Instances of category A material – the gravest form of abuse – rose from 2,621 visual files to 3,086.

• Female children were overwhelmingly targeted, accounting for 94% of prohibited AI images in 2025
• Portrayals of newborns to toddlers rose from five in 2024 to 92 in 2025

Industry Response

The legislative amendment could "constitute a vital step to ensure AI products are safe before they are released," stated the chief executive of the internet monitoring organization.

"AI tools have enabled so victims can be targeted all over again with just a simple actions, providing criminals the ability to create possibly limitless amounts of advanced, lifelike exploitative content," she continued. "Material which further commodifies survivors' trauma, and renders children, especially girls, less safe both online and offline."

Support Session Information

The children's helpline also published details of support interactions where AI has been referenced. AI-related risks mentioned in the sessions include:

• Using AI to evaluate weight, physique and appearance
• AI assistants dissuading children from consulting safe guardians about harm
• Being bullied online with AI-generated material
• Digital blackmail using AI-faked images

During April and September this year, Childline delivered 367 counselling interactions where AI, chatbots and associated terms were discussed, four times as many as in the same period last year.

Fifty percent of the mentions of AI in the 2025 interactions were related to mental health and wellbeing, encompassing utilizing chatbots for support and AI therapy apps.