British Tech Firms and Child Safety Agencies to Examine AI's Ability to Generate Abuse Images

Tech firms and child protection organizations will receive permission to evaluate whether AI systems can generate child abuse images under new UK legislation.

Significant Rise in AI-Generated Harmful Material

The announcement coincided with revelations from a protection watchdog showing that cases of AI-generated CSAM have more than doubled in the past year, rising from 199 in 2024 to 426 in 2025.

Updated Regulatory Framework

Under the changes, the authorities will permit designated AI developers and child protection groups to inspect AI systems – the underlying technology for conversational AI and image generators – and ensure they have adequate protective measures to prevent them from creating images of child exploitation.

"Ultimately about preventing abuse before it happens," declared the minister for AI and online safety, noting: "Specialists, under rigorous protocols, can now detect the danger in AI models promptly."

Addressing Legal Obstacles

The amendments have been implemented because it is illegal to produce and possess CSAM, meaning that AI developers and other parties cannot create such content as part of a testing process. Previously, authorities had to wait until AI-generated CSAM was uploaded online before dealing with it.

This legislation is aimed at averting that issue by enabling to halt the production of those materials at their origin.

Legislative Structure

The changes are being added by the authorities as revisions to the criminal justice legislation, which is also establishing a prohibition on owning, creating or distributing AI models designed to generate exploitative content.

Real-World Impact

This recently, the official toured the London base of Childline and listened to a simulated call to counsellors featuring a account of AI-based abuse. The call portrayed a adolescent seeking help after facing extortion using a explicit AI-generated image of himself, constructed using AI.

"When I hear about young people facing extortion online, it is a cause of extreme frustration in me and justified concern amongst families," he stated.

Alarming Data

A prominent internet monitoring organization reported that cases of AI-generated exploitation content – such as online pages that may include numerous images – had significantly increased so far this year.

Instances of category A material – the most serious form of abuse – rose from 2,621 visual files to 3,086.

• Female children were predominantly targeted, making up 94% of illegal AI images in 2025
• Depictions of infants to toddlers increased from five in 2024 to 92 in 2025

Industry Reaction

The law change could "represent a crucial step to guarantee AI products are safe before they are launched," commented the chief executive of the internet monitoring organization.

"AI tools have made it so victims can be victimised all over again with just a simple actions, giving offenders the capability to make potentially endless amounts of sophisticated, photorealistic exploitative content," she added. "Material which further commodifies survivors' trauma, and makes young people, particularly girls, less safe on and off line."

Support Session Information

Childline also released details of counselling sessions where AI has been referenced. AI-related risks discussed in the sessions include:

• Using AI to rate weight, physique and looks
• Chatbots dissuading children from talking to safe guardians about harm
• Being bullied online with AI-generated material
• Online extortion using AI-faked images

During April and September this year, Childline delivered 367 counselling sessions where AI, chatbots and associated terms were discussed, four times as many as in the equivalent timeframe last year.

Fifty percent of the references of AI in the 2025 interactions were related to mental health and wellness, encompassing utilizing chatbots for assistance and AI therapeutic apps.