AI Toys Are Feeding Children Sexual Content and Dangerous Instructions. Europe Must Act Now.

These are not edge cases or freak one-off outputs. Researchers found that AI guardrails systematically weakened during extended interactions, meaning the longer a child engaged with the toy, the more likely the conversation was to drift into harmful territory. The toys are, by design, built to sustain long and emotionally engaging conversations. That is precisely the problem.

The Corporate Shell Game

OpenAI's own usage policies require customers to "keep minors safe" and prevent exposure to age-inappropriate content. Yet the company simultaneously permits paying customers to integrate GPT-4o into children's toys while maintaining that ChatGPT itself is not intended for children under 13. That contradiction is not an oversight; it is a structural arrangement that grants OpenAI plausible deniability whenever something goes wrong downstream.

After FoloToy's access to OpenAI's API was suspended following public outcry, the company resumed sales within a week, claiming it had completed "rigorous safety audits." Independent researchers found similar problems persisting. The suspension had functioned as a press management exercise, not a genuine safety intervention.

The pattern across the AI toy sector mirrors the broader failure mode in consumer AI: products are rushed to market, minimal safeguards are implemented, and when harm occurs the blame is distributed so thinly across the supply chain that nobody is held meaningfully accountable.

Why the EU and UK Cannot Afford to Watch From the Sidelines

European families are not insulated from this. AI toys built on US-developed large language models are sold freely across the EU and UK, subject to whatever safety measures, or lack thereof, the manufacturer has chosen to implement. The regulatory gap is real and it is being exploited.

Andrea Jelinek, former chair of the European Data Protection Board, has previously highlighted that connected toys capable of recording and processing children's voice data in private spaces such as bedrooms fall squarely within the scope of the General Data Protection Regulation's strictest provisions on children's data. Under Article 8 of the GDPR, processing children's personal data requires verifiable parental consent, yet the AI toys identified in these investigations collect voice recordings, emotional response patterns, and conversation transcripts with mechanisms that critics describe as wholly inadequate.

The EU AI Act, which entered into force on 01/08/2024, classifies AI systems that interact with vulnerable groups, including children, under a high-risk category requiring conformity assessments and transparency obligations before market placement. Whether AI toys will be robustly caught by that framework in practice depends entirely on how the European AI Office enforces it. So far, enforcement guidance on consumer AI products targeted at children has been limited.

Professor Virginia Dignum of Umea University, one of Europe's most cited AI ethics researchers and a former member of the European Commission's High-Level Expert Group on AI, has argued consistently that safety by design must be a precondition for market access in any AI system deployed around children, not an afterthought triggered by investigative journalism. Her position, shared by a growing number of researchers at institutions including the Alan Turing Institute in London, is that voluntary compliance frameworks are structurally incapable of protecting children from harm when the commercial incentive points the other way.

The Developmental Stakes Are Higher Than the Headlines Suggest

Beyond the immediate and distressing content failures, there is a longer-term developmental concern that deserves equal weight. AI chatbots in toys are engineered to validate, agree, and sustain engagement. That is not how healthy human relationships work, and it is not what young children need from the interactions that shape their emotional and cognitive development.

Constant AI-generated validation has been linked by clinical researchers to what is described informally as "AI psychosis," a pattern of increasingly distorted thinking and detachment from reality associated with excessive reliance on chatbot companions. This phenomenon has been connected, in documented cases, to self-harm and, in extreme instances, violence. The idea that these same underlying models are now being marketed as companions for infants and toddlers should prompt serious alarm.

"Combined with extensive data collection and subscription models that exploit emotional bonds, these products are not safe for kids five and under, and pose serious concerns for older children as well," said Robbie Torney, head of AI and digital assessments at Common Sense Media, one of the few organisations systematically testing these products against child safety benchmarks.

Key Safety Failures Identified Across AI Toy Products

• Insufficient content filtering that permits sexual and violent topics to reach children during normal use
• Conversation guardrails that degrade during extended interactions, the precise scenario the toys are designed to encourage
• No meaningful age verification at the point of interaction
• Voice recording and emotional data collection from private domestic spaces without adequate parental controls or transparency
• Subscription models structured to deepen emotional dependency between child users and the AI companion

What Regulators Must Do

The UK's Online Safety Act places duties on platforms hosting user-generated or AI-generated content likely to reach children, but its application to standalone connected toys remains legally ambiguous. The Information Commissioner's Office has issued its Age Appropriate Design Code, which imposes strong obligations on online services directed at children, but again the coverage of AI toys operating partly offline or via proprietary apps sits in a grey zone that manufacturers have been content to exploit.

What is needed is not another consultation or a voluntary industry charter. It is a mandatory pre-market conformity requirement, enforced by the European AI Office and mirrored by the ICO and Ofcom in the UK, that prohibits any AI toy using a general-purpose large language model from reaching retail shelves without demonstrated, independently audited safeguards specific to child users. The audit must include adversarial testing over extended conversations, not just initial prompt filtering.

API providers such as OpenAI must also be held directly liable when their models are deployed in children's products in ways that produce foreseeable harm. Delegating safety entirely to downstream manufacturers while continuing to collect API revenue is not a neutral commercial arrangement; it is complicity structured to avoid legal consequence.

The toys sitting on shelves across Europe right now are not safe. Parents deserve to know that, and regulators need to act as though they know it too.