Dutch court orders X and Grok to halt AI-generated sexual abuse content
A court in the Netherlands has ordered X and its AI system Grok to stop generating non-consensual sexualised content, marking a first binding ruling against an AI image generator in Europe.
A Dutch court has ordered X Corp and its AI system Grok, developed by xAI, to stop generating non-consensual sexualised images and child sexual abuse material in the Netherlands.
The ruling was issued by the Amsterdam District Court and includes a penalty of €100,000 per day for each company if they fail to comply.
The case was brought by Offlimits, an organisation focused on online sexual abuse, and Fonds Slachtofferhulp, a victim support group. They argued that Grok’s image generation tools allowed users to create explicit images of individuals without consent, including so-called “undressing” images generated from uploaded photos.
The court found that the companies had not sufficiently prevented this type of content from being generated. Evidence presented during the case showed that such images could still be produced, despite claims by the companies that safeguards had been implemented.
As a result, the court ordered xAI to stop generating and distributing non-consensual sexualised images of people in the Netherlands and to prevent any output that qualifies as child sexual abuse material under Dutch law. X’s EU-based entity was also ordered to stop offering Grok’s functionality on the platform while these violations persist.
The judgment also clarified responsibility. The court rejected the argument that users alone are responsible for harmful outputs. Instead, it found that the platform operators, as designers and controllers of the system, are responsible for preventing illegal content.
The decision is based on existing legal frameworks. The court held that generating non-consensual intimate images violates EU data protection rules, while facilitating child sexual abuse material constitutes unlawful conduct under Dutch civil law.
The ruling is considered the first in Europe to impose a binding injunction on an AI image generator for this type of content. It comes alongside ongoing investigations and regulatory actions against X and Grok by authorities in the EU, Ireland, and the United Kingdom.
