- What: Dutch foundations have filed a lawsuit against X (formerly Twitter) over "nudify" tools in its Grok AI.
- The Penalty: Plaintiffs are seeking a recurring fine of €100,000 ($108,000) for every day the features remain available.
- Legal Defense: Lawyers for Grok stated in court that it is "impossible to 100% prevent abuse" of the AI tool.
- Key Violations: The case alleges breaches of the GDPR, the Digital Services Act (DSA), and the Dutch Criminal Code regarding child sexual abuse material (CSAM).
Lawyers representing X and its AI chatbot, Grok, have admitted in an Amsterdam District Court that the company cannot fully prevent users from abusing the platform's "nudify" capabilities. The admission comes as part of a high-stakes legal challenge brought by Dutch foundations, including Offlimits, which are seeking to ban the features and impose a staggering €100,000 daily penalty on the social media giant.
The lawsuit, which centers on Grok’s ability to generate non-consensual sexual imagery and "undress" subjects, marks a critical turning point in how European regulators and courts view AI-generated content. According to reports from the NL Times and DutchNews.nl, the plaintiffs argue that the tool’s current version has allowed for an increase in extremist content and the creation of deepfake pornography, including imagery involving minors.
A "Zero Percent" Success Rate for Total Safety
During the proceedings, legal counsel for Grok reportedly pivoted to a defense based on the technical limitations of AI moderation. By stating that it is "impossible to 100% prevent abuse," the defense team is challenging the legal expectation that tech platforms must provide a perfectly safe environment before deploying powerful generative models.
However, Dutch law presents a formidable barrier to this defense. Under the Dutch Criminal Code, the law does not differentiate between sexual abuse images of real children and those created through artificial intelligence. This means that if Grok can be manipulated into creating CSAM, X could be held liable under the same statutes governing the distribution of physical child abuse material.
"Now it’s a matter of enforcing the law," said Hoving, a representative involved in the case, as reported by the NL Times. The foundations argue that the "nudify" features are not just a byproduct of a neutral tool but a direct violation of safety standards that the company has failed to adequately mitigate.
Massive Financial and Regulatory Stakes
The plaintiffs, supported by Boekx Lawyers, are not merely seeking a symbolic victory. They have requested that the court impose a penalty of €100,000 for every day that Grok and X fail to remove the nudity options. This financial pressure is paired with a comprehensive list of alleged legal violations, including:
- The General Data Protection Regulation (GDPR): Regarding the unauthorized use and manipulation of personal "portrait rights" and biometric data.
- The Digital Services Act (DSA): A landmark European law designed to make the internet more transparent and hold platforms accountable for systemic risks.
- The Dutch Criminal Code: Specifically addressing the creation and dissemination of illegal sexual content.
- Civil Law Standards: Pertaining to the right to privacy and protection against reputational harm.
Otto Volgenant, a legal expert cited in the proceedings, noted that the case targets the systemic failure of the platform to protect users' portrait rights—the legal right of an individual to control how their likeness is used.
Impact on the AI Industry and Developers
This case serves as a warning shot to the entire AI industry, particularly companies pursuing "unfiltered" or "anti-woke" AI models. For developers, the Dutch court's decision could set a precedent that technical impossibility is not a valid legal defense for deploying tools that can generate harmful content.
If the Amsterdam District Court rules against X, it would signal that AI companies must prioritize safety and "guardrails" over the raw capabilities of their models, or face debilitating fines that could bankrupt smaller startups and severely dent the revenue of major corporations.
"For the first time, the industry is seeing a court challenge the 'neutral tool' defense, potentially forcing AI developers to prove 100% compliance with safety laws before a single user can prompt the model."
For users and victims of deepfake technology, the case represents a significant push for accountability. The "nudify" trend has plagued social media platforms for years, but this is one of the most direct legal attempts to hold the model creator—rather than just the individual user—responsible for the output.
What’s Next
The case is scheduled to be heard in the Amsterdam District Court on Thursday, March 12, 2026. The legal community and AI safety advocates are watching closely, as a victory for the foundations could trigger a wave of similar litigation across the European Union under the Digital Services Act.
If the court grants the requested ban and daily penalty, X will be forced to choose between a total shutdown of Grok's image-generation features in the region or facing a mounting financial burden that could exceed €36 million annually. The decision will likely define the boundaries of "acceptable risk" for generative AI in the Western world.
Sources
- NL Times: Impossible to 100% prevent abuse, Grok lawyers say in Dutch case against nudify tools
- DutchNews.nl: Dutch groups take X and Grok to court over child nudity tool
- Cybernews: Dutch sue X and Grok over AI “undressing” tool, seek €100K daily penalty
- Offlimits: Offlimits launches legal action against X’s AI chatbot Grok