Anyone who uses AI to create a realistic nude image of a real person can cause enormous harm. Minnesota therefore does not want to wait until such images are distributed. The state is targeting the technology that enables their creation in the first place. Elon Musk’s AI company xAI says that approach is unconstitutional — and has taken Minnesota to court. The dispute over sexualized deepfakes could become a landmark case over how far AI providers can be held responsible for what users do with their systems.
xAI filed suit on July 27 in the U.S. District Court for the District of Minnesota against Attorney General Keith Ellison. The case, X.AI LLC v. Ellison, challenges a law passed in May that restricts access to so-called “nudification technology.” The law took effect on August 1.
What makes the case unusual is that xAI does not dispute Minnesota’s legitimate interest in combating non-consensual intimate deepfakes. Instead, the company argues that the law is written so broadly that it also restricts lawful uses of generative image AI.
Minnesota Is Targeting the Tool, Not Just the User
That is the key difference between Minnesota’s law and many previous rules on intimate deepfakes.
HF 1606 prohibits operators of websites, apps, software and other services from providing users with access to technology capable of realistically altering images or videos of identifiable people by adding or exposing intimate body parts. Advertising such services is also prohibited.
In other words, the law does not merely target someone who creates or publishes a manipulated image.
It intervenes one step earlier and places responsibility on the provider not to make the relevant capability available in the first place.
Violations can trigger civil penalties of up to $500,000 for each prohibited access, download or use. Victims may also bring their own lawsuits seeking damages, punitive damages, injunctive relief and legal costs.
The law passed Minnesota’s House of Representatives by a vote of 132 to 1 and the state Senate by 65 to 0. Governor Tim Walz signed it on May 7.
The Critical Detail: Consent Is Not Part of the Definition
At first glance, xAI suing over a law designed to prevent sexual deepfakes can easily look like a company defending questionable AI applications.
Legally, the issue is more complicated.
The law’s definition of “nudify” focuses on whether a realistic image of an identifiable person has been modified to add an intimate body part that was not visible in the original image. Whether the person depicted consented is not part of that definition.
That is where xAI’s constitutional argument begins.
The company says the statute could therefore apply to images created with the subject’s consent — potentially even images people generate of themselves. According to xAI, satirical, artistic and other forms of protected expression could also fall within the law’s scope.
Minnesota’s definition of an “intimate part” is also broader than just genitalia. It incorporates language from the state’s criminal law that includes areas such as the groin, inner thigh, buttocks and breast.
xAI argues that this makes the law substantially broader than a conventional prohibition on deepfake pornography.
There is one explicit exception: the restrictions do not apply to tools that require users to employ significant individualized technical or artistic skill.
No Safe Harbor for Companies That Build Safeguards
For the AI industry, another part of the dispute may prove even more important.
According to xAI, the law does not provide an explicit safe harbor for companies that use filters, moderation systems or other safeguards to prevent abuse.
That matters because general-purpose image models have thousands of legitimate uses, while users continuously try to circumvent safety systems with unusual prompts, prompt injection techniques and other workarounds.
xAI points out that its own policies prohibit the publication and distribution of non-consensual intimate content. The company also provides reporting and removal procedures for both real and synthetic intimate material published without consent.
But that leads directly to the political question Minnesota is asking:
Is it enough for an AI provider to prohibit abuse and remove harmful material afterward — or must the company prevent certain images from being generated at all?
That is where two very different models of platform responsibility collide.
xAI Tried to Stop the Law Before It Took Effect
Alongside its lawsuit, xAI sought emergency relief to block the law before August 1.
U.S. District Judge Donovan Frank rejected that request on July 31. His ruling focused initially on timing: the law had been signed almost three months earlier, while xAI did not seek emergency relief until just days before it was scheduled to take effect.
The delay, the judge concluded, weighed against the company’s claim of immediate and irreparable harm.
As a result, the law took effect as scheduled on August 1.
But the rejection of the emergency motion did not resolve the underlying constitutional question. The lawsuit continues, and xAI is seeking a declaration that the statute is unconstitutional as well as a permanent injunction against enforcement.
The U.S. Already Has a Federal Law — But It Works Differently
Minnesota’s approach becomes clearer when compared with the federal TAKE IT DOWN Act, which took effect in 2025.
The federal law criminalizes the intentional publication of non-consensual intimate imagery under certain circumstances, including AI-generated material. Platforms must also provide procedures allowing victims to request removal.
Its basic logic is therefore:
A non-consensual image is published → the victim can act → the platform must remove it.
Minnesota goes further:
A tool could enable such an image to be created → the provider may be prohibited from offering that capability in the first place.
For victims, that preventive approach has an obvious advantage. Once a sexual deepfake has been generated and distributed online, completely removing it can be virtually impossible.
For AI companies, however, the same logic creates a much larger responsibility for user behavior.
Grok Makes the Case Particularly Sensitive
The fact that xAI is the plaintiff makes the dispute especially controversial.
Grok came under international criticism earlier this year over sexualized AI-generated images involving real people. Following public and regulatory pressure, xAI said it had introduced technical restrictions intended to prevent certain manipulations of real individuals.
That history makes the company’s political position vulnerable.
It does not, however, settle the constitutional issue.
Even when a state is pursuing a legitimate and compelling public interest, its laws still have to comply with the First Amendment. xAI argues that Minnesota’s statute restricts expression based on content and is broader than necessary.
Minnesota Attorney General Keith Ellison argues that non-consensual AI-generated nude imagery can cause severe personal, emotional and professional harm and that the state has a responsibility to prevent it.
For the AI Industry, the Case Is About Much More Than Nude Images
That is what makes the lawsuit economically significant.
If U.S. states are allowed to require providers to prevent specific outputs from being generated in the first place, the chain of responsibility in generative AI changes fundamentally.
So far, much of the legal responsibility has fallen on users: they create, publish or distribute illegal material. Platforms then moderate, remove content or respond to complaints.
Minnesota’s approach moves part of that risk into the product itself.
For AI companies, that could eventually mean stronger regional restrictions, geoblocking of certain functions or additional technical control layers.
And if individual states develop different rules, companies could face an increasingly fragmented regulatory landscape in which the same model functions differently depending on where a user is located.
That outcome is not inevitable. But the Minnesota case could help determine how much regulatory freedom states have to impose such obligations in the first place.
The Bigger Question
The central issue in this case is not whether people should have a right to create AI-generated nude images of others without their consent.
Even xAI explicitly acknowledges Minnesota’s legitimate interest in protecting people from non-consensual intimate deepfakes.
The dispute begins one level earlier.
How far can a government hold an AI provider responsible for what users might create with a general-purpose tool?
Minnesota’s answer is clear: when the potential harm is particularly severe, it is not enough to intervene only after an image has been created or distributed. The technology itself must have limits.
xAI argues that such limits must distinguish much more precisely between illegal abuse and lawful expression — and that providers cannot face enormous liability risks even when they deploy safeguards.
That makes the case bigger than Grok and bigger than AI-generated nude images.
It could become an important test of one of the defining questions of the next phase of AI regulation: Should liability rest primarily with the person who misuses the tool — or increasingly with the company that made the tool powerful enough to be misused?
SK
