San Francisco’s city attorney has demanded Apple and Google remove 13 AI-powered apps that generate nonconsensual nude images. The move highlights growing pressure on tech giants to address explicit deepfake content and protect users.
San Francisco city attorney David Chiu has issued cease-and-desist letters to Apple and Google, demanding the immediate removal of 13 apps that use artificial intelligence to create nonconsensual nude images. The legal notices, sent Thursday, accuse both companies of enabling the distribution and monetization of explicit deepfake technology through their app stores.
Chiu’s office alleges that Apple and Google have profited from these apps, which use in-app payments-of which the companies take a share-to let users generate sexualized images without consent. The letters urge the tech giants to “sever” business ties with the developers behind these tools and to strengthen their app moderation systems to prevent similar apps from appearing in the future.
“Generating non-consensual intimate images is illegal, harmful, and completely unacceptable,” Chiu told WIRED. He emphasized that California law prohibits supporting services that create deepfake pornography and called on Apple and Google to take responsibility for preventing sexual abuse facilitated by their platforms.
Researchers have repeatedly flagged apps on both the App Store and Google Play Store that allow users to create AI-generated sexual images, sometimes even labeling them as suitable for children. Despite existing developer policies banning pornography, abuse, and harassment, both companies have struggled to keep such apps off their platforms. Google spokesperson Dan Jackson stated that the company has already removed “hundreds” of apps with nudifying features, including those identified by Chiu’s office, and has restricted related search terms like 'nudify' in the Play Store. Apple did not comment before publication.
The rise of deepfake “nudification” technology has accelerated over the past five years, with apps, websites, and bots enabling users-primarily men-to upload photos of women and girls and digitally remove clothing or place them in explicit scenarios. The technology has become more sophisticated, producing increasingly realistic images and videos, sometimes for free or for a small fee. Investigations by WIRED and Indicator Media have documented cases in at least 90 schools where deepfake sexual abuse images of minors were created.
Chiu noted the severe impact on victims, stating, “These images are used to bully, humiliate, and threaten women and girls. This industry has a horrific impact on one’s reputation, mental health, and autonomy. There have been victims who’ve been suicidal.”
The 13 targeted apps-eight on the App Store and five on the Play Store-are marketed as “face-swapping” tools, but their ability to create sexual deepfakes becomes apparent once users engage with them. One app’s website, with over a million downloads, showcases a variety of AI-generated image styles, many of which are overtly sexualized. Another app advertises “free and uncensored” video creation. WIRED has chosen not to name the apps to avoid driving traffic to them.
Apple and Google have faced repeated scrutiny over the presence of such apps. Earlier this year, the Tech Transparency Project (TTP) identified around 100 similar apps and related advertisements across both platforms, estimating a combined 480 million downloads and approximately $120 million in revenue. Katie Paul, TTP’s director, said, “Apple and Google make a lot of promises in their marketing about how trusted and safe their app stores are. And that is just not what is playing out in reality.”
Recent academic research from Cornell University and Georgetown University found 420 apps offering face-swapping features on the two app stores. Of 155 apps tested, 70 percent could be used to create nude face swaps, often without any safety measures to prevent abuse. The study concluded that many AI image generation apps are “dual-use,” presenting as harmless but capable of producing harmful content.
Chiu said his office will continue to pursue the issue, describing the scale and harm of the technology as “absolutely horrifying.” He called on Apple and Google to immediately remove the flagged apps and improve their screening systems, warning that legal options remain on the table if the companies fail to act.
Concerns about AI-generated content are not limited to explicit imagery. As highlighted in a recent report, even short user posts on platforms like Reddit can manipulate AI-powered search results, raising broader questions about the reliability and safety of AI technologies in digital ecosystems. For more on how online content can influence AI systems, see this analysis of how Reddit posts can sway AI search results.
Apple, founded in 1976 and headquartered in Cupertino, California, operates one of the world’s largest app marketplaces, the App Store, which surpassed 1.8 million apps and 650 million weekly visitors as of 2025. Google’s Play Store, launched in 2012, remains the largest Android app platform globally, with over 3 million apps and billions of downloads annually. Both companies have faced increasing regulatory scrutiny over content moderation and platform safety in recent years.
