\n<\/p>\n
Earlier this week, the California attorney general\u2019s office announced that it was investigating xAI over reports that the startup\u2019s chatbot, Grok, was being used to create nonconsensual sexual imagery of women and minors. On Friday, the government followed up by sending a cease-and-desist letter to the company, demanding that it take immediate action to stop the production of nonconsensual intimate images and CSAM \u2014 child sexual abuse material.<\/p>\n
\u201cToday, I sent xAI a cease-and-desist letter, demanding the company immediately\u00a0stop the creation and distribution of deepfake, nonconsensual, intimate images and child sexual abuse material,\u201d said California AG Rob Bonta in a press release. \u201cThe creation of this material is illegal. I fully expect xAI to immediately comply. California has zero tolerance for [CSAM].\u201d<\/p>\n
The AG\u2019s office additionally claimed that xAI appeared to be \u201cfacilitating the large-scale production\u201d of nonconsensual nudes, the likes of which are being \u201cused to harass women and girls across the internet.\u201d The agency said it expects xAI to prove that it is taking steps to address these issues within the next five days. <\/p>\n
At the heart of the backlash is Grok\u2019s \u201cspicy\u201d mode feature, which xAI created to generate explicit content. The issue has spread beyond California; Japan, Canada, and Britain have opened investigations into Grok, and Malaysia and Indonesia have temporarily blocked the platform altogether. Despite xAI instituting some restrictions on its image-editing features late Wednesday, the California AG\u2019s office moved ahead with its cease-and-desist letter.<\/p>\n
X\u2019s safety account has previously denounced this kind of user activity, saying: \u201cAnyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.\u201d TechCrunch reached out to xAI for comment and was met with an automated email that says \u201cLegacy Media Lies.\u201d TC also reached out to the California AG\u2019s office for more context.<\/p>\n
The advent of free generative AI tools has led to a disturbing swell of non-consensual sexual material. Many platforms have been grappling with this problem, not just X. The sordid activity has drawn the attention not just of state leaders but also of Congress. Indeed, on Thursday, lawmakers sent a letter to the executives of several companies \u2014 including X, Reddit, Snap, TikTok, Alphabet, and Meta \u2014 asking how they planned to stem the proliferation of sexualized deepfakes.<\/p>\n<\/div>\n