Google has filed a lawsuit against a Chinese-based operation known as Outsider Enterprise for allegedly using the Gemini AI model to facilitate large-scale phishing campaigns. According to Google’s legal filings, the group operated a “phishing-as-a-service” business via Telegram, providing non-technical scammers with the tools and instructions needed to create fraudulent websites that impersonate Google, YouTube, and government agencies.
The legal action targets the group’s use of generative AI to automate the creation of deceptive content. Google alleges that Outsider Enterprise provided nearly 300 scam templates to its clients, including guides on how to leverage Gemini to refine the appearance and wording of sites designed to steal user credentials, such as those mimicking New York’s E-ZPass system.
To combat these attempts, Google collaborated with major telecommunications providers including AT&T, Verizon, and T-Mobile to block malicious text messages. The company also relies on on-device scam detection within Google Messages, a feature Google claims stops 10 billion scam texts every month.
How Outsider Enterprise used Gemini for phishing
Outsider Enterprise functioned as a middleman for cybercriminals by lowering the technical barrier to entry for phishing. According to the court documents, the group utilized Telegram channels to distribute “phishing-as-a-service” kits. These kits allowed individuals without coding skills to deploy sophisticated fraudulent websites quickly.
The group specifically integrated Google’s Gemini AI into its workflow to enhance the authenticity of these sites. By using the AI to generate convincing text and layout structures, the scammers could more effectively imitate official portals. The filings highlight that the group’s library of nearly 300 templates targeted a wide array of services, ranging from tech giants like YouTube to essential government infrastructure.
What are the impacts of AI-powered phishing?
How is Google defending against these AI scams?
By partnering with AT&T, Verizon, and T-Mobile, Google attempted to stop the delivery of the "smishing" (SMS phishing) messages before they reached the user's handset.
Beyond network blocks, the company utilizes an AI-powered detection system embedded in Google Messages. Google reports that this specific feature blocks 10 billion scam texts monthly, serving as a critical layer of defense against the templates distributed by groups like Outsider Enterprise.