State-sponsored threat actors are leveraging Google's Gemini AI to enhance their malicious cyber activities, despite the company's efforts to detect and prevent such misuse. The Google Threat Intelligence Group (GTIG) has documented this activity in a report titled 'AI Threat Tracker: Advances in Threat Actor Usage of AI Tools'. The report reveals that Gemini is being used across multiple stages of attack campaigns, from initial reconnaissance to data exfiltration.
One notable example involves a China-linked actor who posed as a capture-the-flag competition participant to persuade Gemini to provide exploitation guidance. The actor then used this technique to obtain advice on phishing, exploitation, and webshell development. Similarly, an Iranian group called MUDDYCOAST posed as university students to bypass safety guardrails and obtain assistance in developing custom malware.
These threat actors have also been observed using Gemini to research and develop tools for data exfiltration, lateral movement, and C2 (command-and-control) infrastructure. For instance, a suspected Chinese threat actor demonstrated particular interest in attack surfaces they appeared unfamiliar with, including cloud infrastructure, vSphere, and Kubernetes.
Google's mitigations involve disabling accounts after detection rather than real-time blocking, creating a window where actors can extract value before disruption. The company has also identified experimental malware that suggests how threats may evolve, including tools that query language models during execution to generate malicious code on the fly.
However, the report also highlights the evolving nature of these threats, with threat actors continuously adapting their techniques to bypass detection. This includes the use of social engineering pretexts and the development of custom malware, such as webshells and C2 servers.
The report encourages further discussion and collaboration among security researchers and practitioners to address these emerging threats. It also invites readers to share their thoughts and experiences in the comments section, highlighting the importance of collective knowledge and learning in the face of evolving cyber threats.
