Dive Brief:
- Cyber threat actors have recently begun using AI to develop malware, in a dramatic evolution of the technology’s role in the hacking ecosystem, Google said on Wednesday.
- New strains of malware use AI to grow and change in real time during the attack phase, potentially making detection and defense much more difficult, Google’s threat intelligence researchers said in a report.
- The recent trend represents the latest phase in an AI arms race between attackers and defenders.
Dive Insight:
Over the past few years, researchers have consistently found that hackers are using AI more as an enhancer of phishing lures than as a malware-generation assistant. AI malware toolkits exist on the dark web, experts have said, but they do not represent the most widespread or worrisome use of the technology. Google’s new findings, however, suggest that AI’s role in offense may be entering a new phase.
Five newly discovered malware families — FRUITSHELL, PROMPTFLUX, PROMPTSTEAL, PROMPTLOCK and QUIETVAULT — exhibit novel AI-powered capabilities, Google reported, including the ability to hide their code from security software, create attack capabilities on demand and dynamically generate scripts. “While still nascent,” Google said, “this represents a significant step toward more autonomous and adaptive malware.”
PROMPTFLUX uses Google’s Gemini AI to regenerate its own code to better avoid detection, hiding the newly reconstituted file in Windows’s Startup folder. One version of the malware used Gemini to rewrite its entire source code every hour. “This type of obfuscation technique is an early and significant indicator of how malicious operators will likely augment their campaigns with AI moving forward,” Google said. The company did not attribute PROMPTFLUX to a specific threat actor, but it said the names of files used to drop the malware were consistent with the behavior of “financially motivated actors.”
Currently, PROMPTFLUX’s code includes inactive components and a feature to limit its interaction with the Gemini API, suggesting that it remains under development. Google said it had “taken action to disable the assets associated with this activity,” adding that PROMPTFLUX alone could not hack into systems.
PROMPTSTEAL, meanwhile, uses Hugging Face’s platform to query a large language model (LLM) and generate short Windows commands that collect and steal information from target systems. The software masquerades as an image-generation tool that generates and executes reconnaissance commands in the background. By dynamically generating new scripts to perform commands, the malware could help hackers stay active on a target machine without alerting defenders who are looking for specific snippets of code.
Google said it had observed APT28, a Russia-linked group associated with the country’s GRU military intelligence agency, using PROMPTSTEAL in Ukraine. Authorities in that country previously reported on the malware’s appearance. Google said those attacks were the first time it had seen malware querying an LLM in the wild.
“Although some recent implementations of novel AI techniques are experimental,” Google said, “they provide an early indicator of how threats are evolving and how they can potentially integrate AI capabilities into future intrusion activity.”
The newly discovered uses of AI in malware highlight the need for defenders to replace traditional static detection tools with software that can identify a broader range of anomalous activity.
“Attackers are moving beyond ‘vibe coding’ and the baseline observed in 2024 of using AI tools for technical support,” Google researchers wrote. “We are only now starting to see this type of activity, but expect it to increase in the future.”
Threat actors continue to use AI for other purposes too. The Google report described a China-link group using Gemini for “crafting lure content, building technical infrastructure, and developing tooling for data exfiltration.” To work around Gemini’s protections against malicious use, the threat actor posed as a participant in a capture-the-flag exercise and persuaded Gemini to provide “helpful information that could be misused to exploit the [targeted] system.” The China-linked actor then used the CTF ruse in many future Gemini prompts.
Other nation-state groups weren’t as lucky. An Iran-linked hacking group tried to use Gemini to develop custom malware, but in doing so, it revealed information about its operations — including the domain of its command-and-control server — that helped Google disrupt its activities.
