AI Autopsy: How Significant Is Google’s AI-Enabled Malware Discovery?

Kate O’Flaherty asks whether network defenders should be worried about a new breed of LLM-powered threats 

The cybersecurity industry has never been short of hyperbole. But that’s a challenge for network defenders trying to separate genuine threats from vendor FUD. A new collection of AI-enabled malware reported by Google has drawn some alarming headlines, but how concerned should CISOs be?

What does the report say?

In the report, the Google Threat Intelligence Group (GTIG) claimed to have discovered the first use of “just-in-time” AI in malware, so-called because it can dynamically change behaviour mid-attack to evade detection and download new functions on demand.

“While most of this is experimental, it’s a step forward from hypothetical” John Hultquist

“For the first time, GTIG has identified malware families, such as PromptFlux and PromptSteal, that use LLMs during execution”, Google wrote. These variants stand out due to their ability to “dynamically generate malicious scripts, obfuscate their own code to evade detection, and leverage AI models to create malicious functions on demand, rather than hard-coding them into the malware”, it said.

While still a nascent threat, they “represent a significant step toward more autonomous and adaptive malware”, the report warned.

“While most of this is experimental, it’s a step forward from hypothetical, which is what we’ve observed previously. It won’t be long before this is simply practical,” John Hultquist, GTIG chief analyst, tells Assured Intelligence.

Digging deeper

There are actually several malware variants covered in the report:

• PromptFlux: A dropper written in VBScript that decodes and executes an embedded decoy installer to mask its activity. It can “regenerate” using the Google Gemini API. “It prompts the LLM to rewrite its own source code, saving the new, obfuscated version to the Startup folder to establish persistence. PromptFlux also attempts to spread by copying itself to removable drives and mapped network shares,” according to GTIG
• PromptLock: Cross-platform ransomware written in Go. ESET first reported the proof-of-concept malware, which originated in an academic report. It leverages an LLM to dynamically generate and execute malicious Lua scripts at runtime, with capabilities including reconnaissance, data exfiltration and encryption on Windows and Linux systems
• PromptSteal: A data miner written in Python and packaged with PyInstaller. It contains a compiled script that uses the Hugging Face API to query the LLM Qwen2.5-Coder-32B-Instruct to generate one-line Windows commands. In June 2025, GTIG identified Russian government-backed APT28 (aka Fancy Bear) using the malware against Ukraine
• QuietVault: A credential stealer written in JavaScript that targets GitHub and npm tokens. Captured credentials are exfiltrated by creating a publicly accessible GitHub repository. The malware also uses an AI prompt and on-host installed AI command-line interface (CLI) tools to search for additional potential secrets on the infected system and exfiltrate the files to GitHub. GTIG has observed it in operations
• FruitShell: A publicly available reverse shell written in PowerShell that establishes a remote connection to a configured command and control server, and allows adversaries to execute arbitrary commands on a compromised system. The code family contains hard-coded prompts designed to bypass detection by LLM-powered security systems. GTIG has also observed this malware in the wild

Should CISOs be concerned?

PromptFlux is particularly significant because it suggests adversaries are experimenting with LLMs to develop “dynamic obfuscation techniques”, according to GTIG. This is concerning, but only up to a point. The current version “does not demonstrate an ability to compromise a victim network or device”, GTIG said, adding that it has “taken action to disable the assets associated with this activity”.

On the other hand, APT28 has been using PromptSteal alongside AI LLMs in real-life attacks. “APT28’s use of PromptSteal constitutes our first observation of malware querying an LLM deployed in live operations,” Google warned.

“Each generation of the malware is different from the last” David Sancho

All of the malware samples are similar in nature. They perform malicious actions and call the Google Gemini API to prompt Google’s chatbot to modify the source code, David Sancho, a senior threat researcher at Trend Micro, tells Assured Intelligence.

“This means that each generation of the malware is different from the last,” he says. “When used for specific malicious effects, the malware does not carry code to perform those malicious actions, relying on the chatbot to generate code instead.”

It is not the first example of AI-enabled malware. Still, on current evidence, it’s the first to show AI operating inside the kill chain, Rik Ferguson, VP of security intelligence at Forescout, tells Assured Intelligence. “Google’s work shows PromptFlux and PromptSteal calling an LLM mid-execution to mutate, plan and fetch fresh functions on demand, while QuietVault goes after developer tokens and tries to dodge model guardrails,” he explains.

Tony Gee, a principal cybersecurity consultant at 3B Data Security, thinks the discoveries are “pretty minor”. He highlights a LinkedIn post by Marcus Hutchins, which has since been partially redacted.

Hutchins points out that PromptFlux itself doesn’t know how to evade anti-virus (AV); it just asks Gemini to do so. “Gemini doesn’t know how to evade AV. So in short, it doesn’t work,” Gee tells Assured Intelligence.

The report highlights how adversaries are increasingly using AI in the creation and execution of malware. Google is building its LLM with guardrails to prevent abuse like this – unlike the unrestricted AI platforms appearing on underground markets. These are already being used by adversaries across the attack lifecycle, GTIG said.

“While adversaries are certainly trying to use mainstream AI platforms, guardrails have driven many to models available in the criminal underground,” GTIG tech lead, Billy Leonard, said when launching the research.

Those tools are unrestricted and can offer “a significant advantage” to the less skilled, he points out. “There are several of these available now, and we expect they will lower the barrier to entry for many criminals.”

The impact on defenders

The report shows the potential for adversaries to use LLMs alongside malware in live attacks to evade detection. But this shouldn’t be a significant concern yet, experts agree.

PromptLock and QuietVault will have “limited to no impact” on network defenders right now, Martin Kraemer, CISO advisor at KnowBe4, tells Assured Intelligence. “As the Google researchers state, the PromptLock malware is currently in the development and testing phase and easily detected by more than a third of leading industry scanners according to VirusTotal.”

“Gemini doesn’t know how to evade AV. So in short, PromptFlux doesn’t work” Tony Gee

At the same time, while the QuietVault information stealer has been observed in operations, it has also been detected by more than half of the leading industry scanners.

For defenders, these malware strains are “not particularly more difficult to detect”, adds Trend Micro’s Sancho. “In fact, it could be said that they rely on the connection to Google, so that could be a telltale sign,” he adds. “In a network where Google Gemini is not being used, a call to it via API would be a distinct signal.”

The report shows attackers’ intent to develop self-modifying malware that would evade signature-based detection by regenerating hourly, says KnowBe4’s Kraemer. “However, the developers have not yet fully evolved and perfected the feature.”

Getting the basics right

The threat posed by AI-powered malware is no different from that posed by other malware. But because the malware families identified by Google are more complex, they also have more weaknesses, according to Trend Micro’s Sancho.

“Since they depend on a third-party API call to happen correctly, they are distinctly more prone to failure,” he argues. “In fact, Google has already disabled the API keys they used, so new infections are already ineffective. This will happen again in the future, so it makes this strategy a weak one for future malware strains.”

As 3B Data Security’s Gee points out, CISOs should be shoring up the basics before worrying about future threats such as these. “We still get massive companies taken out for months simply by someone phoning up a service desk and asking for a password reset,” he explains.

Gee advises “reining in the hysteria with AI malware” and going back to basics. This should include endpoint detection and response, zero trust, strong passwords, rapid incident response, multi-factor authentication and “getting a handle on credential and session token abuse”, he says.

“Let’s get on that journey to maturity by getting the basics right before stressing about and sensationalising AI-powered malware,” Gee concludes.