• Posted by: Michael Johnson
• September 25, 2024

ChatGPT macOS Vulnerability SpAIware Could Enable Persistent Spyware

A recently patched security vulnerability in OpenAI's ChatGPT app for macOS could have allowed attackers to inject persistent spyware into the AI tool’s memory, posing a serious risk to user privacy. The exploit, dubbed SpAIware, was discovered by security researcher Johann Rehberger, who demonstrated how the flaw could enable continuous data exfiltration of any information typed or received in ChatGPT, even across future chat sessions.

Exploiting ChatGPT’s Memory Feature

The root of the issue lies in ChatGPT’s new memory feature, introduced by OpenAI in February 2024 and rolled out to Free, Plus, Team, and Enterprise users earlier this month. This feature allows ChatGPT to retain information across multiple chat sessions, saving users the hassle of repeating themselves. Users also have the option to delete specific memories, but as OpenAI notes, deleting a chat does not erase its memory; users must delete the memory itself.

This capability, while convenient, opened the door for potential misuse. Rehberger’s research builds on earlier findings that demonstrated how indirect prompt injection could manipulate ChatGPT’s memory to store false or even malicious information. The malicious instructions stored in ChatGPT’s memory would then persist across all new conversations, effectively making the exfiltration vulnerability more severe.

How SpAIware Works

In a hypothetical attack scenario, a user could be lured into visiting a malicious website or opening a compromised document. If the document or website is analyzed using ChatGPT, it could contain hidden instructions that update ChatGPT’s memory to send all future conversation data to an attacker-controlled server.

“Since the malicious instructions are stored in ChatGPT’s memory, all new conversations going forward will contain the attacker’s instructions and continuously send all chat conversation messages and replies to the attacker,” said Rehberger. “The data exfiltration vulnerability became a lot more dangerous as it now spawns across chat conversations.”

Patch and Mitigation

Following responsible disclosure, OpenAI patched the issue in ChatGPT version 1.2024.247, effectively closing the data exfiltration vector. Users are strongly advised to update their ChatGPT app to the latest version to prevent exploitation.

Rehberger also recommended that users regularly review the memories stored by ChatGPT for any suspicious or incorrect entries and clean them up if necessary. “This attack chain was quite interesting to put together and demonstrates the dangers of having long-term memory being automatically added to a system, both from a misinformation/scam point of view, but also regarding continuous communication with attacker-controlled servers,” he added.

Emerging AI Security Concerns

This disclosure comes at a time when AI security is under heightened scrutiny. Researchers recently uncovered a novel AI jailbreaking technique called MathPrompt, which leverages large language models' (LLMs) advanced capabilities in symbolic mathematics to bypass their safety mechanisms.

MathPrompt employs a two-step process: first, it transforms harmful natural language prompts into symbolic mathematics problems and then presents these mathematically encoded prompts to a target LLM. The study, which tested 13 state-of-the-art LLMs, found that the models responded with harmful output 73.6% of the time on average when presented with mathematically encoded prompts, compared to approximately 1% with unmodified harmful prompts.

Microsoft's AI Safety Advancements

In response to such vulnerabilities, Microsoft has introduced a new Correction capability for its AI models. This feature, building on the existing Groundedness Detection, aims to identify and correct inaccuracies (i.e., hallucinations) in real-time before they reach end-users. “This groundbreaking capability allows Azure AI Content Safety to both identify and correct hallucinations in real-time before users of generative AI applications encounter them,” the company stated.

The rapid advancements in AI technology bring not only benefits but also new security challenges. The SpAIware vulnerability in ChatGPT and the MathPrompt jailbreaking technique highlight the need for ongoing vigilance and robust security measures in the development and deployment of AI systems. Users are encouraged to stay informed and proactive in securing their AI tools and data against emerging threats.