AI sidebar spoofing attack exploits trust in browser extensions

SquareX has released research detailing vulnerabilities in AI browser sidebars, including OpenAI's newly launched Atlas browser, which expose users to a novel attack termed the AI Sidebar Spoofing attack.

The report from SquareX's research team outlines how malicious browser extensions can be used to impersonate trusted AI sidebar interfaces across popular browsers and AI-enabled browsers such as Comet, Brave, Edge, Firefox, and Safari. By crafting nearly identical replicas of legitimate AI sidebars, threat actors are able to deceive users into executing harmful actions including credential theft, device hijacking, and password exfiltration.

SquareX's research demonstrates that the trust users place in AI sidebars-often the primary interface for these tools-can be exploited by attackers. Malicious extensions can generate convincing, AI-style responses, embedding dangerous instructions within otherwise benign interactions.

"AI has become an essential tool for millions of users to learn new skills and complete tasks. Unfortunately, this has created a dangerous dynamic where people blindly follow AI-generated instructions without the expertise to identify security risks. With no visual or workflow difference, the AI Sidebar Spoofing attack exploits the trust users place on these AI interfaces, tricking them into performing malicious tasks that they may not fully understand or are aware of," explains Vivek Ramachandran, Founder and CEO of SquareX. 

The company provided several examples illustrating the threat posed by this attack. In one case study, a user asked an AI sidebar for guidance on how to withdraw cryptocurrency from an account. The compromised sidebar provided authentic-looking instructions but substituted a phishing link for the actual Binance login page. Trusting the sidebar, the user entered their credentials at the phishing site, allowing attackers to access the user's cryptocurrency account.

Other case studies demonstrated instances where AI sidebar responses instructed users to run commands that led to the exfiltration of passwords, allowed attackers to take control of devices, and even facilitated the execution of ransomware attacks remotely. SquareX cautioned that these cases represent only several potential attack scenarios, and more variants may emerge as the technique is adopted by threat actors.

The research highlights that the attack is not unique to one particular browser. AI-enabled features found in major consumer browsers as well as specialised AI browsers make the attack broadly applicable, regardless of organisational restrictions on browser use. This means any browser with an AI sidebar can become a target for these spoofing attacks.

Another aspect highlighted in SquareX's findings is the ease with which these attacks can be carried out. Malicious extensions require only basic permissions-the same types of permissions often requested by widely-used extensions such as Grammarly or password managers. As a result, such extensions can evade detection during ordinary permission analysis and may remain dormant, behaving like legitimate tools until a user provides a prompt that creates an opportunity for malicious activity.

According to SquareX, detection of these spoofed AI sidebar attacks cannot rely solely on the inspection of extension permissions. Because the malicious behaviour may not be present until triggered by specific user actions, identifying these threats requires a more dynamic approach.

SquareX stressed the importance of enterprises equipping themselves with capabilities for real-time behavioural analysis of browser extensions, as well as the implementation of detailed browser-native safeguards. These solutions can help detect, warn, or block users from following instructions generated by spoofed sidebars, thus mitigating the risk of users inadvertently assisting attackers.

The research underscores the growing issue of browser-native threats posed by extensions, which often exploit trusted interfaces to deceive users. With increasing integration of AI features into browsers and reliance on sidebars for everyday tasks, authorities and organisations face new challenges in safeguarding both consumer and enterprise environments from rapidly evolving attack vectors such as AI Sidebar Spoofing.