Your product has an AI feature now. Here is what testing it looks like

A year ago, “we have added AI” usually meant a chat box bolted onto a help page. Now it means something that reads your email, searches your documents, calls your internal APIs and acts on what it finds. The second thing has an attack surface the first one did not, and most penetration test proposals still scope it as a web form.

The one idea that explains most AI vulnerabilities

A language model cannot reliably tell the difference between the instructions you gave it and text it merely read. Everything else follows from that. If your assistant summarises an email, the email can give it orders. If it reads a web page, the page can. If it reads your server logs, the log line can - and that is not hypothetical, as we will come to.

This is why prompt injection sits at number one in the OWASP Top 10 for LLM Applications, in both the 2025 edition and the 2026 one published in August 2026. The 2026 list widened it to cover injection through images, audio and video, not just text.

What moved in the 2026 OWASP list

• LLM01 Prompt Injection and LLM02 Sensitive Information Disclosure hold the top two places.
• Excessive Agency rose from sixth to third - the biggest move on the list, and a direct consequence of agents being given real tools.
• Hidden Context Exposure is new at eighth, replacing System Prompt Leakage. It covers everything the model can see that the user cannot, not just the system prompt.
• Unbounded Consumption rose four places - cost and availability abuse is now a mainstream finding.
• Improper Output Handling fell from fifth to tenth, which does not mean it stopped mattering. It is where model output reaches a browser, a shell or a database.

Three real, patched, named cases

EchoLeak - a zero-click data leak from Microsoft 365 Copilot

CVE-2025-32711, CVSS 9.3, found by Aim Security and fixed server-side ahead of public disclosure in June 2025. One crafted email, no user interaction at all, caused Copilot to pull internal files and send their contents out. The chain is worth understanding because every link is a design decision someone made for good reasons: it slipped past the cross-prompt injection classifier, dodged link redaction using reference-style Markdown, used automatic image pre-fetching to make an outbound request with no click, and relayed the data through a Microsoft Teams preview API on an allowed domain.

Microsoft confirmed no exploitation in the wild. The lesson is not that Copilot is unsafe; it is that the exfiltration channel was built out of entirely ordinary features.

ShadowLeak - exfiltration your network never sees

Radware found that hidden HTML instructions in an email - tiny text, or white on white - would be followed by ChatGPT’s Deep Research agent when a user asked it to go through their inbox, sending inbox data to an attacker’s URL. The important part is where it happened: Radware describe it as leaking “directly from OpenAI’s cloud infrastructure, making it invisible to local or enterprise defenses.” No proxy log, no DLP alert, no confirmation prompt. Reported in June 2025, fixed in early August, acknowledged in September.

The Gemini Trifecta - the AI as the vehicle

Tenable disclosed three now-patched flaws in Google Gemini in September 2025. The one we quote most often: prompt injection hidden in the User-Agent header of an HTTP request to a cloud service. Gemini Cloud Assist later summarises those logs and carries out the instruction inside them. A second used a poisoned Chrome search history; a third used the browsing tool’s page summarisation to exfiltrate saved data without rendering any link or image.

The researcher’s summary is the sentence to take to your architects: “AI itself can be turned into the attack vehicle, not just the target.”

So what should a test of your AI feature actually cover?

Derived from OWASP’s published lists rather than from anybody’s marketing, this is the scope we would argue for:

• Every untrusted path into the context window. Not just the chat box: emails, uploaded documents, fetched web pages, repository issues, calendar invites, log lines, HTTP headers, filenames, and in 2026, images and audio.
• Hidden context. What can the model see that the user cannot, and can the user get it out? System prompts, retrieved documents, tool definitions, other tenants’ data.
• Tools and agency. What can it call, with whose identity, and with what privileges? If an attacker hijacks the goal, what is within reach? This is the question that turns a content problem into a breach.
• Exfiltration channels that need no click. Image pre-fetching, Markdown reference links, allowed-domain relay APIs, and server-side egress your own network never sees.
• Output handling. Model output reaching a renderer, a shell, a SQL layer or a template is still XSS, command injection and SSRF. The model is just the new source of untrusted input.
• Retrieval and memory. RAG corpus poisoning, cross-tenant retrieval, and whether anything written into long-term memory can steer later sessions.
• Supply chain. Models, plugins, MCP servers and datasets all have provenance, and most teams cannot tell you theirs.
• Cost and availability. Unbounded consumption is a real denial-of-wallet finding.
• And the ordinary application test. The AI feature sits on a web app with authentication, authorisation, IDOR, multi-tenancy and API flaws. None of the OWASP AI lists replaces that, and we have found more real risk in the plumbing than in the model on every engagement so far.

Testing an AI feature