AI Application Security Review
Security testing for AI features in real products — what the model can be persuaded to do, what it can reach, and what your application does with what it returns.
Why AI features need their own review
An AI feature changes the shape of an application\u2019s trust boundaries. Text that arrives from a user, a document, or a third-party page is no longer only data to be stored and displayed; it becomes instructions that influence what your software does next. When that software can also call tools, query internal services, or read a shared knowledge base, a sentence in an uploaded file starts to resemble a request to your API.
Conventional application testing does not naturally cover this. It examines the request your code expects, not the instruction your model was persuaded to follow. Equally, a review that examines only prompts and guardrails misses the part that determines actual impact: the permissions behind the tools and the access boundaries around the data.
How we approach it
We start from consequence rather than from technique. First we establish what the feature can do and reach — which tools it can invoke, whose authority it acts under, what the retrieval layer can return, and where its output flows. That determines what a successful manipulation would actually be worth, and therefore where the testing effort belongs.
We then test it directly and, importantly, indirectly: through the content the model ingests without a human reviewing it, which is how these features are compromised in practice rather than in demonstrations.
Our recommendations favour architecture over wording. Instruction-level defences are worth having, but they are mitigations, not boundaries. A tool that cannot exceed the requesting user\u2019s permissions is a boundary; a prompt asking the model to behave is not.
What we examine
The final scope depends on what your AI feature does. This is the surface we normally cover, adjusted to your architecture.
- Instruction and prompt handling
- Whether untrusted input can alter the model’s instructions, override guardrails, extract the system prompt, or reach behaviour the feature was never meant to expose. This includes indirect injection through documents, web pages, filenames, and other content the model ingests without a human reading it first.
- Retrieval and context boundaries
- Whether a retrieval pipeline can be induced to return content the requesting user is not entitled to see. In multi-tenant products this is the single most consequential AI security question, because embeddings and vector stores frequently inherit weaker access control than the primary database.
- Agent and tool access
- What actions the model can invoke, on whose authority, and with what limits. We look for tools exposed with broader permissions than the user holds, missing confirmation on consequential actions, and chains where a model decision becomes a privileged operation without an authorization check.
- Output handling
- What happens downstream when model output is trusted: rendered as HTML, executed as code or a query, passed to a shell or template, or used to construct a request. Treating model output as trusted input is one of the most common serious defects in AI features.
- Data flow and disclosure
- What leaves your environment, where it goes, what is retained, and whether sensitive data reaches a model provider, a log, a trace, or a cache in a way your privacy commitments do not cover.
- Resource and cost abuse
- Whether an unauthenticated or low-privilege user can drive disproportionate model spend, exhaust quota for other tenants, or use your product as a free proxy to a paid model.
- Surrounding application controls
- The conventional surface around the AI feature: authentication on the endpoints that reach the model, authorization on the data it can access, tenant isolation, and the cloud permissions held by the components that run inference.
What you receive
- A technical report covering each finding with reproduction steps, the exact input or sequence used, evidence, business impact, and remediation guidance.
- A clear separation between findings that are deterministic and those that are probabilistic, with the reliability we observed stated explicitly.
- Guidance on architectural mitigations, not only prompt-level ones, since instruction defences alone are not a security boundary.
- A live debrief with the engineers who built the feature.
- A retest of remediated findings, recorded against each original item.
What this review is not
These sit outside a security review. Naming them plainly is more useful than implying coverage we do not provide.
- Model quality, accuracy, factual correctness, or hallucination rates, which are product quality concerns rather than security findings.
- Bias, fairness, or content-moderation evaluation, which require a different discipline and different expertise.
- Formal AI regulatory certification or compliance attestation.
- Training-data provenance auditing or intellectual property review.
- Testing a model provider’s own infrastructure, which we are not authorized to test.
- Social engineering, denial-of-service, and destructive techniques, which are excluded from all our engagements by default.
Common questions
Is this just prompt injection testing?
No. Prompt injection is one class of finding, and on its own it is often not the most serious one. The consequential question is what an injected instruction can actually reach: which tools are exposed, what data the retrieval layer can return, whose permissions the action executes under, and what the surrounding application does with the output.
An AI feature with no tools and no private data has a limited injection impact. An AI feature that can call internal APIs on a user’s behalf is a different matter entirely, and that is where we focus.
Can you test a feature that is still in development?
Yes, and it is often the better moment. Reviewing an AI feature before launch means tool permissions, retrieval boundaries, and output handling can still be changed at the architectural level rather than patched with instruction wording afterwards.
We will need a working environment we can exercise, and it should reflect the permission model you intend to ship.
How do you handle findings that are not perfectly reproducible?
Honestly and explicitly. Model behaviour is probabilistic, so some findings reproduce every time and some reproduce intermittently. The report states which is which, and where a finding is intermittent we record how often it succeeded under the conditions we tested.
We do not report a single lucky result as a reliable exploit, and we do not dismiss an intermittent one either — an attacker only needs it to work occasionally.
Do you test the underlying model or the provider?
Neither. We test your application: how it constructs context, what it permits, what it trusts, and what it exposes. Testing a third-party model provider’s infrastructure would be outside the authorization you can grant us, and we do not do it.
Where a risk genuinely originates in a provider’s behaviour or configuration options, we report it as something for you to consider in your architecture and your contracts.
Which frameworks do you reference?
We use recognised AI security references — including the OWASP Top 10 for LLM Applications and the MITRE ATLAS knowledge base — as coverage checklists so that classes of issue are not missed.
They inform the work but do not define it. A checklist cannot know what your agent is allowed to do in your environment, and that is usually where the real finding is.
Have an AI feature reviewed before it ships
Tell us what the feature does, what it can reach, and when it launches. We will come back with a scope and an approach in writing.