Distinguish direct input from indirect content

Direct prompt injection enters through a user’s conversation with the AI feature. Indirect injection arrives through material the system consumes: documents, web pages, retrieved passages, tool results, or other external content. The party supplying that material may have different permissions from the user running the workflow.

Inventory these sources and identify who can edit them. Include content stored in memory and reused later. A test limited to the chat box misses important paths in a retrieval or agent product. OWASP LLM guidance provides a useful risk reference; product-specific boundaries determine the actual test objectives.

Define an unauthorized outcome before testing

Choose concrete outcomes to prevent: revealing a different tenant’s document, retrieving information outside the user’s role, bypassing an approval, or invoking a tool against an unauthorized business object. Establish a legitimate task and its expected result before testing variations.

For example, a support assistant may read a ticket and propose a refund, while a separate server-side check verifies the user, account, amount, and approval before any action. Assess the checks that bind the proposed action to that identity and object.

Evaluate controls outside model instructions

  • Retrieval permissions: enforce document access and tenant filters before sensitive material reaches model context.
  • Tool authority: validate the caller, action, object, and allowed parameters at the tool or service boundary.
  • Human approval: make approvals specific to the actual action and invalidate them when material parameters change.
  • Context and memory: restrict who can write persistent state and prevent one user’s content from silently changing another’s workflow.
  • Output handling: validate generated content before rendering, interpreting, or executing it in another component.
  • Data minimization: avoid placing secrets or unnecessary customer information into model-accessible context.

Prompt wording can support intended behavior, but it should not carry the entire authorization decision. Test relevant layers individually and in the composed workflow. Read the MCP testing guide when tools are connected through MCP.

Use controlled fixtures and record variability

Supply synthetic documents with known ownership, users with different roles, and separate test tenants. Include the retrieval sources, tool responses, and persistent state relevant to the workflow. Agree limits for side effects and use controlled resources for business actions.

Record versions, configuration, inputs, starting state, observed behavior, and repeatability. Model behavior can vary between attempts. A single refusal does not establish a robust boundary; a surprising answer alone does not establish unauthorized impact.

Require actionable reporting and retesting

Each finding should explain what the attacker could control, what identity the workflow used, the boundary crossed, and the data or action affected. Separate observed evidence from untested hypotheses. Include remediation guidance and a retest that confirms the legitimate workflow still functions.

AI Application & Agent Security Assessment scopes these tests alongside RAG, memory, tools, providers, authorization, and tenant concerns. Engagements start at $4,950 USD depending on scope, with one scoped retest.

Revisit tests when authority or context changes

Review new providers, tools, retrieval sources, approval mechanisms, and agents before assuming an earlier result still applies. Maintain a small set of product-specific regression cases tied to previously observed failures, while leaving room for fresh assessment.

Continuous AI Assurance provides an agreed cadence for quarterly testing and material change reviews. It complements internal engineering checks and records what was actually reviewed.