When does an AI assessment make sense?

Prioritize independent testing when an AI feature can read customer information, retrieve permissioned documents, call business tools, or act across identities. Common triggers include an enterprise launch, a regulated customer’s review, a new agent, or a major model/provider change. Ordinary employee use of a hosted writing assistant is a different scope from a production agent with CRM access.

A useful brief names the buyer requirement, release date, affected workflows, and expected evidence. If the requirement itself is unclear, a Security Gate Sprint can first separate documentation gaps from testing needs.

Map the system around the model

List instruction and data sources: user messages, uploaded files, retrieved documents, external pages, tool responses, and stored memory. Trace outputs into the UI, database, downstream APIs, or other agents. Include model routing, fallback providers, retrieval indexes, service identities, and tenant boundaries.

Define the authorized result for each role. For example, a support agent may summarize one customer’s tickets but require separate approval to issue a refund. That distinction gives testers an objective permission boundary and business impact to assess.

Define coverage by trust boundary

  • LLM behavior: direct and indirect prompt injection, instruction precedence, sensitive-context exposure, and unsafe output handling.
  • RAG: document and metadata permissions, retrieval filters, tenant isolation, cache behavior, and what happens when permissions change.
  • Agents and tools: excessive agency, tool selection, action authorization, approvals, delegated identities, and limits across chained actions.
  • MCP integrations: client/server trust, authorization, requested scopes, session handling, and tool metadata. See the MCP assessment guide .
  • Memory and context: who can write persistent state, how it influences later decisions, and whether context crosses users or tenants.
  • Model/provider boundaries: routing, data exposure, fallback behavior, and differences between configured environments.

OWASP LLM guidance and OWASP Agentic Applications guidance offer risk categories for planning. Map relevant categories to specific product workflows and record exclusions; framework alignment is not certification.

Prepare realistic, controlled test access

Provide synthetic documents and accounts for at least the distinct roles and tenants in scope. Document expected access, permission changes, connector ownership, and approval steps. Include service identities and background tasks where they behave differently from interactive users.

Use controlled resources for actions with side effects. Agree authorized environments, third-party permissions, stop conditions, escalation, and evidence handling in the rules of engagement . Identify any workflow that must be simulated instead of exercised.

Ask for findings that connect behavior to impact

A report should identify the starting conditions, configuration, relevant inputs, observed result, and failed security boundary. Capture model/provider versions, retrieval and tool configuration, test identities, and meaningful reproduction limits. Separate an undesirable answer from actual unauthorized data access or action.

Include an agent identity and tool-permission map and a role/tenant/workflow coverage matrix. Link each path to the acting identity, accessible data, permitted actions, approval requirements, and evidence. Require remediation priorities and a scoped retest. AI responses can vary; record repeatability and residual uncertainty instead of treating one successful or failed attempt as universal evidence.

Budget for the actual workflow scope

The DevBrows AI Application & Agent Security Assessment has a starting engagement range of $12,000–$20,000 USD . Workflows, providers, tools, MCP servers, roles, memory, and tenants determine the quote. Expanded scope can exceed this range. The agreed assessment includes reporting, remediation guidance, and one retest.

A SaaS Pentest starts at $9,500 . A combined proposal should distinguish conventional application coverage from AI workflow coverage and identify overlap. Confirm scope and deliverables before comparing prices.

Keep evidence current as the product changes

A report describes the system tested at a point in time. A new provider, write-enabled tool, shared retrieval index, or privileged agent can create a new review requirement. Maintain a change log and agree which changes need targeted testing.

Continuous AI Assurance combines quarterly testing, agreed change reviews, questionnaire support, and an annual reassessment under a custom annual engagement. Use the annual planning guide to define capacity and triggers.