The quality of an AI agent's output is bounded by how precisely you define what "done" looks like — vague criteria produce confident-sounding code that misses the real requirement.
What Makes Acceptance Criteria "Verifiable by an Agent"
When you write acceptance criteria for a human reviewer, you can rely on shared context, professional judgment, and the ability to ask clarifying questions mid-review. An AI agent has none of those fallbacks. It works from what is written. If your criteria depend on implicit knowledge — "the UI should feel snappy," "the API should behave correctly," "performance should be acceptable" — the agent will interpret them through its training distribution, not your production requirements. The result is code that passes a loose reading of the spec while failing against what you actually needed.
Criteria that an AI agent can verify must meet three properties. They must be observable: the criterion must describe something the agent can inspect directly in the output — code structure, return values, HTTP status codes, timing characteristics, or test results. They must be unambiguous: there must be one clear way to determine pass or fail, not a judgment call. And they must be self-contained: the agent must be able to verify the criterion without context that lives only in someone's head or in a Confluence page the agent cannot access.
Compare these two versions of the same acceptance criterion for a rate-limiting middleware:
Human-readable but agent-ambiguous: "The middleware should prevent abuse by throttling requests."
Agent-verifiable: "When a client sends more than 100 requests within a 60-second window, the middleware returns HTTP 429 with a Retry-After header set to the number of seconds remaining in the current window. Requests within the limit return the response from the next middleware in the chain unchanged."
The second version describes observable outputs, defines the exact threshold, specifies the response format, and leaves no room for interpretation. An agent implementing this can write a test for every sentence.
The important distinction is not that AI agents are less capable than humans at understanding nuance. It is that explicit criteria serve both purposes: they guide the agent's implementation, and they give you — the engineer reviewing the output — a concrete checklist to verify against. The discipline of writing agent-verifiable criteria forces you to resolve ambiguities that would have caused rework even with a human implementer.
Learning tip: For each acceptance criterion you write, ask: "Could I write a deterministic test for this statement right now?" If the answer is no, the criterion is not yet agent-verifiable. Rewrite it until the test is obvious.
Writing Given/When/Then Criteria for Code Tasks
Given/When/Then (GWT) is a format borrowed from behavior-driven development, and it is one of the most effective structures for writing acceptance criteria that AI agents can directly translate into working code and tests.
The structure maps cleanly to how agents process implementation tasks. Given establishes the preconditions — the starting state of the system. When specifies the trigger — the action, input, or event being tested. Then defines the observable outcome — what must be true after the action. Each clause gives the agent a different type of constraint: setup requirements, input boundary, and expected behavior.
Here is how GWT criteria look in practice for a password reset feature:
Given a user account exists with email "[email protected]"
And the account has not requested a password reset in the last 5 minutes
When a POST request is sent to /auth/reset-password with body { "email": "[email protected]" }
Then the response returns HTTP 200 with body { "message": "Reset email sent" }
And a password reset token is created in the database with a 1-hour expiry
And an email is dispatched to "[email protected]" containing the reset link
Given a user account exists with email "[email protected]"
And a password reset was already requested 3 minutes ago
When a POST request is sent to /auth/reset-password with body { "email": "[email protected]" }
Then the response returns HTTP 429 with body { "error": "Too many reset attempts", "retry_after": 120 }
And no new token is created in the database
And no email is dispatched
Given no account exists with email "[email protected]"
When a POST request is sent to /auth/reset-password with body { "email": "[email protected]" }
Then the response returns HTTP 200 with body { "message": "Reset email sent" }
And no token is created and no email is dispatched
Notice that the third scenario deliberately returns a 200 even though the account does not exist. This is a security decision (user enumeration prevention) that would have been missed without explicit GWT criteria. An agent implementing from a vague "add password reset" requirement would very likely return a 404, exposing which emails are registered. The GWT format forces you to think through edge cases as first-class requirements, not afterthoughts.
For code-level tasks, GWT translates directly to unit test structure:
Given the UserService is initialized with a mock UserRepository
And the repository's findByEmail returns null for any input
When validateUser("[email protected]", "anypassword") is called
Then the method returns { success: false, error: "INVALID_CREDENTIALS" }
And the repository's findByEmail is called exactly once
And no password comparison is performed
The "called exactly once" and "no password comparison performed" clauses are behavioral constraints on the implementation itself, not just the output. They prevent the agent from writing a valid but insecure implementation that short-circuits in the wrong place.
Learning tip: Write at least three GWT scenarios for every feature: the happy path, one edge case, and one failure mode. The failure mode scenario is where most AI-generated implementations fall short — they implement the happy path correctly and stub out failures with generic error responses.
Writing Behavioral Constraints — Performance Budgets, Security Rules, and API Compatibility
Acceptance criteria describe what the code must do. Behavioral constraints describe how it must do it — or what it must never do. Both categories are essential for agent-delegated implementation. The distinction matters because agents will optimize for meeting explicit criteria while ignoring unstated dimensions.
Performance budgets translate vague performance expectations into measurable thresholds the agent can design toward:
Performance constraints:
- The endpoint must respond within 200ms at p99 for payloads up to 1MB, measured under
a sustained load of 500 concurrent requests in the test environment.
- Database queries must not exceed 3 per request. Batch operations are permitted.
- The function must not allocate more than 50MB of heap memory for a single invocation.
Use streaming for inputs larger than 10MB.
Security rules describe behaviors that are prohibited, not just required:
Security constraints:
- Never log the contents of the Authorization header or any field named "password",
"token", "secret", or "key" at any log level.
- All SQL queries must use parameterized statements. String interpolation into query
templates is not permitted under any condition.
- The endpoint must validate the Content-Type header. Reject requests with
Content-Type other than "application/json" with HTTP 415.
- User input must be validated against the Zod schema before any database operation.
Schema validation errors must be returned as HTTP 422, never 500.
API compatibility constraints preserve contracts with existing callers:
Compatibility constraints:
- The public method signatures of UserRepository must not change. Callers must
not require updates.
- The response shape of GET /api/users/:id must remain backward-compatible with
version 1 of the mobile client. This means the "name" field must continue to
exist at the top level even if the internal model is refactored to use
"firstName" and "lastName".
- Do not change the HTTP status codes returned by any existing endpoint.
Only add new status codes for new error conditions.
The pattern across all three categories is the same: make the constraint measurable and testable. "Fast" is not a constraint. "Under 200ms at p99" is. "Secure" is not a constraint. "No string interpolation in SQL" is. An agent working against measurable constraints can check its own output. An agent working against qualitative adjectives cannot.
Learning tip: Treat performance budgets and security rules the same way you treat interface contracts: put them in the spec document, not just in your head. When you delegate implementation to an AI agent, the spec is the only briefing it receives. If the constraint is not written, the constraint does not exist from the agent's perspective.
How AI Agents Use Acceptance Criteria to Self-Verify
One of the most powerful patterns in agentic development is asking the agent to verify its own output against the criteria you provided. This is not just a quality check — it is a way to surface gaps between the spec you wrote and the spec you intended, before you spend time reviewing the code yourself.
The technique works because agents are often better at systematically checking a list than at spontaneously identifying all edge cases during generation. When you ask an agent to generate and then verify, you split the work into two phases that benefit from different reasoning modes.
You can also use this technique to identify criteria you forgot to write. When an agent verifies its output and confidently reports compliance, then you find a bug during your own review, that gap almost always traces back to a criterion that was missing or ambiguous in the spec. The agent followed the spec correctly; the spec was incomplete.
Learning tip: Build self-verification into your workflow as a standard step, not an optional quality gate. Treat an agent's self-verification report the same way you treat a CI run: a necessary but not sufficient condition for merging. Read the report, spot gaps in coverage, and use those gaps to improve your spec template for future tasks.
Hands-On: Writing and Using Agent-Verifiable Acceptance Criteria
This exercise walks through a real feature — adding rate limiting to a user registration endpoint — from vague requirement to fully agent-verifiable spec, then using that spec to drive implementation and self-verification.
Step 1: Start with the business requirement and extract verifiable behaviors
You receive this requirement: "We're getting bot signups. Add rate limiting to registration."
Send this prompt to your AI agent to begin decomposing it:
I have this business requirement: "Add rate limiting to the user registration endpoint to prevent bot signups."
Help me decompose this into agent-verifiable acceptance criteria. For each criterion:
1. Express it in Given/When/Then format
2. Identify the observable outputs (HTTP codes, response bodies, database state, headers)
3. Flag any security edge cases we should cover explicitly
The current endpoint is: POST /api/auth/register
Tech stack: Node.js, Express, Redis for session storage, PostgreSQL for users.
Do not write any code yet. Focus only on the acceptance criteria.
Expected output: A structured list of GWT scenarios covering the happy path (registration succeeds within rate limit), the rate-limited path (429 with retry information), and edge cases like IP-spoofing via forwarded headers.
Step 2: Add behavioral constraints to the spec
Once you have the GWT criteria, extend the spec with explicit constraints:
Good. Now add behavioral constraints to the spec in these three categories:
1. Performance constraints: The rate limiting check must add no more than 5ms of latency
to requests that are not rate-limited. The Redis check must be non-blocking.
2. Security constraints:
- Rate limit by IP address extracted from the X-Forwarded-For header, but only
trust that header if the request comes from our known load balancer IPs
(10.0.0.0/8). For all other requests, use the direct connection IP.
- The rate limit counter key must be namespaced (e.g., "rate_limit:register:{ip}")
to avoid collisions with other rate-limited endpoints.
- Do not reveal the exact rate limit threshold in the error response body.
Return only the retry_after value.
3. Compatibility constraints:
- The shape of the success response from the registration endpoint must not change.
- Add a new response header X-RateLimit-Remaining to all non-rate-limited responses.
Output the complete spec as a single document with both the GWT criteria and the constraints.
Expected output: A single, coherent spec document with GWT scenarios and three constraint sections — ready to hand back to the agent as implementation input.
Step 3: Drive implementation from the spec
Now use the spec document as the implementation brief:
Using the acceptance criteria and constraints in the spec below, implement the rate limiting
middleware for POST /api/auth/register.
<spec>
[paste the complete spec document from Step 2]
</spec>
<existing_code>
[paste the current registration route and middleware setup]
</existing_code>
Requirements:
- Implement as Express middleware
- Use the `ioredis` package already in the codebase (do not introduce new dependencies)
- Use the sliding window algorithm for rate limiting
- After implementing, show the complete middleware and the updated route registration
Expected output: A working middleware implementation and updated route. Before you review it, proceed to step 4.
Step 4: Ask the agent to self-verify against the spec
Now verify your implementation against the acceptance criteria and constraints in the spec.
For each GWT scenario, confirm:
- Which part of the implementation handles it
- What test you would write to verify it
For each constraint category (performance, security, compatibility), confirm:
- How the implementation satisfies it
- Any constraint you were unable to satisfy, and why
Output as a structured checklist. Flag any gaps.
Expected output: A checklist with each criterion mapped to specific implementation lines. Any gaps (criteria the agent could not satisfy, or constraints it bent) will surface here — before you spend time in a code review.
Step 5: Write the tests based on the self-verification report
Using the test scenarios identified in the self-verification report, write a complete
test suite using Vitest and supertest for the rate limiting middleware.
For each GWT scenario, write one test. Test names must be written in the format:
"Given [precondition], when [action], then [expected outcome]"
Include setup and teardown to reset the Redis counter between tests.
Do not mock Redis — use the test Redis instance at process.env.REDIS_TEST_URL.
Expected output: A complete test file with descriptive test names derived directly from the GWT scenarios, with proper setup/teardown.
Step 6: Identify spec gaps from failed tests
After running the tests, if any fail, send this prompt:
This test is failing:
[paste the failing test and the error output]
Before fixing the implementation, tell me:
1. Which acceptance criterion or constraint this test is checking
2. Whether the spec was ambiguous about this scenario
3. Whether the fix belongs in the implementation or whether the spec itself needs updating
If the spec needs updating, show the updated criterion before making any code changes.
This step closes the loop: failing tests reveal either implementation bugs or spec gaps. Diagnosing which it is before reaching for a fix prevents you from papering over a spec problem with an ad-hoc implementation change.
Learning tip: The order of operations matters: write the spec, implement from the spec, self-verify against the spec, write tests from the spec, then review. Engineers who jump straight to implementation and add acceptance criteria afterward are writing documentation, not driving behavior. The spec needs to exist before the implementation for it to be meaningful as a verification target.
The Difference Between Output Criteria and Process Constraints
A common confusion when writing specs for AI agents is conflating two distinct types of requirements: what the output must be, and how the implementation must arrive at it.
Output criteria describe the externally observable behavior of the finished code. They are what you verify through tests. "The function returns null when the user is not found" is an output criterion. "The endpoint returns HTTP 201 with a Location header on successful creation" is an output criterion. They do not prescribe implementation details.
Process constraints describe requirements on the implementation itself — the internal structure, the algorithms used, the libraries permitted, or the patterns that must or must not appear. "All database queries must use parameterized statements" is a process constraint — it constrains how the query is constructed, not just what it returns. "The retry logic must use exponential backoff with jitter" is a process constraint. "Do not use synchronous file I/O" is a process constraint.
Both types are necessary, but they serve different purposes and require different verification methods. Output criteria are verified by running the code. Process constraints often require reading the code.
When writing specs for AI agents, be explicit about which type you are writing:
Output criteria (verified by test):
- POST /api/users returns HTTP 201 and { "id": "<uuid>", "email": "<submitted_email>" }
on successful creation
- POST /api/users returns HTTP 409 when the email already exists
Process constraints (verified by code review):
- User IDs must be generated using crypto.randomUUID(), not sequential integers
- The password must be hashed with bcrypt at a cost factor of 12 before storage
- The plaintext password must not be stored in any variable with a lifetime beyond
the hashing operation
When you ask an agent to self-verify, output criteria can largely be verified automatically. Process constraints require you — the engineer — to review the implementation. Make this explicit in your self-verification prompt so the agent distinguishes between "I can confirm this by tracing my output" and "you need to verify this in the code."
Learning tip: When you catch an AI agent producing insecure or structurally wrong code that technically passes its own tests, the root cause is almost always a missing process constraint. Add the constraint to your spec template so it propagates to every future task, not just the one where you caught the problem.
Key Takeaways
- Agent-verifiable criteria must be observable, unambiguous, and self-contained. Criteria that depend on implicit shared context or qualitative judgment cannot be acted on reliably by an AI agent — or checked against systematically by you.
- Given/When/Then format forces you to resolve edge cases before implementation begins. Writing three scenarios — happy path, edge case, failure mode — surfaces requirements that vague descriptions hide, and gives the agent direct test-case input.
- Behavioral constraints (performance budgets, security rules, API compatibility) belong in the spec, not in your head. From an agent's perspective, any constraint not written in the spec does not exist.
- Asking agents to self-verify exposes gaps in your spec, not just bugs in the code. When a self-verification report looks clean but you find a bug in review, the spec was missing the criterion that would have caught it.
- Output criteria and process constraints are different types of requirements that require different verification methods. Output criteria are tested by running the code; process constraints require reading it. Make the distinction explicit so both you and the agent know what is being checked and by whom.