A flaky-looking CI failure and a genuine regression look identical from the GitHub Actions summary — both say "1 test failed." The difference only shows up when you actually reproduce the failure and look at what happened in the browser, which is exactly what Playwright MCP gives an agent the ability to do interactively rather than just reading a static test report. This topic chains three servers: Playwright to reproduce and diagnose the failure, Jira to create an accurately-scoped bug ticket instead of a vague "test failing" placeholder, and GitHub to land the fix and close the loop on both the code and the ticket.
Assume Playwright MCP connected (npx -y @playwright/mcp@latest), mcp-server-atlassian connected and scoped to the WEBAPP project, and github-mcp-server connected (GITHUB_TOOLSETS=repos,issues,pull_requests) with a PAT scoped to acme/webapp.
Workflow Overview: CI Test Failure to Jira Bug Ticket to Merged Fix
The full chain:
- Reproduce and diagnose — run the failing test (or an interactive equivalent) via Playwright MCP, inspect the actual page state at the point of failure, and distinguish a real regression from test flakiness.
- File an accurate ticket — create a Jira bug ticket grounded in what was actually observed (the real DOM state, the real error), not a copy of the CI log's top line.
- Fix and re-verify — implement the fix, update the test if the test itself needed adjusting, and re-run via Playwright MCP to confirm the fix actually holds before it goes near a PR.
- Close the loop — open the PR, merge, and transition the Jira ticket, following the same GitHub-Jira link discipline from earlier in this module.
This workflow assumes a CI failure worth investigating interactively — not every red build needs this treatment. A test that's failed intermittently for weeks with a known flaky-selector cause doesn't need a fresh Playwright investigation each time; this workflow earns its keep on new or unexplained failures.
claude mcp list
Tips
- Reserve this full workflow for new or unexplained CI failures — a known-flaky test with a tracked root cause doesn't need a fresh interactive investigation and Jira ticket every time it fails.
- Keep Playwright MCP's browser session ephemeral to this investigation — don't let a debugging session's cookies, localStorage, or auth state leak into a later unrelated task in the same conversation.
- Confirm all three servers are connected before starting — a missing Jira or GitHub connection mid-workflow means restarting the investigation's context from scratch rather than resuming cleanly.
Step 1: Analyzing the Playwright Test Failure and Identifying the Root Cause
Start with what CI actually reported, not a guess at what a test named "checkout flow" probably tests:
> here's the CI failure for the checkout-flow.spec.ts test:
"Timed out waiting for selector '[data-testid=order-confirmation]'".
Reproduce this locally using Playwright MCP — navigate through the
same checkout flow the test exercises and tell me what actually
happens on the confirmation step.
Claude Code drives the browser through the flow using browser_navigate, browser_click, browser_type, and takes a snapshot at the point where the test expects the confirmation element:
> snapshot the page after submitting the checkout form
- heading "Something went wrong" [level=2]
- paragraph: "We couldn't process your payment. Please try again."
- button "Retry"
This is the moment this workflow earns its cost over reading a CI log alone: the test didn't time out because a selector changed or the page got slow — checkout is actually failing, with a real error message, for a real reason. That's a materially different bug report than "test selector needs updating."
> check the network tab / requests made during that checkout submission —
what did the payment endpoint actually return?
{
"url": "/api/checkout/payment",
"status": 502,
"body": { "error": "upstream payment provider timeout" }
}
> is this reproducible consistently, or did it work on a second attempt?
Try the same flow two more times.
Attempt 2: same 502, same error page.
Attempt 3: same 502, same error page.
→ Consistent failure, not intermittent. Confirmed regression, not flake.
Three consistent reproductions is enough to rule out flakiness with reasonable confidence — don't skip this check. Filing a ticket for a genuinely flaky test as if it were a hard regression wastes triage effort on both ends; conversely, dismissing a real regression as "probably just flaky" because you didn't bother reproducing it is worse.
Tips
- Reproduce interactively and inspect actual page state and network responses before concluding what broke — a timed-out selector is a symptom, and Playwright MCP's snapshot and network visibility usually reveal the real underlying error in one or two steps.
- Run the reproduction multiple times before deciding it's a genuine regression versus flakiness — a single failed attempt doesn't distinguish the two, and misclassifying either wastes someone's time downstream.
- Capture the actual error response body, not just the HTTP status code — "502" tells you something failed, the upstream provider's actual error message tells you what to investigate next.
Step 2: Creating a Jira Bug Ticket Automatically from the Test Report
With a confirmed, reproducible root cause, file a ticket that reflects what was actually found — this is where the earlier interactive investigation pays for itself, since the ticket can describe a real failure mode instead of a vague test-report summary:
> create a Jira bug ticket in WEBAPP for this. Title should reflect
the actual failure (payment provider timeout), not the test name.
Include: what's broken, the reproduction steps, the actual API
response, and a note that this was confirmed reproducible 3/3 times.
Priority: P1 (checkout is broken for all users, not edge case).
{
"key": "WEBAPP-156",
"summary": "Checkout fails: payment provider returning 502 timeout",
"description": "Checkout submission consistently fails (3/3 reproductions) with a 502 from /api/checkout/payment, error body 'upstream payment provider timeout'. Discovered via CI test failure in checkout-flow.spec.ts, confirmed via manual reproduction — this is a real regression, not test flakiness. User sees a generic 'Something went wrong' error with no indication payment is affected.\n\nSteps to reproduce:\n1. Add item to cart\n2. Proceed to checkout, fill payment details\n3. Submit\n4. Observe 502 from payment endpoint, error page shown",
"priority": "P1"
}
Link the originating CI run for anyone who wants to see the raw test output later, and attach the Playwright investigation's evidence rather than leaving it only in the chat transcript:
> attach a comment to WEBAPP-156 with the CI run URL and a note that
the payment provider itself may be the root cause, not our code —
this might need escalation to whoever owns that integration, not
just a code fix
That caveat matters — this investigation found where the failure surfaces, not necessarily that the fix belongs in acme/webapp's codebase at all. Being explicit about that in the ticket saves whoever picks it up from assuming a pure code fix is possible when the actual issue might be a third-party outage or a misconfigured integration.
Tips
- Title and prioritize the ticket based on the actual confirmed failure, not the test's name or the raw CI error string — "payment provider 502" is a far more actionable title than "checkout-flow.spec.ts timeout."
- Include reproduction confidence (3/3, not 1/1) directly in the ticket description — it tells whoever triages this whether to treat it as an urgent confirmed regression or something needing further investigation first.
- State plainly in the ticket if the root cause might sit outside your own codebase (a third-party provider, another team's service) — don't let a ticket implicitly promise a code fix when escalation might be the actual next step.
Step 3: Fixing the Code, Updating the Test, and Closing the Jira Ticket
Suppose investigation (possibly a day later, once the payment provider's status is checked) reveals this was a real but transient provider outage, and the actual code gap is that your checkout flow doesn't retry or surface a clear error for this specific failure mode:
> the payment provider outage has resolved (confirmed via their status
page). But we should still handle this failure mode better —
currently a 502 from the payment endpoint shows a generic error.
Add a specific retry-once-then-clear-error path for payment 502s
in src/checkout/PaymentForm.tsx.
async function submitPayment(details) {
try {
return await api.post('/checkout/payment', details);
} catch (err) {
if (err.status === 502) {
// one retry for transient upstream timeouts before surfacing to user
try {
return await api.post('/checkout/payment', details);
} catch {
throw new PaymentRetryExhaustedError(
'Payment provider is temporarily unavailable. Please try again in a moment.'
);
}
}
throw err;
}
}
Update the test to cover this new path, then re-verify with Playwright MCP before touching GitHub — this re-verification is the step that actually justifies calling the fix "done," as opposed to just "written":
> update checkout-flow.spec.ts to mock a 502-then-success sequence
from the payment endpoint and assert the retry succeeds silently.
Then use Playwright MCP to run this flow against a local dev server
with the mock in place, and confirm it passes interactively before
we commit anything.
Mock configured: first payment call → 502, second call → 200.
Running checkout flow via Playwright MCP...
Confirmation page reached. order-confirmation element present.
✓ Retry path confirmed working end to end.
Now land the fix and close both the code and ticket sides together, the same GitHub-Jira link discipline from earlier in this module:
> create branch fix/WEBAPP-156-payment-502-retry off main, commit the
fix and updated test, and open a PR titled "Add retry for transient
payment 502s (WEBAPP-156)". Body should note the original CI failure,
the confirmed provider outage, and that the actual code gap addressed
here is the missing retry/error-messaging path, not the outage itself.
> transition WEBAPP-156 to "Done" and add a comment noting the PR
and that this addresses the code-side gap; the provider outage
itself has already resolved independently.
Being precise about what the fix actually addresses — the missing retry logic, not the outage — keeps the ticket's resolution honest. A future reader of WEBAPP-156 shouldn't come away thinking a code change prevented the provider outage; they should understand the code change makes the app more resilient the next time something similar happens.
Tips
- Re-verify a fix interactively via Playwright MCP, with the specific failure condition mocked, before committing — a fix that looks correct in a code review can still miss the actual runtime sequence that triggered the original bug.
- Update the test alongside the fix so the exact failure mode this investigation uncovered has permanent regression coverage — the whole point of the investigation was learning something the original test didn't check for.
- Write the PR and closing Jira comment precisely about what the fix does and doesn't address — conflating "we added resilience" with "we fixed the outage" misrepresents the work to anyone reading the ticket later.
Tips
Tips
- Reproduce a CI failure interactively via Playwright MCP — actual page state, actual network responses, multiple attempts — before deciding whether it's a real regression or flakiness; don't file or dismiss a ticket based on the raw CI log alone.
- File the Jira ticket with the real observed failure, confirmed reproduction count, and an honest statement of whether the root cause even lives in your codebase — a ticket this precise is what makes the rest of the chain (fix, PR, closure) fast instead of a second investigation.
- Re-verify the fix through Playwright MCP with the original failure condition mocked before it reaches a PR, and word the closing PR and Jira comment around exactly what the fix addresses, not the entire original incident.