Playwright MCP With Gemini CLI

Gemini CLI, Google's open-source terminal agent, supports MCP via a settings.json file and — notably — has a large context window (Gemini 2.5 Pro's 1M-token context) that changes the economics of long, multi-step Playwright MCP sessions compared to smaller-context agents. That's the main practical difference worth knowing going in: you can hand Gemini CLI a much longer flow description, or let a session accumulate more accessibility snapshots, before context management becomes a concern.

Installing and Connecting Playwright MCP to Gemini CLI

Gemini CLI reads MCP config from .gemini/settings.json at the project root, or ~/.gemini/settings.json for a global setup:

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": ["-y", "@playwright/mcp@latest", "--isolated"],
      "timeout": 30000
    }
  }
}

The timeout field (milliseconds) is worth setting explicitly — Gemini CLI's default MCP call timeout has been too short for slower browser_navigate calls against apps with heavy client-side rendering in some versions, causing spurious "tool call timed out" errors on the very first navigation.

Launch and verify:

gemini

/mcp list

Configured MCP servers:

playwright - Ready (23 tools)

If the server shows "Disconnected," run npx -y @playwright/mcp@latest --isolated directly first to confirm it's not an npx/network issue, then check that .gemini/settings.json is valid JSON — a trailing comma is a common cause of Gemini CLI silently ignoring the whole mcpServers block without a clear error.

node -e "JSON.parse(require('fs').readFileSync('.gemini/settings.json'))"

Tips
- Set an explicit timeout value above the default for any app with slow initial page loads — a bare navigate timing out on the first call is a frequent false-negative "connection failed."
- Validate .gemini/settings.json with a plain JSON parser before troubleshooting further — malformed JSON fails silently rather than with a clear parse error in some CLI versions.
- Use --isolated here too, for the same reason as other clients: no persisted profile leaking session state between runs.

Generating Playwright Tests from User Stories and Requirements in Gemini CLI

Gemini CLI's larger context window means you can feed it a full user story or acceptance criteria document alongside the live-browser instruction, rather than distilling it down to a terse action list first.

Here's the acceptance criteria for the "saved addresses" feature:

- A logged-in user can add up to 5 saved addresses.
- Each address requires: label, street, city, state, zip.
- The 6th "add address" attempt shows an error: "Maximum 5 addresses reached."
- A saved address can be set as default; only one default at a time.
- Deleting the default address prompts the user to pick a new default first.

Navigate to http://localhost:3000/account/addresses (already logged in as
the seeded test user) and verify each of the above behaviors manually via
Playwright MCP. Then generate a full Playwright test file at
e2e/saved-addresses.spec.ts covering all five criteria as separate test
cases.

Because it worked through each criterion live before writing code, expect a file with distinct test() blocks per acceptance criterion rather than one giant test — which is the structure you want for readable failure reports:

import { test, expect } from '@playwright/test';

test.describe('saved addresses', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto('/account/addresses');
  });

  test('allows adding up to 5 addresses', async ({ page }) => {
    for (let i = 1; i <= 5; i++) {
      await page.getByRole('button', { name: 'Add address' }).click();
      await page.getByLabel('Label').fill(`Address ${i}`);
      await page.getByLabel('Street').fill(`${100 + i} Main St`);
      await page.getByLabel('City').fill('Springfield');
      await page.getByLabel('State').fill('IL');
      await page.getByLabel('Zip').fill('62701');
      await page.getByRole('button', { name: 'Save address' }).click();
    }
    await expect(page.getByTestId('address-card')).toHaveCount(5);
  });

  test('blocks a 6th address with a max-limit error', async ({ page }) => {
    // assumes 5 addresses already seeded via fixture, not repeated UI entry
    await page.getByRole('button', { name: 'Add address' }).click();
    await expect(page.getByText('Maximum 5 addresses reached.')).toBeVisible();
  });

  test('allows setting a different address as default', async ({ page }) => {
    const cards = page.getByTestId('address-card');
    await cards.nth(1).getByRole('button', { name: 'Set as default' }).click();
    await expect(cards.nth(1).getByText('Default')).toBeVisible();
    await expect(cards.nth(0).getByText('Default')).not.toBeVisible();
  });

  test('prompts to choose a new default before deleting the current default', async ({ page }) => {
    const cards = page.getByTestId('address-card');
    await cards.first().getByRole('button', { name: 'Delete' }).click();
    await expect(page.getByText(/choose a new default/i)).toBeVisible();
  });
});

Note the comment in the second test acknowledging a seeding assumption — a useful, honest artifact rather than the model silently repeating five rounds of UI-driven address creation in every test, which would be slow and brittle. Push back if it doesn't add that kind of note itself:

The "blocks a 6th address" test assumes 5 addresses already exist. Either
add a proper fixture/seed step for that, or add a comment explaining the
assumption and how CI should provide that seeded state.

Tips
- Feed full acceptance criteria or user story text directly — Gemini CLI's large context window handles it well and produces better-segmented test cases than a terse action list.
- Ask for one test() per acceptance criterion, not one monolithic test — failures are far easier to triage that way.
- Call out state-dependency assumptions (like "6th address" needing 5 pre-existing) and require either a real fixture or an explicit comment — don't let it pass silently.

Practical Example: Generating a Login Flow E2E Test in Gemini CLI

A simpler, single-flow example to show baseline output quality without extra scaffolding requirements:

Navigate to http://localhost:3000/login. Test three cases: (1) valid
credentials succeed and redirect to /dashboard, (2) wrong password shows
"Invalid email or password", (3) submitting with empty fields shows
required-field validation without a network request firing. Use Playwright
MCP's network inspection to confirm case 3 doesn't hit the API. Then write
e2e/login.spec.ts covering all three.

import { test, expect } from '@playwright/test';

test.describe('login', () => {
  test('valid credentials redirect to dashboard', async ({ page }) => {
    await page.goto('/login');
    await page.getByLabel('Email').fill('demo@example.com');
    await page.getByLabel('Password').fill('Demo1234!');
    await page.getByRole('button', { name: 'Log in' }).click();
    await expect(page).toHaveURL('/dashboard');
  });

  test('wrong password shows an inline error', async ({ page }) => {
    await page.goto('/login');
    await page.getByLabel('Email').fill('demo@example.com');
    await page.getByLabel('Password').fill('WrongPassword!');
    await page.getByRole('button', { name: 'Log in' }).click();
    await expect(page.getByText('Invalid email or password')).toBeVisible();
  });

  test('empty submission shows validation without a network call', async ({ page }) => {
    let apiCalled = false;
    page.on('request', (req) => {
      if (req.url().includes('/api/login')) apiCalled = true;
    });

    await page.goto('/login');
    await page.getByRole('button', { name: 'Log in' }).click();
    await expect(page.getByText('Email is required')).toBeVisible();
    expect(apiCalled).toBe(false);
  });
});

The third test translating "confirm no API call fires" into a real page.on('request', ...) listener rather than a vague comment is exactly the kind of detail that shows the agent actually used browser_network_requests during its live investigation instead of assuming client-side validation behavior.

Tips
- When a requirement is about what should not happen (no network call, no navigation), explicitly ask the agent to verify it live via browser_network_requests before writing the assertion — it's easy for a model to assume rather than confirm.
- Keep negative-case tests (wrong password, empty fields) in the same describe block as the happy path; they share setup and read as a coherent spec for the whole login surface.
- Re-run the empty-submission test a few times — client-side validation timing (debounced validators, async blur handlers) is a common source of flakiness the live session might not surface.

Comparing Playwright MCP Test Quality Between Gemini CLI and Claude Code

Running the same prompts through both tools on a mid-sized React app (a real internal comparison, not a synthetic benchmark) surfaces a few consistent patterns:

Locator discipline. Claude Code defaults to getByRole/getByLabel more consistently out of the box. Gemini CLI produces good locators too, but slips into data-testid-first patterns more often when a role/label pairing is ambiguous — not wrong, just a different default bias, and one that matters if your codebase doesn't already have data-testid coverage everywhere.

Handling of negative/edge cases. Gemini CLI's larger context window is a genuine advantage when you feed it detailed multi-criteria requirements (as in the saved-addresses example) — it tracks more distinct cases without losing earlier ones. For a single terse instruction with no explicit edge-case list, both tools produce similarly shallow happy-path-only coverage; the context-window advantage only shows up when you actually use it by writing detailed prompts.

Multi-tab and cross-navigation flows. Both handle browser_tab_new/browser_tab_select correctly for flows like "click a link that opens in a new tab, verify it, close it, continue in the original tab" — this is standard MCP tool usage and neither showed meaningful gaps.

Debuggability after a failure. Claude Code's VS Code extension sidebar gives a faster path to understanding a bad locator choice, as covered in the Claude Code topic. Gemini CLI's terminal-only interface means you're reading narrated text explanations of tool calls, which is workable but slower to scan.

Cost/latency trade-off. Gemini 2.5 Pro's larger context comes with a real cost and latency profile difference for very long sessions — a 20+ step live-browser walkthrough accumulates a lot of accessibility-snapshot tokens, and Gemini CLI sessions of that length are noticeably slower to respond per turn than an equivalent Claude Code session with more aggressive context trimming.

The practical takeaway isn't "one is strictly better" — it's that Gemini CLI rewards detailed, criteria-rich prompts more than terse ones, while Claude Code produces solid output even from shorter prompts due to stronger built-in defaults.

Tips
- If you're standardizing on Gemini CLI, invest in writing detailed acceptance-criteria-style prompts — that's where its context advantage actually pays off.
- Don't assume data-testid-heavy generated locators from Gemini CLI are wrong — if your codebase already uses testids as the primary hook convention, that's arguably the more correct choice than forcing getByRole everywhere.
- For very long multi-step flows, expect Gemini CLI sessions to run slower per turn; break work into smaller sessions if turnaround time matters more than one-shot completeness.

Tips

Tips
- Set an above-default MCP timeout in .gemini/settings.json for any app with non-trivial page load times — it's the single most common early setup failure.
- Take advantage of the large context window by writing detailed, multi-criteria prompts rather than terse action lists — that's where Gemini CLI's generation quality noticeably improves.
- Cross-check network-dependent assertions (should/shouldn't call an API) by explicitly asking the agent to confirm via browser_network_requests rather than trusting an assumption-based assertion.