·

Real World Workflow Container Debugging And Image Optimization

Walk through a real production workflow that uses Docker MCP so your AI agent can manage containers, images, and logs end to end.

This topic ties the whole module together into one continuous session, the kind you'd actually run when a service starts misbehaving and you also want to leave it in better shape than you found it. The scenario: a Node.js API container that's crash-looping in a staging Compose stack, and once it's fixed, a pass to shrink the production image and tighten its build cache before shipping. Any of the clients from this module — Claude Code, OpenCode, Gemini CLI, Cursor — can drive this; the prompts below are client-agnostic.

Workflow Overview: From Failing Container to Optimized Production Image

The stack: three services in docker-compose.yml — api (Node/Express), db (Postgres), redis (session cache). Alerting fired because api is restarting every few seconds in staging.

services:
  api:
    build: .
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
      - DATABASE_URL=${DATABASE_URL}
      - REDIS_URL=redis://redis:6379
    depends_on:
      - db
      - redis
  db:
    image: postgres:16-alpine
    environment:
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
    volumes:
      - db-data:/var/lib/postgresql/data
  redis:
    image: redis:7-alpine
volumes:
  db-data:

The workflow has three phases, each with a clear exit condition before moving to the next:

  1. Reproduce and collect — get the crash-loop's exact shape: exit code, restart count, logs, and current config, before forming any hypothesis.
  2. Root cause across layers — the bug could be in the Dockerfile, the compose config, or the application/runtime config, and you don't know which yet; check all three systematically rather than guessing.
  3. Rebuild, verify, and shrink — once the crash is fixed and confirmed, use the same session to reduce image size and tighten build cache before calling it done.

Skipping straight to "let's just add a healthcheck and depends_on condition" without confirming the actual root cause first is the most common way this kind of session goes sideways — the fix works by accident (timing changes just enough) while the real bug ships anyway.

Tips
- Resist the urge to apply a plausible-looking fix before Phase 1 is complete — restart count and exit code alone often rule out entire categories of hypothesis in seconds.
- Keep the three phases as explicit checkpoints in your prompts — a session that jumps straight to "fix it" tends to produce a fix for the symptom the agent noticed first, not necessarily the actual cause.
- Treat the image optimization phase as a required step of the workflow, not an optional afterthought — a fix that reintroduces a bloated image is only half the job.

Step 1: Reproducing the Failure and Collecting Logs, Exit Codes, and Inspect Output

The api service in the staging compose stack is restarting repeatedly.
Inspect it, tell me the exit code, restart count, and OOMKilled status,
then fetch the last 50 log lines.

"State": {
  "Status": "restarting",
  "ExitCode": 1,
  "OOMKilled": false,
  "RestartCount": 22,
  "FinishedAt": "2026-08-21T11:03:14Z"
}

Error: connect ECONNREFUSED 172.19.0.2:5432
    at TCPConnectWrap.afterConnect (node:net:1595:16)
    at Object.<anonymous> (/app/dist/db.js:12:5)
Unhandled promise rejection, exiting.

Exit code 1, not 137 — this rules out an OOM kill outright, so the memory-limit line of investigation from earlier modules doesn't apply here. The error is a Postgres connection refusal at startup.

Inspect the db container's health and state. Is Postgres actually up
and accepting connections at this point?

"State": { "Status": "running", "Health": { "Status": "healthy" } }

db reports healthy — so this isn't (yet) the "app doesn't wait for the database" pattern covered in Topic 3, because the database genuinely is up. Something else is preventing the connection.

Inspect the api container's environment variables. Is DATABASE_URL set,
and does its host/port match what db actually exposes?

"Env": [
  "NODE_ENV=production",
  "DATABASE_URL=postgres://postgres:secret@localhost:5432/app",
  "REDIS_URL=redis://redis:6379"
]

There it is: DATABASE_URL points at localhost:5432, not db:5432. Inside a container, localhost refers to the container's own network namespace — there's no Postgres listening there. This is a configuration bug, not a timing or resource bug, and it would have survived any depends_on/healthcheck fix untouched.

Tips
- Always check the actual Env values on the failing container, not just what you believe you set in .env — a stale shell export, a Compose variable substitution gap, or a wrong .env file being loaded can silently override what you think is configured.
- Rule out OOM and dependency-startup timing early with two cheap checks (exit code, dependent service health) before assuming the bug is in your compose orchestration — both are one inspect_container call each.
- Collect all three data points — exit code, dependency health, and env vars — before forming a hypothesis; each one alone is ambiguous, but together they usually point at exactly one cause.

Step 2: AI Root Cause Analysis Across Dockerfile, Compose, and Runtime Config

With DATABASE_URL identified as pointing at the wrong host, trace it back to where that value actually originates.

Read the .env file and docker-compose.yml in this project. Where does
DATABASE_URL's value come from, and why would it resolve to localhost
instead of the "db" service name?

DATABASE_URL=postgres://postgres:secret@localhost:5432/app

The root cause: .env was written for running the API directly on the host during early development (before Compose existed for this project) and never updated when the service moved into the stack. Compose interpolates ${DATABASE_URL} from .env as-is — it has no way to know the value should differ inside the container network.

Confirm this hypothesis: if I change DATABASE_URL's host from localhost
to "db" (the compose service name), will DNS resolution work correctly
inside the api container's network namespace?

Exec into api and run: getent hosts db

172.19.0.2      db

Confirmed — Compose's built-in DNS resolves the service name correctly; the .env value was simply wrong for this context. This is also a good moment to check whether the Dockerfile itself contributes to the confusion, since a stale ENV DATABASE_URL=... baked into the image would override the compose-level value in a way that's harder to spot:

Check the Dockerfile — is DATABASE_URL hardcoded with an ENV
instruction anywhere that could shadow the compose-level value?

ENV NODE_ENV=production
ENV PORT=3000

Good — the Dockerfile isn't part of the problem. Root cause is isolated to the .env file's stale value, and the compose file itself is otherwise correctly wired. Apply and verify:

DATABASE_URL=postgres://postgres:secret@db:5432/app

Update .env with the corrected DATABASE_URL, bring the stack down and
back up, and confirm api reaches a stable running state with no
restarts over the next 30 seconds.

myapp-api-1    Up 32 seconds (healthy)
RestartCount: 0

Tips
- When a bug could plausibly live in the Dockerfile, the compose file, or a runtime config file, check all three explicitly and rule each in or out — don't stop at the first plausible-looking cause, especially when multiple config layers can shadow each other.
- getent hosts <service-name> from inside a container is the fastest way to confirm or refute a DNS/networking hypothesis — cheaper and more direct than reasoning about compose network topology from the YAML alone.
- After applying a fix, verify stability over a real time window (30+ seconds of no restarts), not just the first successful start — some misconfigurations cause a slow failure that a single healthy check would miss.

Step 3: Rebuilding, Verifying, and Shrinking the Final Image

With the crash fixed, use the same session to address image quality before calling the incident closed. Current Dockerfile:

FROM node:20
WORKDIR /app
COPY . .
RUN npm install
RUN npm run build
ENV NODE_ENV=production
ENV PORT=3000
EXPOSE 3000
CMD ["node", "dist/server.js"]

Now that api is stable, review this Dockerfile for size and build cache
issues, and check current image size with docker images.

REPOSITORY   TAG    SIZE
myapp-api    dev    1.14GB

Rewrite as a multi-stage build: separate dependency install, build, and
a minimal alpine runtime stage. Keep NODE_ENV and PORT. Add a non-root
user. Rebuild and show the new size plus docker history.


FROM node:20-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev

FROM node:20-alpine AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-alpine AS runner
WORKDIR /app
ENV NODE_ENV=production
ENV PORT=3000
RUN addgroup -S app && adduser -S app -G app
COPY --from=deps /app/node_modules ./node_modules
COPY --from=build /app/dist ./dist
COPY package.json ./
USER app
EXPOSE 3000
CMD ["node", "dist/server.js"]

REPOSITORY   TAG      SIZE
myapp-api    dev      1.14GB
myapp-api    dev-v2   162MB

Confirm functional parity: bring the stack up using the new image,
run the same DATABASE_URL/Redis connectivity checks from earlier, and
confirm api still reaches "healthy" with the multi-stage build.

myapp-api-1    Up 11 seconds (healthy)

One more pass worth doing before shipping — check whether .dockerignore exists, since COPY . . in the build stage without one drags .git, node_modules, and local .env files into the build context unnecessarily:

Does this project have a .dockerignore? If not, create one excluding
.git, node_modules, .env*, and common editor/OS files. Rebuild and
confirm the build context size drops.

.git
node_modules
.env*
*.log
.DS_Store

Sending build context to Docker daemon: before 340MB, after 4.2MB

A 340 MB build context on every single build — even one that hits the cache and doesn't need most of that data — adds real time to every iteration, especially over a slower connection to a remote Docker context. This is an easy, high-leverage fix that's unrelated to the original incident but directly improves the team's daily build experience.

Closing the loop:

Summarize this session as an incident postmortem: root cause, fix,
image size improvement, and the .dockerignore change, suitable for a
PR description and a team Slack update.

A representative summary the agent produces:

Root cause: .env set DATABASE_URL to localhost, a leftover from pre-Compose local development; inside the container this resolved to the container's own namespace instead of the db service, causing ECONNREFUSED and a crash loop (22 restarts before intervention).
Fix: Updated .env to use the Compose service name db as the Postgres host. Verified via DNS resolution check and 30+ seconds of stable uptime post-fix.
Image optimization: Converted to a 3-stage build (deps/build/runner) on node:20-alpine with a non-root user. Image size reduced from 1.14GB to 162MB. Added .dockerignore, reducing build context from 340MB to 4.2MB per build.

Tips
- Always verify functional parity after an image rewrite using the same checks that confirmed the original fix — a smaller image that silently reintroduces the original bug is a worse outcome than not optimizing at all.
- Treat a missing .dockerignore as a default check on any Dockerfile review, not just when troubleshooting an active incident — it's cheap to add and the build-context-size win compounds over every future build.
- Close every AI-assisted incident session by asking for a written summary while the full context is still in the conversation — reconstructing root cause and fix details after the session ends is much slower than having the agent draft it immediately.

Tips

Tips
- Run the three phases — reproduce/collect, root-cause across all config layers, then rebuild/verify/shrink — as a repeatable template for any container incident, not just this scenario; the discipline of ruling out OOM, dependency timing, and env/config mismatches in order catches most real-world Docker bugs.
- Never treat a fix as confirmed until you've verified it under the same conditions that originally triggered the failure (same stack, same startup sequence, a real time window of stability) — a fix that "should" work is not the same as one you've watched work.
- Fold image optimization into the same session as bug fixes when you touch a Dockerfile anyway — the marginal cost is low once the AI already has full build and runtime context loaded, and it's an easy improvement to skip if you treat it as a separate task for "later."