Running Tests in Docker for Reproducible CI

“It passes on my machine” is a statement about an environment nobody wrote down. Running the suite inside a container writes it down: the Node version, the system libraries, the browser build, the locale and the time zone all become explicit and identical everywhere. The cost is a layer of indirection and some care about caching and resource limits, both of which are manageable. This guide covers building a test image for a Vitest and Playwright suite, keeping rebuilds fast, running it in CI, setting limits so the container behaves like the runner it will live on, and reproducing a CI failure locally in one command. It sits under continuous integration test orchestration.

Root Cause Analysis

Environment differences cause a specific and frustrating class of failure: tests that pass in one place and fail in another, for reasons unrelated to the code. The usual culprits are few and predictable. Time zone and locale differ between a developer’s machine and a CI runner, so date formatting assertions diverge. Font availability differs, so screenshot comparisons fail. The browser build differs, so a rendering detail shifts by a pixel. And the system libraries a headless browser needs are present on one image and missing on another.

Containers remove all four by construction, but they introduce two of their own if used carelessly. A container reports the host’s core count while being limited to fewer, so every default worker calculation is wrong — the cause of a great many “flaky in CI only” reports. And an unpinned base image changes underneath you, which reintroduces exactly the drift you containerised to avoid.

The third consideration is speed. A naive image rebuilds dependencies on every source change because the layers are ordered wrongly, turning a thirty-second test run into a four-minute one. Layer ordering is the whole of the fix, and it is worth getting right on the first day.

Reproducible Setup

Start from Playwright’s official image, which already carries the browsers and their system dependencies at matching versions — the single largest source of container pain when assembled by hand.

# Dockerfile.test
FROM mcr.microsoft.com/playwright:v1.47.0-jammy

ENV TZ=UTC \
    LANG=en_US.UTF-8 \
    CI=true \
    NODE_ENV=test

WORKDIR /app

# Dependencies first: this layer is reused whenever the lockfile is unchanged.
COPY package.json package-lock.json ./
RUN npm ci

# Source last: a code change invalidates only this layer.
COPY . .

CMD ["npx", "vitest", "run"]

docker build -f Dockerfile.test -t app-tests:local .
docker run --rm app-tests:local

Pinning the image to an exact Playwright version rather than a floating tag is what makes the browser build reproducible; a moving tag silently upgrades Chromium and invalidates every screenshot baseline at an unpredictable moment.

Implementation

Step 1 — Order layers so a source change does not reinstall dependencies. Copy the lockfile, install, then copy the source. Reversing these two costs several minutes on every run.

COPY package.json package-lock.json ./
RUN npm ci                      # cached while the lockfile is unchanged
COPY . .                        # invalidated by any source edit

Step 2 — Set resource limits that match the runner, and read them in the test config. The container reporting host cores is the classic trap, and the fix is to be explicit on both sides.

docker run --rm --cpus=2 --memory=4g \
  -e TEST_WORKERS=2 \
  app-tests:local

// vitest.config.ts — trust the environment over the reported core count
import os from 'node:os';

const workers = Number(process.env.TEST_WORKERS ?? Math.min(os.cpus().length, 2));

export default defineConfig({
  test: { poolOptions: { threads: { maxThreads: workers, minThreads: workers } } },
});

Step 3 — Use a compose file for suites that need services. A database or a message broker belongs in the same definition as the tests, so the whole environment starts and stops together and behaves identically everywhere.

# docker-compose.test.yml
services:
  postgres:
    image: postgres:16.4-alpine
    environment: { POSTGRES_PASSWORD: test, POSTGRES_DB: app_test }
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 2s
      retries: 30

  tests:
    build: { context: ., dockerfile: Dockerfile.test }
    depends_on:
      postgres: { condition: service_healthy }
    environment:
      DATABASE_URL: postgres://postgres:test@postgres:5432/app_test
    command: npx vitest run

Step 4 — Run the same image in CI, with a registry cache so it is not rebuilt each time. The point is defeated if CI builds a different image from the one developers use.

# .github/workflows/test.yml
      - uses: docker/setup-buildx-action@v3
      - uses: docker/build-push-action@v6
        with:
          file: Dockerfile.test
          tags: ghcr.io/${{ github.repository }}/tests:${{ github.sha }}
          push: true
          cache-from: type=registry,ref=ghcr.io/${{ github.repository }}/tests:cache
          cache-to: type=registry,ref=ghcr.io/${{ github.repository }}/tests:cache,mode=max
      - run: docker run --rm --cpus=2 --memory=4g ghcr.io/${{ github.repository }}/tests:${{ github.sha }}

Step 5 — Give developers a one-command path to reproduce a CI failure. This is the payoff, and it should be in the README rather than in someone’s shell history.

# scripts/test-in-docker.sh
docker compose -f docker-compose.test.yml run --rm tests "$@"
# ./scripts/test-in-docker.sh npx playwright test specs/checkout.spec.ts

Step 6 — Keep one source of truth for the browser version. The Playwright package version in the lockfile and the image tag must agree, or the browsers shipped in the image will not match the ones the library expects and launching will fail with a message about a missing executable. A check in the build makes the coupling explicit rather than tribal knowledge.

# scripts/check-playwright-version.sh
pkg=$(node -p "require('./package.json').devDependencies['@playwright/test'].replace(/[^0-9.]/g,'')")
img=$(grep -oP 'playwright:v\K[0-9.]+' Dockerfile.test)
[ "$pkg" = "$img" ] || { echo "version mismatch: package $pkg vs image $img"; exit 1; }

Verification

Verify that the container sees the limits you set, rather than the host’s resources — this single check prevents the most common containerised-test failure.

docker run --rm --cpus=2 --memory=4g app-tests:local \
  node -e "console.log('reported cores:', require('os').cpus().length)"
# reported cores: 8      ← the host's count, not the limit
cat /sys/fs/cgroup/cpu.max
# 200000 100000          ← the real limit is 2

Then verify that the environment is genuinely pinned by asserting on it from inside the suite. A test that checks the time zone and locale costs nothing and catches an image change immediately.

import { test, expect } from 'vitest';

test('the environment is the one we pinned', () => {
  expect(Intl.DateTimeFormat().resolvedOptions().timeZone).toBe('UTC');
  expect(process.env.LANG).toBe('en_US.UTF-8');
  expect(process.version.startsWith('v20.')).toBe(true);
});

Finally, verify local and CI agree by running the same image both places on the same commit and comparing results. A difference at this point is a real environment leak — usually a mounted volume or a variable passed in one place and not the other.

Troubleshooting

Symptom: the browser fails to launch inside the container. Diagnosis: missing system libraries, or a sandbox that the container’s security profile blocks. Fix: use the official Playwright image rather than assembling dependencies onto a plain Node base — it exists precisely because that assembly is tedious and version-sensitive.

Symptom: tests are far slower in the container than outside it. Diagnosis: a bind-mounted source directory on a platform where file access across the boundary is slow, or resource limits set too low. Fix: copy the source into the image for CI runs and reserve the mount for local iteration, and check that the memory limit is not forcing constant garbage collection.

Symptom: heap errors appear only in the container. Diagnosis: worker count derived from the host’s core count while memory is limited. Fix: pass the worker count in explicitly as in Step 2, and set NODE_OPTIONS=--max-old-space-size to a value consistent with the container’s memory limit.

Symptom: the image behaves differently after a rebuild with no source change. Diagnosis: a floating base tag, or a package manager pulling a newer transitive dependency. Fix: pin the base image by exact version — a digest is better still — and commit the lockfile; reproducibility is the entire point and a moving tag quietly removes it.

FAQ

Related

• Back to Continuous Integration Test Orchestration
• Right-sizing CI runners for test throughput — choosing the limits this container should enforce.
• Reducing flaky screenshots with deterministic rendering — the failures a pinned browser removes.
• Testing timezone-dependent code with fake timers — controlling in code what the image pins in the environment.