Eliminating Test Order Dependence

An order-dependent test is one that passes when the suite runs in its usual sequence and fails when it does not — because some earlier test left state behind that it silently relies on, or because it leaves state behind that breaks a later one. These are the flakes that appear when you enable parallelism, add a test in the middle of a file, or shard the suite, and they are maddening precisely because the test that fails is rarely the test that is wrong. This guide covers exposing them deliberately with shuffling, bisecting to the culprit pair, and removing the four kinds of shared state that cause almost all of them. It sits under flaky test mitigation.

Root Cause Analysis

Every order dependency is a piece of state that outlives a test. In a JavaScript suite there are four common homes for it, and recognising which one you have is most of the diagnosis.

The module registry is the first. A module imported by one test keeps its top-level state — a cached client, a counter, a memoised value — for every subsequent test in the same worker, so a test that mutates it changes the world for its neighbours. The second is genuine globals: globalThis, window, process.env, and anything attached to them by a library at import time. The third is timers and pending asynchronous work, where a test that resolves after its own assertions have finished lands its side effects inside the next test. The fourth is external state — database rows, files on disk, a mock server’s handler list.

What makes these hard is the asymmetry of blame. The failing test is the victim; the test that leaked is usually green and often far away in the file tree. Debugging the failure alone is therefore unproductive, and the techniques below are all about identifying the pair rather than the failure.

Reproducible Setup

Both runners can randomise order, which is what turns a latent dependency into a reproducible failure.

// vitest.config.ts
export default defineConfig({
  test: {
    sequence: {
      shuffle: { files: true, tests: true },
      // A fixed seed makes a failure reproducible; omit it in the nightly hunt.
      seed: process.env.TEST_SEED ? Number(process.env.TEST_SEED) : undefined,
    },
    isolate: true,
  },
});

# a leaky test usually reveals itself within a handful of shuffled runs
for seed in 1 2 3 4 5 6 7 8; do
  TEST_SEED=$seed npx vitest run --silent >/dev/null || echo "failed with seed $seed"
done
# failed with seed 5

Implementation

Step 1 — Reproduce with a fixed seed. A failing seed is a deterministic reproduction, which turns the problem from a ghost into an ordinary bug.

TEST_SEED=5 npx vitest run --reporter=verbose
# FAIL  src/features/cart/cart.test.ts > applies the member discount
#   expected 45 to be 40

Step 2 — Bisect to the culprit pair. Run the failing test alone: if it passes, something before it is responsible. Halve the set of preceding files repeatedly until one remains.

npx vitest run src/features/cart/cart.test.ts            # passes alone → a predecessor is to blame
npx vitest run src/features/{a,b,c,d}/*.test.ts src/features/cart/cart.test.ts   # still fails
npx vitest run src/features/{a,b}/*.test.ts src/features/cart/cart.test.ts       # passes
npx vitest run src/features/{c,d}/*.test.ts src/features/cart/cart.test.ts       # fails → c or d
npx vitest run src/features/c/*.test.ts src/features/cart/cart.test.ts           # fails → c is the culprit

Step 3 — Fix module-level leakage at the source. The usual offender is a module that caches something on first import. Reset the registry between tests, or better, make the module expose a factory so nothing is cached at module scope at all.

// src/lib/client.ts — the leak: a singleton created on first import
let client: ApiClient | undefined;
export const getClient = () => (client ??= new ApiClient(process.env.API_URL!));

// src/lib/client.ts — the fix: no module-level state
export const createClient = (url = process.env.API_URL!) => new ApiClient(url);

// if the singleton must stay, reset the registry explicitly
import { beforeEach, vi } from 'vitest';

beforeEach(() => {
  vi.resetModules();
});

Step 4 — Snapshot and restore globals rather than setting them. Setting process.env.TZ in one test changes it for every later test in the worker; restoring it in an afterEach costs nothing.

import { beforeEach, afterEach, vi } from 'vitest';

let envSnapshot: NodeJS.ProcessEnv;

beforeEach(() => { envSnapshot = { ...process.env }; });
afterEach(() => {
  process.env = envSnapshot;
  vi.unstubAllGlobals();
  vi.useRealTimers();
});

Step 5 — Make pending work finish before the test does. An unawaited promise or an uncleaned timer lands in whichever test happens to be running when it resolves.

// leaks: the debounce fires during a later test
test('debounces search input', () => {
  typeInto(input, 'shoes');          // schedules a 300ms timer, never awaited
  expect(fetchSpy).not.toHaveBeenCalled();
});

// contained: the timer is owned and disposed by this test
test('debounces search input', async () => {
  vi.useFakeTimers();
  typeInto(input, 'shoes');
  await vi.advanceTimersByTimeAsync(300);
  expect(fetchSpy).toHaveBeenCalledOnce();
  vi.useRealTimers();
});

Step 6 — Keep the shuffle on permanently. Once the suite is clean, randomised order stops new dependencies from accumulating. Running it only during an investigation guarantees the problem returns.

A useful habit once the suite is clean: when a test needs setup that another test also needs, extract it into a fixture rather than relying on the earlier test having run. The distinction is subtle but decisive — a fixture states the dependency and provides it, while an implicit reliance on execution order states nothing and works only by luck. Most order dependencies started life as a reasonable shortcut of exactly this kind.

Verification

Verify by running the suite many times with different seeds and confirming every run agrees. This is the definition of order independence, and it is cheap to check.

for seed in $(seq 1 25); do
  TEST_SEED=$seed npx vitest run --silent >/dev/null || echo "seed $seed failed"
done
# (no output)

Then verify the specific pair is genuinely fixed rather than merely reordered away. Run the two files together, in both orders, and confirm both pass.

npx vitest run src/features/c/settings.test.ts src/features/cart/cart.test.ts
npx vitest run src/features/cart/cart.test.ts src/features/c/settings.test.ts
# both green

Finally, verify isolation itself is not being relied upon to hide the problem. Vitest’s isolate: true gives each file a fresh module registry, which masks module-level leakage between files while leaving it live between tests in the same file. Running once with isolation disabled tells you whether the fix was real.

npx vitest run --no-isolate
# if this fails and the isolated run passes, module state is still leaking

Troubleshooting

Symptom: shuffling produces a different failure every run. Diagnosis: several independent dependencies, which is common in a suite that has never been shuffled. Fix: work them one at a time from a fixed seed; each fix usually removes a family of failures because the same leaky module is behind many of them.

Symptom: the failure disappears when you add a console log. Diagnosis: a timing-sensitive leak — pending asynchronous work whose resolution order changed. Fix: look for unawaited promises and uncleaned timers in the predecessor rather than in the failing test, and prefer fake timers so the ordering is under the test’s control.

Symptom: tests pass locally but fail in CI with the same seed. Diagnosis: the seed controls order but not the worker assignment, so which tests share a worker differs with the worker count. Fix: pin the worker count when reproducing, and treat the combination of seed and worker count as the reproduction recipe.

Symptom: a mock server keeps handlers between tests. Diagnosis: handlers added with server.use persist until reset. Fix: call the reset in an afterEach — this is the single most common external-state leak in suites using request interception, and it is a one-line fix.

FAQ

Related

• Back to Flaky Test Mitigation
• Detecting flaky tests by repeating runs in CI — the hunt that surfaces these.
• Resetting state between tests without slowing CI — the external-state half of the fix.
• Resetting the module registry between tests — the module-state half in depth.