Choosing Mutation Score Targets Per Module

A single mutation score for a whole repository is the same mistake as a single coverage percentage: it averages code where a surviving mutant means a mispriced order with code where it means a slightly different log line. The useful configuration sets different bars for different kinds of code, derives each bar from what that area actually achieves today, and raises it deliberately rather than aspirationally. This guide is for tech leads deciding what a mutation gate should require, and for engineers being asked to hit a number someone chose in a meeting. It covers classifying modules by consequence, deriving a floor from observed data, expressing per-module thresholds in Stryker, and ratcheting the bar without turning the gate into an obstacle. It assumes the workflow in mutation testing and assertion quality.

Root Cause Analysis

Uniform targets fail in both directions at once. Set at 80, they are trivially met by presentational code whose mutants are mostly equivalent, while being genuinely hard for a date-arithmetic module where the bar should be higher still. The average hides both facts, and the team ends up writing pointless tests in one place while a genuinely under-tested module sits comfortably above the line.

The second failure is targets chosen as round numbers. Eighty percent has no more claim on truth than seventy-three; it just looks tidy. A target should come from two inputs: the consequence of an undetected defect in that module, and the score the module reaches today. The first says where you want to be, the second says what the next step is. A gate set far above today’s floor blocks work for weeks and gets removed; a gate set at today’s floor prevents regression, which is most of the value.

The third is that scores move for reasons unrelated to test quality. Deleting dead code raises the score; adding a new module lowers it. A gate that reads the absolute number will fire on both, so the gate should compare against a stored baseline for the same module rather than a constant.

Reproducible Setup

Get per-file scores out of the JSON report so the classification is based on data rather than impressions.

// scripts/per-module.ts
import { readFileSync } from 'node:fs';

const report = JSON.parse(readFileSync('reports/mutation/mutation.json', 'utf8'));

const rows = Object.entries(report.files).map(([file, data]: [string, any]) => {
  const killed = data.mutants.filter((m: any) => m.status === 'Killed').length;
  const timeout = data.mutants.filter((m: any) => m.status === 'Timeout').length;
  const survived = data.mutants.filter((m: any) => m.status === 'Survived').length;
  const total = killed + timeout + survived;
  return { file, total, score: total ? ((killed + timeout) / total) * 100 : 0 };
});

for (const r of rows.sort((a, b) => a.score - b.score)) {
  console.log(`${r.score.toFixed(1).padStart(6)}  ${String(r.total).padStart(4)}  ${r.file}`);
}

npx tsx scripts/per-module.ts
#  41.2    68  src/domain/scheduling.ts
#  63.9    97  src/domain/tax.ts
#  88.0   125  src/domain/pricing.ts
#  94.1    34  src/lib/permissions/rules.ts

That ordering is the whole conversation: scheduling is the module to work on, and pricing is already close to where a money-adjacent module should be.

Implementation

Step 1 — Classify by consequence, in writing. Put the classification in the repository next to the configuration so it is reviewable and so nobody has to remember why a module has the bar it does.

// mutation-targets.json
{
  "src/domain/pricing/**":      { "tier": "critical", "target": 90 },
  "src/lib/permissions/**":     { "tier": "critical", "target": 90 },
  "src/domain/**":              { "tier": "core",     "target": 80 },
  "src/adapters/**":            { "tier": "adapter",  "target": 65 },
  "src/components/**":          { "tier": "excluded", "target": null }
}

Step 2 — Derive the starting bar from today’s floor, not from the target. For each tier, take the lowest score currently observed in that tier and set the gate a couple of points below it. The target is where you are heading; the gate is what stops you sliding back.

// scripts/derive-gate.ts
const floors = { critical: 88.0, core: 41.2, adapter: 60.5 };
const gate = Object.fromEntries(
  Object.entries(floors).map(([tier, floor]) => [tier, Math.floor(floor) - 2]),
);
console.log(gate);   // { critical: 86, core: 39, adapter: 58 }

Step 3 — Run Stryker per tier so each gets its own threshold. Stryker’s thresholds.break is global to a run, so per-module bars mean per-module runs — cheap, because each run’s glob is small.

#!/usr/bin/env bash
# scripts/mutate-tiers.sh
set -euo pipefail

npx stryker run --mutate "src/domain/pricing/**/*.ts" --reporters clear-text \
  --thresholds.break 86

npx stryker run --mutate "src/domain/**/*.ts,!src/domain/pricing/**" \
  --reporters clear-text --thresholds.break 39

npx stryker run --mutate "src/adapters/**/*.ts" --reporters clear-text \
  --thresholds.break 58

Step 4 — Ratchet on success, never on hope. When a run comes in comfortably above its gate, raise the gate to just below the new floor. Automating the raise keeps the ratchet honest and removes the quarterly argument about numbers.

// scripts/ratchet.ts — raise the gate when the score clears it by a margin
import { readFileSync, writeFileSync } from 'node:fs';

const MARGIN = 3;
const gates = JSON.parse(readFileSync('mutation-gates.json', 'utf8')) as Record<string, number>;
const scores = JSON.parse(readFileSync('reports/mutation/tier-scores.json', 'utf8')) as Record<string, number>;

let changed = false;
for (const [tier, score] of Object.entries(scores)) {
  const next = Math.floor(score) - 1;
  if (score - gates[tier] > MARGIN && next > gates[tier]) {
    console.log(`ratchet ${tier}: ${gates[tier]} → ${next}`);
    gates[tier] = next;
    changed = true;
  }
}
if (changed) writeFileSync('mutation-gates.json', JSON.stringify(gates, null, 2) + '\n');

Step 5 — Treat new modules as exempt until they have a baseline. A module added this week has no observed floor, so gating it on a tier constant blocks the work that created it. Give it a grace period, record its first full-run score, and gate from then on.

Verification

Verify the gate fires for the right reason. Weaken a test in a critical module, run its tier, and confirm the build fails with the module named.

npx stryker run --mutate "src/domain/pricing/**/*.ts" --thresholds.break 86
# Mutation score: 83.20%
# ERROR Stryker: Final mutation score 83.20 under break threshold 86, setting exit code to 1

Then verify it does not fire for the wrong reason. Add a new, well-tested file to the same tier and confirm the score and the gate both behave — a gate that fails whenever the codebase grows will be disabled within a month.

Finally, confirm the tier classification matches reality by sampling. Pick two files from the critical tier and two from the adapter tier, read one survivor from each, and ask whether the consequence matches the bar. Classification drifts as code moves, and this five-minute check catches it long before anyone complains about the gate.

Troubleshooting

Symptom: a tier’s score drops every time someone deletes tests for removed features. Diagnosis: the score is a ratio, so removing killed mutants alongside their code changes the denominator. Fix: compare against the stored baseline for the same file set rather than a global constant, and refresh the baseline on the nightly full run.

Symptom: the critical tier never moves despite effort. Diagnosis: the remaining survivors are equivalent mutants, which no assertion can kill. Fix: mark them explicitly so they leave the denominator, and accept a ceiling below 100. If more than a few percent of a module is equivalent, that is a hint the code has redundant guards worth simplifying.

Symptom: engineers game the gate by excluding files. Diagnosis: exclusion is easier than writing an assertion and nothing reviews it. Fix: require exclusions to live in the committed classification file from Step 1, where a reviewer sees them in the diff, rather than as inline comments scattered through the source.

FAQ

Related

• Back to Mutation Testing & Assertion Quality
• Finding weak assertions with mutation scores — the work that raises a tier’s floor.
• Keeping mutation testing fast in CI — per-tier runs without a slow pipeline.
• Enforcing coverage thresholds in a monorepo — the same per-area reasoning for coverage.