Publishing JUnit Reports to CI Dashboards

Almost every CI system can render a test report natively — a list of failures with messages, a duration per test, a history of which tests fail most — and almost every JavaScript project ships none of the data it needs. The format is JUnit XML, a twenty-year-old schema that both Vitest and Playwright emit with one line of configuration, and the work is less about producing it than about producing it correctly: merging sharded runs so the totals are real, naming suites so failures are attributable, and attaching enough context that a red dashboard entry answers the question rather than starting a search. This guide covers that end to end, and follows test observability and reporting.

Root Cause Analysis

Teams skip JUnit output for an understandable reason: the console already tells you what failed, and the report seems redundant. It stops being redundant the moment a failure has to be understood by somebody who was not watching the run — a reviewer looking at a pull request, an engineer picking up a red nightly build, anyone trying to answer “has this test failed before”.

The second reason is that first attempts often produce a report that is technically valid and practically useless. Sharded runs write one file per shard, so a naive upload registers four separate test runs with a quarter of the tests each; the dashboard’s history then shows wild swings in test count that have nothing to do with the suite. Suite names default to file paths, so a failure reads as src/index.test.ts in a repository with eleven files by that name. And failure messages get truncated to their first line, which for an assertion error is the least informative part.

Underneath both is a mismatch of model. JUnit XML describes suites containing test cases; JavaScript runners describe files containing nested describe blocks. The mapping is not automatic, and the defaults optimise for validity rather than for readability, which is why a little configuration goes a long way.

Reproducible Setup

Both runners need the reporter enabled and an output path that CI can find. Keep it conditional so local runs stay clean.

// vitest.config.ts
import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    reporters: process.env.CI ? ['default', 'junit'] : ['default'],
    outputFile: { junit: './reports/junit/vitest.xml' },
    // Include the full assertion diff rather than only the first line.
    printConsoleTrace: true,
  },
});

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: process.env.CI
    ? [['list'], ['junit', { outputFile: 'reports/junit/playwright.xml' }], ['blob']]
    : [['list']],
});

mkdir -p reports/junit && echo "reports/" >> .gitignore

Implementation

Step 1 — Name suites so a failure is attributable. The default suite name is a file path, which is ambiguous in any repository with repeated file names. Add a project or package prefix, which both runners support.

// packages/ui/vitest.config.ts
export default defineConfig({
  test: {
    name: 'ui',                       // appears as the suite prefix in the XML
    reporters: ['default', 'junit'],
    outputFile: { junit: '../../reports/junit/ui.xml' },
  },
});

Step 2 — Merge sharded Playwright runs before publishing. The blob reporter exists for this: each shard writes a blob, and merge-reports reconstructs a single report with retries and attachments intact.

# .github/workflows/test.yml
jobs:
  e2e:
    strategy:
      matrix: { shard: [1, 2, 3, 4] }
    steps:
      - run: npx playwright test --shard=${{ matrix.shard }}/4 --reporter=blob
      - uses: actions/upload-artifact@v4
        with: { name: blob-${{ matrix.shard }}, path: blob-report }

  report:
    needs: e2e
    if: always()
    steps:
      - uses: actions/download-artifact@v4
        with: { pattern: blob-*, path: all-blobs, merge-multiple: true }
      - run: npx playwright merge-reports --reporter=junit ./all-blobs > reports/junit/playwright.xml

Step 3 — Merge Vitest shards by concatenating suites. Vitest has no blob equivalent, but JUnit XML merges structurally: collect the testsuite elements from each file into one testsuites root and recompute the totals.

// scripts/merge-junit.ts
import { readFileSync, writeFileSync, readdirSync } from 'node:fs';
import { XMLParser, XMLBuilder } from 'fast-xml-parser';

const parser = new XMLParser({ ignoreAttributes: false, attributeNamePrefix: '@' });
const builder = new XMLBuilder({ ignoreAttributes: false, attributeNamePrefix: '@', format: true });

const suites: any[] = [];
for (const file of readdirSync('reports/shards')) {
  const doc = parser.parse(readFileSync(`reports/shards/${file}`, 'utf8'));
  const found = doc.testsuites?.testsuite ?? doc.testsuite ?? [];
  suites.push(...(Array.isArray(found) ? found : [found]));
}

const sum = (key: string) => suites.reduce((n, s) => n + Number(s[key] ?? 0), 0);
const merged = {
  testsuites: {
    '@name': 'vitest',
    '@tests': sum('@tests'),
    '@failures': sum('@failures'),
    '@time': suites.reduce((n, s) => n + Number(s['@time'] ?? 0), 0).toFixed(3),
    testsuite: suites,
  },
};

writeFileSync('reports/junit/vitest.xml', builder.build(merged));

Step 4 — Publish with an action that renders, rather than merely storing. An uploaded artifact is an archive; a test-report action turns the same file into a readable check on the pull request.

      - uses: dorny/test-reporter@v1
        if: always()
        with:
          name: Test results
          path: reports/junit/*.xml
          reporter: java-junit
          fail-on-error: false

Step 5 — Always publish, including on failure. The report matters most when the run is red, and a step without if: always() is skipped precisely then. This is the single most common mistake in a reporting pipeline.

There is a related trap worth naming: a job that is skipped entirely because an earlier job failed will not publish either, however carefully its own steps are guarded. In a fan-out and fan-in pipeline the reporting job needs if: always() at the job level as well as on its steps, or a single failed shard silently removes the report for the whole run — which is, once again, exactly the run you wanted the report for.

Verification

Check the XML is well formed and carries the totals you expect before trusting any dashboard built on it.

xmllint --noout reports/junit/vitest.xml && \
xmllint --xpath 'string(/testsuites/@tests)' reports/junit/vitest.xml
# 412

Then confirm the merged total matches the sum of the shards, which is the assertion that catches a partially-downloaded artifact or a shard that failed to upload.

for f in reports/shards/*.xml; do xmllint --xpath 'string(/testsuites/@tests)' "$f"; echo; done | paste -sd+ | bc
# 412            ← must equal the merged total above

Finally, verify a failure renders usefully. Break a test deliberately and read the published report: the entry should name the suite, the test and the assertion, and the message should include the expected and received values rather than a bare AssertionError.

npx vitest run --reporter=junit --outputFile=/tmp/x.xml || true
xmllint --xpath '//testcase[failure]/failure/@message' /tmp/x.xml
# message="expected 3 to be 4 // Object.is equality"

Troubleshooting

Symptom: the dashboard shows far fewer tests than the run executed. Diagnosis: only one shard’s report was published, or the merge silently found a single file because the download pattern did not match. Fix: assert the merged total against the shard sum as in the verification step, and fail the job if they differ rather than publishing a partial truth.

Symptom: every failure reads AssertionError with no detail. Diagnosis: the reporter is writing only the error name because the message was multi-line and got truncated at the first newline. Fix: check the failure element body rather than the attribute — most renderers show the body, and the useful diff lives there.

Symptom: the report is missing when the run fails. Diagnosis: the publish step is conditional on the test step succeeding, which is the default in most pipelines. Fix: add if: always() to every step between the tests and the publish, including the artifact upload and download.

Symptom: suite names are absolute paths from the runner machine. Diagnosis: the reporter is emitting resolved paths, which differ between local and CI and make history comparisons fail. Fix: set the root directory explicitly so paths are repository-relative, and prefix suites with the project name as in Step 1.

FAQ

Related

• Back to Test Observability & Reporting
• Annotating pull requests with test failures — the same data, placed where it is read.
• Tracking test duration trends over time — what to do with the timings in these reports.
• Sharding Vitest across GitHub Actions runners — the runs being merged here.