Reusing Build Artifacts Between CI Jobs
A pipeline that builds the application in the unit job, again in the end-to-end job, and a third time in the visual regression job is not just slow — it is testing three different artifacts, any of which could differ from what ships. Building once and passing the output downstream fixes both problems at the same time: it removes the repeated cost, and it guarantees that every job examines the same bytes. This guide covers producing a build artifact, restoring it in downstream jobs, deciding what belongs in it, and — the part most pipelines skip — verifying that the artifact a job restored is the one the build job produced. It sits under continuous integration test orchestration.
Root Cause Analysis
Repeated builds happen because each job is written independently, and each author quite reasonably starts with “install, build, test”. Nothing in a workflow file makes the duplication visible, and the pipeline still passes, so it survives.
The cost is not only wall clock. Two builds of the same commit can differ: a dependency resolved from a floating range, a timestamp baked into a bundle, a code generator that orders output non-deterministically. When the end-to-end job builds its own copy, a failure there cannot be assumed to reproduce against the artifact you will ship, and the investigation starts from an uncertain premise.
There is also a correctness argument specific to testing. A build that runs only in a test job has never been exercised in the configuration that produces the release artifact — different environment variables, different mode, sometimes a different bundler target. Testing the release build is the whole point of having an end-to-end tier, and separate builds quietly give that up.
Reproducible Setup
One build job, producing a directory and a manifest that downstream jobs can check.
# .github/workflows/pipeline.yml
jobs:
build:
runs-on: ubuntu-latest
outputs:
digest: ${{ steps.digest.outputs.value }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: 20, cache: npm }
- run: npm ci
- run: npm run build
- id: digest
run: echo "value=$(find dist -type f -exec sha256sum {} + | sort | sha256sum | cut -d' ' -f1)" >> "$GITHUB_OUTPUT"
- uses: actions/upload-artifact@v4
with:
name: dist-${{ github.sha }}
path: dist
retention-days: 3
The digest is the part worth adding even though nothing forces you to: it is what lets a downstream job assert that it received the artifact it expected rather than a stale one from a previous run.
Implementation
Step 1 — Decide what belongs in the artifact. Include what the tests need to run against; exclude what they can regenerate. Source maps usually belong; node_modules usually does not, because restoring a dependency cache is faster than downloading a hundred-megabyte artifact.
- uses: actions/upload-artifact@v4
with:
name: dist-${{ github.sha }}
path: |
dist
public
!dist/**/*.map.gz
Step 2 — Restore it in each downstream job and verify the digest. The verification is three lines and turns a whole class of confusing failures into an immediate, explicit one.
e2e:
needs: [build]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: 20, cache: npm }
- run: npm ci
- uses: actions/download-artifact@v4
with: { name: dist-${{ github.sha }}, path: dist }
- name: Verify the artifact is the one that was built
run: |
actual=$(find dist -type f -exec sha256sum {} + | sort | sha256sum | cut -d' ' -f1)
[ "$actual" = "${{ needs.build.outputs.digest }}" ] || { echo "artifact mismatch"; exit 1; }
- run: npx playwright test
Step 3 — Serve the built artifact rather than a dev server. This is the point of the exercise: the end-to-end suite must exercise the production bundle, not a differently-compiled development one.
// playwright.config.ts
export default defineConfig({
webServer: {
command: 'npx serve -s dist -l 3000', // the built output, not `npm run dev`
url: 'http://localhost:3000',
reuseExistingServer: false,
timeout: 60_000,
},
use: { baseURL: 'http://localhost:3000' },
});
Step 4 — Keep the artifact small enough that transfer does not eat the saving. Upload and download are not free; on a large bundle they can cost more than the build they replaced. Measure both, and compress before uploading when the artifact is many small files.
du -sh dist # 148M ← too big to move three times
tar -czf dist.tgz dist && du -sh dist.tgz # 31M ← upload this instead
Step 5 — Reuse the same artifact for the deployment. If the pipeline builds again at release time, everything tested above was a rehearsal. Promote the tested artifact instead.
deploy:
needs: [build, e2e]
runs-on: ubuntu-latest
steps:
- uses: actions/download-artifact@v4
with: { name: dist-${{ github.sha }}, path: dist }
- run: npx wrangler deploy # ships exactly what the tests ran against
Step 6 — Decide deliberately what happens when the build fails. Downstream jobs depending on the build will be skipped, which is usually right: there is nothing to test. The exception is a lint or type-check job that does not need the artifact at all, and making it independent of the build lets a developer see every problem in one run rather than discovering the type error only after fixing the build.
types:
runs-on: ubuntu-latest # deliberately not `needs: [build]`
steps:
- uses: actions/checkout@v4
- run: npm ci && npx tsc --noEmit
Verification
Verify that no job builds twice by grepping the workflow for build invocations — it is easy for one to creep back in during an unrelated change.
grep -n "npm run build" .github/workflows/*.yml
# pipeline.yml:14: - run: npm run build ← exactly one
Then verify the digest check actually fires. Corrupt the restored artifact deliberately and confirm the job stops with a clear message rather than running tests against something unexpected.
echo "// tampered" >> dist/assets/index.js
actual=$(find dist -type f -exec sha256sum {} + | sort | sha256sum | cut -d' ' -f1)
[ "$actual" = "$EXPECTED" ] || echo "artifact mismatch"
# artifact mismatch
Finally, verify the end-to-end suite is genuinely serving the build. A quick assertion on a production-only characteristic — a hashed filename, an absent development banner — catches the case where someone quietly reinstated the dev server because it was more convenient locally.
import { test, expect } from '@playwright/test';
test('the suite is running against the production build', async ({ page }) => {
const response = await page.goto('/');
const html = (await response!.text());
expect(html).toMatch(/assets\/index-[a-z0-9]{8}\.js/); // hashed, i.e. built
});
Troubleshooting
Symptom: the download step finds no artifact. Diagnosis: the name includes a value that differs between jobs, or the upload was skipped because an earlier step failed. Fix: derive the name from the commit SHA as above so both sides compute the same string, and confirm the upload step runs even when tests fail if downstream jobs need it.
Symptom: transferring the artifact is slower than rebuilding. Diagnosis: many small files, which upload poorly. Fix: archive before uploading and extract after downloading; a single compressed file moves far faster than ten thousand small ones, and the extra step costs a couple of seconds.
Symptom: tests pass against the artifact and the deployment breaks. Diagnosis: the release step rebuilds rather than promoting. Fix: deploy the artifact the tests consumed, as in Step 5 — a rebuild at release time makes every earlier test a statement about a different file.
Symptom: the digest differs between two runs of the same commit. Diagnosis: a non-reproducible build — an embedded timestamp, a random chunk order, an unpinned dependency. Fix: this is worth chasing, because it also means your ability to reason about what shipped is weaker than you thought; start by diffing the two outputs to find which file moved.
FAQ
Should the artifact include node_modules?
Almost never. It is large, it compresses poorly, and it is perfectly reconstructible from the lockfile, which is what the dependency cache is for. The exception is a job that must run without network access, where bundling dependencies into the artifact is the only option.
How long should artifacts be retained?
Days, not weeks, for pipeline artifacts — they exist to move data between jobs in one run. Release artifacts are a separate concern with their own retention. Short retention keeps storage costs sane and makes it obvious that these files are not an archive.
Does this work across workflows, not just jobs?
Usually yes, with a lookup step, but be careful: fetching an artifact from another workflow run means the identity check matters more, not less, because now you are trusting a run you did not orchestrate. Always verify the digest and the commit before testing against it.
What about monorepos where each package builds separately?
Upload per package, keyed on the package name and the commit, and let each downstream job download only what it needs. This pairs naturally with task-level caching, described in caching test results with Turborepo remote cache, where the build task’s outputs are already declared.
Related
- Back to Continuous Integration Test Orchestration
- Caching dependencies and test artifacts in CI — the cache half of the same problem.
- Right-sizing CI runners for test throughput — what to do once the serial floor is smaller.
- Running end-to-end tests against preview deployments — the alternative when the platform builds for you.