Event-Driven & Queue Mocking

Asynchronous messaging moves a system’s most interesting behaviour out of the request path and into places tests rarely look. An order is accepted synchronously, but the invoice, the warehouse notification and the loyalty points are all produced by consumers reacting to an event minutes later. Those consumers must cope with duplicates, reordering, partial failure and retries — the very conditions a live broker in a test environment almost never produces on demand. This topic belongs to advanced mocking and service isolation patterns and covers how to test event-driven code at the right tier: handlers as plain functions, transports as in-memory fakes, webhooks with genuinely signed payloads, and job queues with controllable time, so that the failure modes asynchronous systems are designed around are the ones your tests actually exercise.

Architectural Scope & Boundaries

The scope is application code that participates in asynchronous messaging: services that publish domain events, consumers that react to them, HTTP endpoints that receive webhooks from third parties, and workers that process jobs from a queue. The common thread is that the trigger for the code is a message rather than a direct call, and the message may arrive late, twice, out of order, or not at all.

The structural decision that makes all of this testable is separating the handler from the transport. A handler is a function that receives a decoded message and performs the business action; it knows nothing about offsets, visibility timeouts or acknowledgements. The consumer is the adapter that pulls messages from the transport, decodes them, calls the handler, and acknowledges or retries. With that split, the handler — where nearly all the logic and nearly all the test cases live — is ordinary code tested with ordinary unit tests.

The consumer adapter has a smaller but important surface: decoding and schema validation, acknowledgement on success, retry or dead-lettering on failure, and concurrency limits. Those are integration concerns, best verified against either an in-memory implementation of the transport interface or, for a handful of tests, a real broker running in a container.

What this topic does not cover is testing the brokers themselves — Kafka’s partition rebalancing, SQS’s delivery guarantees — or load testing throughput. It also stops short of full choreography tests that start every service and publish a real event end to end; those are valuable in small numbers but belong to a dedicated environment rather than the per-commit suite.

The final boundary is contracts. When producer and consumer are owned by different teams, the shape of the message is an interface that can break silently, and the tool for that is not a mock but a contract test — discussed in contract testing, which applies to message payloads just as it does to HTTP.

It is worth being candid about why this area is under-tested in most codebases. Synchronous request handling has an obvious test shape — send a request, check the response — and every framework’s documentation shows it. Asynchronous handling has no equally obvious shape, and the first attempt usually involves starting a broker, publishing a message and sleeping until something happens, which is slow enough and flaky enough that the team quietly stops writing more. The approach here replaces that first attempt with one that is faster than the synchronous equivalent, which is the only reliable way to get the coverage written.

Prerequisites

• Vitest 1.x or 2.x with the node environment
• Handlers written as functions that take a decoded message, separate from transport clients
• A schema for each message type, for example with zod, shared by producer and consumer
• An idempotency key on every message that can cause a side effect
• Docker available in CI for the small number of real-broker integration tests
• Fake timers for jobs with delays, backoff or scheduling

Step-by-Step Implementation

Step 1 — Extract the handler as a plain function. It receives the decoded message and its dependencies, and returns or throws; it does not know which broker delivered the message.

// src/orders/handlers/on-order-placed.ts
import type { OrderPlaced } from '../events';

type Deps = { invoices: InvoiceRepo; mailer: Mailer; processed: ProcessedStore };

export async function onOrderPlaced(event: OrderPlaced, deps: Deps) {
  if (await deps.processed.has(event.eventId)) return { skipped: 'duplicate' };

  const invoice = await deps.invoices.createForOrder(event.orderId, event.totalPence);
  await deps.mailer.send({ to: event.customerEmail, template: 'invoice', data: { invoiceId: invoice.id } });
  await deps.processed.add(event.eventId);
  return { invoiceId: invoice.id };
}

Step 2 — Test the handler’s cases, including duplicates, with in-memory dependencies. This is where the volume goes, and each case runs in microseconds.

// src/orders/handlers/on-order-placed.test.ts
import { test, expect } from 'vitest';
import { onOrderPlaced } from './on-order-placed';
import { anOrderPlaced } from '../../../test/builders/events';
import { memoryDeps } from '../../../test/fakes/order-deps';

test('creates one invoice and one email for a new event', async () => {
  const deps = memoryDeps();
  await onOrderPlaced(anOrderPlaced({ eventId: 'e1' }), deps);
  expect(deps.invoices.all()).toHaveLength(1);
  expect(deps.mailer.sent()).toHaveLength(1);
});

test('does nothing the second time the same event arrives', async () => {
  const deps = memoryDeps();
  const event = anOrderPlaced({ eventId: 'e1' });
  await onOrderPlaced(event, deps);
  expect(await onOrderPlaced(event, deps)).toEqual({ skipped: 'duplicate' });
  expect(deps.invoices.all()).toHaveLength(1);
});

Notice that the fake dependencies are not mocks in the call-recording sense; they are small working implementations — a map for invoices, an array for sent emails, a set for processed keys. That makes assertions about outcomes natural (“exactly one invoice exists”) rather than about calls (“createForOrder was called once”), and it means the same fakes serve every handler test without per-test configuration.

Step 3 — Define a transport interface and give it an in-memory implementation. Producers publish through the interface; tests use the in-memory version and inspect what was published.

// src/messaging/bus.ts
export type Message = { topic: string; key: string; payload: unknown; headers?: Record<string, string> };
export interface Bus {
  publish(message: Message): Promise<void>;
  subscribe(topic: string, handler: (m: Message) => Promise<void>): void;
}

// test/fakes/memory-bus.ts
export function memoryBus(): Bus & { published: Message[]; deliver(): Promise<void> } {
  const published: Message[] = [];
  const handlers = new Map<string, Array<(m: Message) => Promise<void>>>();
  return {
    published,
    async publish(m) { published.push(m); },
    subscribe(topic, h) { handlers.set(topic, [...(handlers.get(topic) ?? []), h]); },
    async deliver() {
      for (const m of published.splice(0)) for (const h of handlers.get(m.topic) ?? []) await h(m);
    },
  };
}

The explicit deliver() call is deliberate. A fake that delivered immediately on publish would hide the asynchrony that real systems have, and tests written against it would pass while assuming an ordering that production does not guarantee. Making delivery a separate step lets a test publish several messages, reorder or duplicate them, and then deliver — which is how the failure conditions below become reproducible.

Step 4 — Test the producer by inspecting published messages. The assertion is on what left the service, not on how it was sent.

test('placing an order publishes an OrderPlaced event keyed by order', async () => {
  const bus = memoryBus();
  await placeOrder({ basketId: 'b1' }, { bus, orders: memoryOrders() });
  expect(bus.published).toEqual([
    expect.objectContaining({ topic: 'orders.placed', key: expect.stringMatching(/^ord_/) }),
  ]);
});

Step 5 — Test reordering and duplication deliberately with the fake transport. Because delivery is explicit, a test can publish a sequence, permute or repeat it, and deliver — reproducing on demand the conditions a real broker produces only occasionally.

test('a shipment event that arrives before its order is parked, not dropped', async () => {
  const bus = memoryBus();
  const store = memoryOrderStore();
  wireConsumers(bus, store);

  await bus.publish(anOrderShipped({ orderId: 'o1' }));
  await bus.publish(anOrderPlaced({ orderId: 'o1' }));
  await bus.deliver();                                         // shipped first, then placed

  expect(store.get('o1')).toMatchObject({ status: 'shipped' });
});

That test encodes a design decision — out-of-order events are buffered rather than discarded — and it is the kind of decision that otherwise lives only in someone’s head until an incident forces it into the open.

Step 6 — Verify the consumer adapter against a real broker, a few times. Decoding, acknowledgement and dead-lettering depend on the broker’s actual semantics, so a small set of tests runs against a containerised instance.

// test/integration/orders-consumer.test.ts
import { GenericContainer } from 'testcontainers';

let broker: StartedTestContainer;
beforeAll(async () => {
  broker = await new GenericContainer('softwaremill/elasticmq-native:1.6.0').withExposedPorts(9324).start();
}, 60_000);
afterAll(() => broker.stop());

test('a message that fails three times lands on the dead-letter queue', async () => {
  // publish a payload the handler rejects, run the consumer, and read the DLQ
});

Keep these tests few and focused on what only a real broker can show. Everything about the handler’s behaviour is already covered below; the container exists to prove that the adapter acknowledges on success, retries on failure, respects the maximum receive count and routes to the dead-letter queue — facts about the broker’s semantics that no fake can promise.

Configuration Reference Table

Two of these settings produce most real-world incidents. A visibility timeout shorter than the slowest handler execution means a message becomes visible again while still being processed, so a second consumer picks it up and the work happens twice — a duplicate that idempotency must absorb. And a partition key chosen carelessly, such as a random value, silently removes the ordering guarantee that downstream consumers were relying on.

Verification & Assertions

The most important assertions in event-driven tests are about side effects and their counts. “An invoice was created” is weaker than “exactly one invoice was created after two deliveries”, and the second is the one that catches the defect idempotency exists to prevent.

test('a retried event after a partial failure completes without duplicating the first step', async () => {
  const deps = memoryDeps();
  deps.mailer.failNext();                                    // email fails on first attempt
  const event = anOrderPlaced({ eventId: 'e7' });

  await expect(onOrderPlaced(event, deps)).rejects.toThrow();
  await onOrderPlaced(event, deps);                          // the broker redelivers

  expect(deps.invoices.all()).toHaveLength(1);               // not two
  expect(deps.mailer.sent()).toHaveLength(1);
});

The partial-failure case in that test is the one teams most often miss, because it requires thinking about the handler as a sequence of steps that can be interrupted between any two. A handler that marks the event processed only at the end, and whose earlier steps are not themselves idempotent, will duplicate work on every retry.

A related assertion that pays off disproportionately is the count of handler invocations under the real broker. When a message is configured to be attempted three times before dead-lettering, the integration test should observe exactly three attempts — not two because a timeout was too generous, and not four because an acknowledgement was lost. That single number verifies the retry configuration, the acknowledgement path and the dead-letter wiring at once.

For producers, assert on the full published message, including the key and headers, because those determine routing and ordering downstream. A message published with the wrong partition key is delivered to consumers in an order that differs from what the producer intended, and no amount of handler testing will reveal it.

Edge Cases & Failure Modes

Handlers that are idempotent in the happy path only. Marking an event as processed before performing its side effects makes a crash mid-handler lose the work; marking it after makes a crash duplicate it. The robust answer is to make each side effect idempotent on its own — an invoice keyed by order, an email keyed by event — and to test each interruption point.

Ordering assumptions that the broker does not guarantee. Kafka orders messages within a partition, SQS standard queues do not order at all, and FIFO queues order per group. A handler written as though an OrderShipped event cannot precede OrderPlaced will fail rarely and confusingly in production. Test the reversed order explicitly.

Poison messages that retry forever. A payload that can never be processed — a schema violation, a reference to a deleted record — will be redelivered until something stops it. Test that such a message reaches the dead-letter destination after the configured number of attempts, rather than blocking the queue.

Time-dependent behaviour tested with real waits. Backoff, delayed jobs and scheduled retries tested with real timers make the suite slow and flaky. Use fake timers, as in testing debounce and throttle with fake timers, so a thirty-second backoff is advanced instantly.

Schema drift between producer and consumer. A producer that adds a required field, or renames one, breaks consumers that deploy later — and in an asynchronous system the break appears as messages failing validation in a queue rather than as an error anyone sees immediately. Validate at the consumer boundary, send invalid messages to the dead-letter queue with a clear reason, and test both behaviours.

Performance & CI Impact

Handler tests with in-memory dependencies are as fast as any unit test in the suite and parallelise freely, which is the main argument for putting the case volume there. A realistic event-driven service might have a few hundred handler cases running in a second or two.

Real-broker integration tests are expensive in a different way: a container takes seconds to start and the tests themselves wait on real delivery. Start the container once per test file or once per worker, keep these tests few — typically a handful per consumer covering acknowledgement, retry and dead-lettering — and run them in a separate project so their startup cost does not slow the unit feedback loop.

Queue names and topic names deserve the same namespacing discipline as database rows. A CI job that consumes from orders-test will happily read messages another concurrent job published there, producing failures that look like application bugs and reproduce only under load.

Flakiness risk concentrates entirely in the real-broker tier, where timing depends on the broker. Poll for outcomes with a timeout rather than waiting fixed durations, and namespace queues and topics per test run so parallel CI jobs sharing a broker cannot consume each other’s messages.

The last cost is conceptual rather than computational. Event-driven code has more failure modes than synchronous code, and a test suite that only covers the happy path gives far less assurance here than it would for a request handler. Budget time for the duplicate, reordering and partial-failure cases specifically; they are where production incidents in asynchronous systems come from.

A final recommendation on structure: keep every message schema, builder and in-memory fake in a shared package that both producers and consumers import. When the producer’s team changes a field, the consumer’s tests fail at compile time rather than in production, and the shared builders mean both sides test against the same idea of what a valid message looks like — a lightweight form of the agreement that contract testing formalises.

In-Depth Guides

• Testing Kafka consumers with an in-memory broker — partitions, offsets and rebalances without a cluster.
• Mocking SQS and SNS clients in Vitest — AWS SDK v3 command mocks and a local queue for integration.
• Testing webhook handlers with signed payloads — real HMAC signatures, replay windows and duplicate deliveries.
• Testing background jobs queued with BullMQ — processors as functions, retries with fake time, and a real Redis for wiring.

Related

• Back to Advanced Mocking & Service Isolation Patterns
• Contract Testing — keeping producer and consumer message shapes in agreement.
• Time & Date Control Strategies — controlling backoff and scheduled delivery.
• Stubbing Redis with ioredis-mock — the store behind many job queues.