The database committed. Where did the event go?

Fictional setting, real lab evidence. Fortex in this series is a fictional setting; Tom and Ben are characters. Their conversations illustrate the reasoning. Measurements and assertions come from the linked Docker lab, not Fortex production incidents.

The Fortex journey · Chapter 6/8

An order between two systems

Fortex needed to forward new orders to warehouse processing. Ben had fixed inventory updates and kept the cache out of authoritative decisions. His next flow was tidy: commit the order in PostgreSQL, then publish OrderCreated.

“Publishing after commit prevents events for rolled-back orders.”

Tom read the error path. “You've handled one direction. Now suppose the process stops here.” He put the cursor between commit and publish.

“We retry the publication next time.”

“How does next time find the order that needs publishing?”

Ben looked at the event variable in memory. Retry was a sensible intention, but nothing durable told a replacement process what to retry. Tom opened the lab so they could inspect that gap as data.

The order exists, but its event does not

Ben had considered a transaction annotation around the entire function. Tom asked him to draw the actual boundary: PostgreSQL owned the transaction, while the broker call crossed into another system. A boundary around code wasn't automatically a commit boundary.

An API commits a new order to PostgreSQL, then publishes OrderCreated for another service. The process stops between those operations. The order appears in the customer's history while the downstream service never learns about it.

An application transaction annotation does not automatically enlist a broker in PostgreSQL's transaction. Reversing the order leaves another failure window: an event may be published for an order whose database transaction later rolls back.

A transactional outbox stores the intent to publish alongside the business change. It closes this particular gap, while leaving other failure windows to address explicitly.

Commit business data and publication intent together

“Then I store the work to do as well,” Ben said. “In the same commit as the order. A restarted worker can find it.”

“Yes. You've made an in-memory promise durable,” Tom replied. “Now keep following it until delivery.”

BEGIN;
INSERT INTO orders(id) VALUES (2);
INSERT INTO outbox(event_id, order_id, sent)
VALUES ('evt-2', 2, false);
COMMIT;

A rollback removes both the order and its intent. A commit leaves a durable database record for a relay to discover after the request ends or the application restarts. Delivery still depends on the relay eventually running, successful retries, retention, and operations; the row alone does not make those conditions true.

A relay finds unsent rows, publishes them, waits for an appropriate acknowledgement, and records success. This article builds on AWS's transactional outbox guidance. The experiment uses Redis Streams, not Kafka, and makes no claim about Kafka durability from its results.

The next gap: published, but not marked sent

Ben asked first: “If it publishes without marking sent, the worker publishes again. Should I move the marker before publication?”

Tom let him place the cursor in the new gap. Ben sighed. “Then we can lose it again. We need to tolerate duplicates to keep retrying.”

The broker acknowledges evt-2. The relay stops before marking the outbox row sent. On restart, it sees an unsent row and publishes again.

Marking the row before sending risks losing an event. Marking it afterward permits duplicates. Outbox implementations commonly preserve retryability and require consumers to tolerate redelivery.

Generate the business event ID when recording the intent and preserve it across publish attempts. The broker may assign different message IDs to those attempts. Deduplicating only by broker message ID would treat them as different business events.

Make the inbox and business effect atomic

“This connects to inventory,” Ben said. “Two valid increments can still represent one event applied twice.”

Tom asked which transaction would own the marker. Ben grouped it with the projection, then noted the boundary: an external call wasn't covered by that database commit.

The lab consumer records the event and updates its projection in one transaction:

BEGIN;
WITH accepted AS (
  INSERT INTO inbox(event_id) VALUES ('evt-2')
  ON CONFLICT DO NOTHING
  RETURNING event_id
)
UPDATE projection SET total = total + 1
WHERE EXISTS (SELECT 1 FROM accepted);
COMMIT;

The unique inbox key arbitrates ownership. Only a transaction that inserts the marker applies the effect. If the inbox marker and business update commit separately, a crash between them can suppress a retry even though the effect never happened.

A database transaction cannot cover an external email or payment operation. That destination needs its own mechanism, such as idempotency keys, or an explicit reconciliation policy. Having an outbox does not establish end-to-end exactly-once execution.

Follow the failure windows in Docker

They used Redis Streams to keep message IDs easy to inspect. Tom asked Ben to distinguish this lab from Fortex's Kafka path. Ben documented every skipped command and resend; a story about a stopped process wasn't evidence of a power-loss test.

The isolated stack runs PostgreSQL 17.6 and Redis Streams 7.4.2. The first branch commits an order and deliberately omits publication. The outbox branch commits an order and intent, then publishes the same event twice to model a relay restarting before recording success. These are controlled command-boundary failpoints, not machine-kill or broker-failover tests.

The two XADD calls receive different stream IDs. Nevertheless, the inbox contains one evt-2, and the projection increments once. Another transaction creates order 3 and evt-3, then rolls back; neither row remains. The report and raw logs contain the real IDs and assertions.

Redis persistence is disabled in this disposable lab. It verifies transaction boundaries and duplicate handling, not survival after power loss. Download the source below and run make reproduce to repeat the experiment.

Operational costs

Ben's proposal now had an outbox, a relay, and an inbox. Tom didn't ask for another pattern. He asked who would act when the oldest unpublished event kept getting older.

Ben added ownership, retention, and a replay path. Preserving the work made recovery possible; it didn't provide an operator automatically.

FOR UPDATE SKIP LOCKED helps distribute rows between workers but does not automatically preserve business ordering. Holding a database lock while waiting for a broker is simple at low volume but lengthens transactions and occupies connections. Claiming work and releasing the transaction sooner requires leases and recovery for dead owners. The lab does not implement a production multi-worker relay.

An outbox adds writes and downstream delay. A job that lives entirely inside one database may only need a job table. When a business change must reliably lead to work in another system, preserving the intent is valuable. The next design question is still what happens when that intent is delivered again.

The page Ben takes into tomorrow

Ben wrote: “Retry needs durable work to find. Deduplication needs a stable identity for that work.” He proposed that Fortex tolerate redelivery and control consumer effects rather than promise exactly-once behavior across the whole path.

Tom let him present that part. A question from the export team then brought them back to the database: who was allowed to see those orders and invoices?

Previous chapter: One expired key, twelve requests hitting the database · Next chapter: A missing tenant filter: can PostgreSQL RLS stop the leak? · Start the journey