Vailati
Go back

CDC or outbox: why I chose the outbox to publish events

Both guarantee that the event goes out if, and only if, the database transaction was committed. The difference is in what goes out, in who owns the contract, and in what you end up operating.

Context

I needed every operation on a points ledger to produce a Kafka event with no chance of writing to the database without publishing, or publishing without writing. I looked at two paths: Change Data Capture with Debezium reading the PostgreSQL replication log, which went as far as a proof of concept, and the outbox pattern, a table written in the same transaction as the operation and drained by a job. I chose the outbox. The cost was code in the application, a job every minute and a table with very high churn; the gain was an event contract defined by the domain, with no consumer coupled to the table schema and no Kafka Connect to operate.

This article is not about which pattern is better. It is about which questions decided it in my case, and how to answer them in yours.

Problem

The underlying problem is the same for both patterns: the dual write. A service that writes to the database and then calls Kafka has two failure points that do not roll back together. If the commit succeeds and the produce fails, the event is lost. If the produce succeeds and the commit fails, the world hears about something that never happened. Retrying does not fix it: it only fixes one side.

Both solutions take Kafka out of the transaction’s path. With CDC, the application only writes to the database; a connector reads the PostgreSQL transaction log, the WAL, through logical replication, and turns every changed row into an event. With the outbox, the application writes, in the same transaction, the business row and a row holding the already-built event; a separate process reads that table and publishes.

Choosing between them is rarely about delivery guarantees, which both provide. It is about three things: what the event is, who controls the schema, and what lands on the operations side.

What I measured

The balance here is uneven, and it is worth being explicit about it: CDC was an infrastructure proof of concept; the outbox runs in production.

The CDC proof of concept. A Kafka Connect with the Debezium PostgreSQL connector, the pgoutput plugin, PostgreSQL with wal_level=logical, capturing a credits table. The connector skipped updates, deletes and truncates, published only inserts, and applied three transformations: extract the new state of the row, use the account number as the message key, and route everything to a fixed topic. It worked within minutes: every INSERT became a message with the table’s columns as fields.

What the proof did not show was a consumer. The application in the same repository never read the topic the connector produced; the events it consumed and produced remained messages assembled in code. The proof proved that the plumbing works, and stopped there. That is a result: the plumbing was never the question.

The outbox in production. Three outbox tables, one per family of event. On a typical day, a little over a million rows are written and deleted in the main one alone. Each job runs every minute, reads a batch of two thousand rows in id order, publishes with an idempotent producer and acks=all, using the account id as the partition key, and deletes the whole batch only if every delivery was acknowledged. The latency between the commit and the message in Kafka is up to one minute. The published event is the same JSON stored in the row, byte for byte, with more than a hundred fields: the full operation, its items and the account data the consumer needs.

The outbox table is pure insert and delete, and that has a cost I only saw later: many dead tuples per day, frequent autovacuum, and indexes that grow faster than the table. I come back to that at the end.

Solution

The decision came from four concrete differences. For each one, here is what CDC would give and what the outbox gives.

1. What the event is. With CDC, the event is the row. A business operation on the ledger touches three tables: the entry, the credits and the debits. Debezium would emit three change streams, one per table, each with that table’s columns, and the consumer would have to put the pieces together to understand what happened. The connector can be configured to emit transaction metadata and group changes, but the reconstruction stays on the consumer’s side. With the outbox, the event is what the domain means to say: “this operation was processed, with these items, for this account”. One message, one contract. The consumers in my case are a lake, a streaming pipeline and a payments partner. None of them wants to know there are three tables.

2. Who owns the schema. With CDC, the event contract is the table schema. Renaming a column, changing a type, moving a field to another table: all of it becomes a contract change with third parties, and the database becomes a public API. There is mitigation, with transformations in the connector and a schema registry, but it is mitigation. With the outbox, the table schema changes freely; the contract is the class that serializes the event, versioned in code, with tests. For a ledger whose data model is still evolving, this weighed more than anything else.

3. Partitioned tables. The ledger’s three tables are partitioned at two levels, and new partitions are born over time. PostgreSQL logical replication publishes changes per leaf partition, unless the publication is configured to publish via the root, and the connector needs to know which tables to capture. Every new partition is one more item in the connector configuration, or a regular expression that has to stay right. It is not insurmountable, but it is one more place to forget on New Year’s Eve. The outbox is a plain table, unpartitioned, and knows nothing about the others.

4. What lands on operations. CDC brings a Kafka Connect to run, monitor and upgrade, a replication slot in PostgreSQL, and a consequence that scares anyone who has seen it: if the connector stops, the slot holds the WAL, the WAL grows until the disk is full, and the database goes down with it. On a managed service that is an alert to configure and a routine to keep. The outbox brings a job in the worker that already existed, one more table, and the watch over it: a batch that does not drain, a stuck row, autovacuum. All within what the team already operated.

With the four on the table, the answer was obvious for my context. I want to publish domain events with their own contract, from partitioned tables, with a team that operates PostgreSQL and Kafka but does not operate Kafka Connect. That is an outbox.

flowchart TD
    Q1{Is the event the row<br/>or the business fact?} -->|row, 1 table,<br/>internal consumer| CDC[CDC: Debezium reads the WAL]
    Q1 -->|business fact,<br/>several tables, external contract| Q2{Who operates<br/>Kafka Connect?}
    Q2 -->|nobody yet| OUT[Outbox: table + job]
    Q2 -->|already in place| HYB[Hybrid: outbox + Debezium<br/>Outbox Event Router]

Where CDC would win

To be fair to the path I did not take. If the event really were the row, an audit or history table, CDC delivers without a line of code in the application and also captures what the application did not do: a manual UPDATE in the middle of the night, a bulk load, a fix-up script. The outbox only knows what went through the code. If I needed millisecond latency instead of up to a minute, CDC would win too; a polling outbox has the job interval as its floor, and replacing polling with notification is extra work.

And there is a third path that joins the two: the application writes the outbox, as it does today, and Debezium reads the outbox table from the WAL instead of a job polling it. Debezium ships a component for exactly that, the Outbox Event Router. The contract stays with the domain; the job goes away, the DELETE goes away, the one-minute latency goes away. The price is Kafka Connect, back again. I did not choose it at the time because reason number four weighed too much; if the team starts operating Connect for some other reason, it is the first thing I would revisit.

What I would do differently

I would have given the proof of concept a stopping criterion. It proved that Debezium publishes inserts, which nobody doubted, and did not prove what mattered: a real consumer rebuilding a business event from three tables and reporting what that cost. A proof of concept without the right question produces a repository nobody consumes.

I would have thought about the outbox table’s life cycle on day one. Inserting and deleting more than a million rows a day is a pattern PostgreSQL autovacuum handles, but not for free: dead tuples, bloated indexes and a REINDEX that ends up in the routine. The alternative I would evaluate today is marking rows as published instead of deleting them and purging by time range, or partitioning the outbox by day and dropping whole partitions.

I would have documented the decision with these four questions, back then. The choice was right, but its record was “we use an outbox”. Two years later, when someone proposes CDC again, the answer has to be rebuilt. This article is, in part, that rebuild.

References

Share this post:
Share this post on LinkedinShare this post on WhatsappShare this post on XShare this post via email
Previous Post
Distributed tracing across the outbox and the in-memory queue