AdvancedTechnical
5 min
Exactly-Once Effects with the Outbox Pattern
ReliabilityMessagingDatabases
Advertisement
Interview Question
You need reliable event publication coupled with database writes. Describe how you’d implement the outbox pattern and ensure idempotency end to end.
Key Points to Cover
- Write DB change and event to the same transaction (outbox table)
- Background relay publishes outbox to the broker with retries
- Use idempotency keys and deduplication at consumers
- Monitor outbox backlog and poison records via DLQ
Evaluation Rubric
Couples DB write and event atomically35% weight
Ensures idempotent, deduplicated processing25% weight
Describes relay/monitoring/DLQ operations20% weight
Considers scaling of relays/consumers20% weight
Hints
- 💡Make outbox records immutable; process in order when needed.
Common Pitfalls to Avoid
- ⚠️Treating the outbox relay as a simple poll loop without proper concurrency control, leading to duplicate event publications or missed events.
- ⚠️Failing to make the outbox table write and the business data write atomic within the same transaction.
- ⚠️Implementing idempotency at the consumer without also ensuring the idempotency key is persisted with the event in the outbox.
- ⚠️Ignoring the possibility of the relay service itself failing and not implementing persistent storage for its progress or state.
- ⚠️Not having a clear strategy for handling events that consistently fail to be published or processed, leading to unbounded retry loops or orphaned messages.
Potential Follow-up Questions
- ❓How do you handle schema changes in outbox?
- ❓What about exactly-once with Kafka transactions?
Advertisement