How to implement reliable inventory synchronization between ERP, WMS, and ecommerce channels using Kafka and idempotent consumers.
Problem statement
In modern commerce, keeping inventory accurate across multiple warehouses, retail storefronts, and e-commerce web applications is a difficult synchronization problem. When systems rely on cron-based database polling, updates lag by minutes, leading to overselling during sales campaigns. Conversely, direct HTTP calls between systems introduce tight coupling, creating cascading outages if a downstream API is offline.
An event-driven architecture built on Apache Kafka solves this by broadcasting inventory changes asynchronously. However, moving to an asynchronous model introduces challenges: out-of-order event delivery and network retries can cause race conditions that corrupt stock counts. This guide shows how we designed a reliable inventory synchronization system with Kafka using idempotent consumers and out-of-order resolution.
Topology
Our system broadcasts changes from ERP and WMS databases using Change Data Capture (CDC). A central sync service consumes these events and updates e-commerce channels:
Producer contract
To prevent schema drift, we enforce a strict JSON Schema on the Kafka topic. The payload contains a unique event ID, the target SKU, the specific warehouse code, the current available stock level, and the timestamp of when the adjustment occurred in the physical warehouse:
{
"eventId": "d655f46f-c1f0-466d-88b9-4a945112fa87",
"sku": "BRK-AXL-0192",
"warehouseCode": "DE-HAM-01",
"availableQty": 128,
"occurredAt": "2026-05-14T08:51:32.412Z"
}Consumer idempotency pattern
To prevent duplicate processing from network retries, the consumer uses a deduplication table. It also checks the `occurredAt` timestamp to ensure a stale, delayed event does not overwrite a newer inventory update:
async function handleInventoryEvent(event: InventoryChanged) {
// 1. Deduplicate by unique event ID using Redis cache
const isProcessed = await dedupeStore.has(event.eventId)
if (isProcessed) return
// 2. Resolve out-of-order delivery
const latestTimestamp = await inventoryRepo.getLastUpdateTime(event.sku, event.warehouseCode)
const eventTime = new Date(event.occurredAt).getTime()
if (latestTimestamp && eventTime < latestTimestamp) {
// Event is stale (newer update already processed), skip write
return
}
// 3. Update database and commit event processing
await inventoryRepo.upsertStock(event.sku, event.warehouseCode, event.availableQty, eventTime)
await dedupeStore.markAsProcessed(event.eventId, 86400) // TTL 24 hours
}Key operational guardrails
When running Kafka in production for inventory synchronization, enforce the following configurations:
- Partition keying: Partition the topic using the SKU as the key. This ensures that all events for a specific product are processed in order by the same consumer instance.
- Dead Letter Queue (DLQ): If an event fails to process due to a missing product record, route it to a DLQ and alert the catalog team. Never block the consumer partition for validation errors.
- At-least-once configuration: Configure the Kafka producers with `acks=all` and consumers with manual commit offsets. This guarantees that events are never lost during broker failovers.
Our take
Apache Kafka alone does not guarantee business correctness. It only guarantees event transportation. If your consumers do not enforce idempotency and out-of-order checks, you are simply replacing API bottlenecks with database race conditions. Always design your data contracts with explicit change timestamps, and track processing states per SKU.
