Updated July 2026. Refreshed for current microservices best practices.
Why Kafka in Microservices?
Apache Kafka is a distributed commit log that decouples producers from consumers. Unlike traditional message queues, Kafka retains events for configurable periods, supports replay, and scales horizontally — making it ideal for event-driven microservices and event sourcing.
Core Concepts
- Topic — named stream of events (e.g.,
orders,payments). - Partition — ordered, immutable sequence within a topic. Parallelism unit.
- Offset — position of a consumer within a partition.
- Consumer group — consumers sharing work; one consumer per partition max.
- Broker — Kafka server storing partitions.
Partition Key Strategy
Use aggregate ID (e.g., orderId) as the partition key so all events for one order stay ordered. Without keys, round-robin assignment breaks ordering guarantees.
Retention and Compaction
Time-based retention: keep events for 7 days (default).
Log compaction: keep latest value per key — useful for changelog topics and snapshots.
Microservices Topic Design
- One topic per aggregate type or bounded context.
- Use schema registry (Avro/Protobuf) for contract evolution.
- Dead-letter topics for poison messages.
- Separate internal vs external event topics for API boundaries.
Related: CQRS with Kafka · Event Sourcing
Want live microservices classes? Join Alkademy for instructor-led microservices architecture courses with hands-on projects.
Go deeper with the book: Practical Microservices Design Using CQRS and Event Sourcing by Kindson Munonye.
Kindson Munonye is a software engineer and technical author specializing in microservices, CQRS, event sourcing, and distributed systems. He publishes free step-by-step tutorials and live classes on Alkademy.