2025-06-01 | ← Blog
Partitions are the most misunderstood part of Kafka. Here is how I actually think about sizing them.
When I first started working with Kafka, partition count felt like an arbitrary number you just picked. 3 felt reasonable. 6 felt safer. Nobody really explained why.
After running Kafka in production on a payment processing platform, here is how I actually think about it.
Partition count = max consumer parallelism you will ever want.
That is it. Everything else is secondary.
If you have 3 partitions, you can have at most 3 active consumers in a group. A 4th consumer will sit idle. Kafka assigns one consumer per partition — no sharing.
Most guides lead with throughput math. I find that misleading.
For most backend services, consumer parallelism is the real bottleneck — not raw throughput. If your service needs to scale to 12 instances under load, you need at least 12 partitions. You probably won't hit per-partition throughput limits before you hit that.
Where throughput does matter: high-volume data pipelines, analytics ingestion, log aggregation. For transactional microservices, think consumers first.
Starting with 1 or 2 partitions "because it's simple".
You can increase partitions later, but it breaks key-based ordering for existing keys. If your consumers depend on per-key ordering (common in payment systems), that is a painful migration.
Over-partition slightly from the start. 6 partitions for a service you expect to scale to 3 instances is fine.
For a new topic on our platform:
For our 6-broker cluster spanning two datacenters, I default to 6 or 12 partitions per topic.
The maximum useful consumers in a group = number of partitions.
That is the hard ceiling. Extra consumers beyond partition count are wasted resources. Size your topics to your scaling targets, not to what feels comfortable today.