# What parallel value should I set for Akka Kafka consumer when we scale out more Kubernetes pods?

**URL:** https://discuss.akka.io/t/what-parallel-value-should-i-set-for-akka-kafka-consumer-when-we-scale-out-more-kubernetes-pods/9482
**Category:** Akka Streams & Alpakka
**Tags:** akka-cluster, kafka
**Created:** [February 3, 2022, 8:34am UTC](https://discuss.akka.io/t/what-parallel-value-should-i-set-for-akka-kafka-consumer-when-we-scale-out-more-kubernetes-pods/9482 "2022-02-03T08:34:37Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![sshtel](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sshtel](https://discuss.akka.io/u/sshtel)
#### Post date: [February 3, 2022, 8:34am UTC](https://discuss.akka.io/t/what-parallel-value-should-i-set-for-akka-kafka-consumer-when-we-scale-out-more-kubernetes-pods/9482/1 "2022-02-03T08:34:37Z")

</div>

Hello, we are using Kubernetes for Akka clustering deployment and management.

Basically, we have 3 replicas or pod to build Akka clustering.  
Each pod application code is subscribing Kafka topic with parallel = 3  
Like

```auto
service.applicationTopic
.subscribe.atLeastOnce(Flow[ApplicationMsg]
.mapAsync(parallelism = 3) { ... }

```

As far as I understand, the sample code can make each application pod have up to 3 consumer connection pools. And we have 3 pods so the cluster will have up to 9 connection pools for the topic. (Btw number of Kafka partition is 8)

We recently decided to increase the number of pod for scale out for better performance.  
If we increase the number of pod from 3 to 7, and each application can have parallel = 3 thread pool will try to connect to Kafka broker of 8 partitions.

In this case, is there any possibility that some pods that deployed earlier than others could occupy all the Kafka broker connections? And some late pods don’t have chance to subscribe the topic due to the limit of Kafka partitions?

And should I decrease the parallel value so that I can expect each pods can have fair chance to connect to each Kafka partitions?

---

<div class="post-metadata">

### Author: ![sshtel](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sshtel](https://discuss.akka.io/u/sshtel)
#### Post date: [February 3, 2022, 8:36am UTC](https://discuss.akka.io/t/what-parallel-value-should-i-set-for-akka-kafka-consumer-when-we-scale-out-more-kubernetes-pods/9482/2 "2022-02-03T08:36:46Z")

</div>

I referred to this discussion to understand thread pool of Kafka consumer group of Akka cluster

> [@Akka kafka consumer parallel processing](https://discuss.akka.io/t/akka-kafka-consumer-parallel-processing/6439):
>
> Hi, I’m working on a Kafka consumer application using Akka Kafka connector. I would like to consumer to process messages parallelly. which consumer group should I use [https://doc.akka.io/docs/alpakka-kafka/current/consumer.html](https://doc.akka.io/docs/alpakka-kafka/current/consumer.html)? how can I configure the parallelism on the consumer side?