You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by ChienHsing Wu <ch...@opentext.com> on 2019/11/13 17:41:32 UTC

Kafka Streams not allowed to change the number of input topic partitions

Hi,

I came across the links below that indicates it's not simple to increase partitions in an input topic afterwards. Is this still true in the current kafka version 2.3.1 and soon to be available 2.4?

https://stackoverflow.com/questions/42329387/failed-to-rebalance-error-in-kafka-streams-with-more-than-one-topic-partition/42334849#42334849?newreg=b7e04be652f74b87a8a28721c865a62d
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-HowtoscaleaStreamsapp,i.e.,increasenumberofinputpartitions?


Thanks, CH

Re: Kafka Streams not allowed to change the number of input topic partitions

Posted by "Matthias J. Sax" <ma...@confluent.io>.

That is correct. And there is no WIP to address this atm. It's a pretty
complicated problem to allow for dynamic scaling.


-Matthias


On 11/13/19 9:41 AM, ChienHsing Wu wrote:
> Hi,
> 
> I came across the links below that indicates it's not simple to increase partitions in an input topic afterwards. Is this still true in the current kafka version 2.3.1 and soon to be available 2.4?
> 
> https://stackoverflow.com/questions/42329387/failed-to-rebalance-error-in-kafka-streams-with-more-than-one-topic-partition/42334849#42334849?newreg=b7e04be652f74b87a8a28721c865a62d
> https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-HowtoscaleaStreamsapp,i.e.,increasenumberofinputpartitions?
> 
> 
> Thanks, CH
>