[GitHub] [druid] zhangyue19921010 commented on pull request #10524: Dynamic auto scale Kafka-Stream ingest tasks

[GitBox <gi...@apache.org>]

zhangyue19921010 commented on pull request #10524:
URL: https://github.com/apache/druid/pull/10524#issuecomment-744143180


   > In my opinion, the dynamic scaling taskcount operation is related to the number of partitions in Kafka. when the data traffic increases, the taskcount has been adjusted to the number of partitions in Kafka. If the data volume keeps increasing, you can only increase the number of partitions or adjust the JVM parameter of PEON to keep consumption.
   
   Hi @Mrxiashu, generally speaking, the number of topic partitions is pretty large in PRD. Adjust taskCount to the number of partitions maybe is relatively rare. But you also can do that by setting `taskCountMax ` as you wish.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org