[GitHub] [druid] zhangyue19921010 commented on pull request #10524: Dynamic auto scale Kafka-Stream ingest tasks

[GitBox <gi...@apache.org>]

zhangyue19921010 commented on pull request #10524:
URL: https://github.com/apache/druid/pull/10524#issuecomment-787857767


   Hi @pjain1 Thanks for asking. Yep, if desired task count becomes more than the number of topic partitions, it will increase task number every `dynamicCheckPeriod ` until meet taskCountMax instead of unlimited growth. So that the increase behavior is controllable. Also to avoid this unnecessary scenario maybe we can document the relationship between taskCountMax and kafka partition like taskcount in IOConfig does? For example  `The number of reading tasks will be less than taskCountMax if taskCountMax > {numKafkaPartitions}.`
   <img width="838" alt="屏幕快照 2021-03-01 下午6 49 59" src="https://user-images.githubusercontent.com/69956021/109487326-079cf080-7abf-11eb-8ed5-12009a2194cb.png">
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org