[GitHub] [druid] FrankChen021 commented on issue #11061: How to improving ingestion performance

[GitBox <gi...@apache.org>]

FrankChen021 commented on issue #11061:
URL: https://github.com/apache/druid/issues/11061#issuecomment-811912104


   Your task count is 30, does that mean there're 30 partitions of your topic ? And is the ingestion speed 5K/s of per peon ? Even it's 5K/s per peon, it's not a very high speed that it needs 10G heap size for each peon. 4G is enough in our clusters to handle messages at much higher speed.
   
   Since streaming ingestion speed is more relevant to streaming middleware, tuning producer and consumer may help. Are you using Kafka ? And what parameters you have adopted to configure Kafka producers and consumers ? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org