You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2018/11/16 05:58:45 UTC

[GitHub] hate13 commented on issue #6623: Fetch offsets optimize

hate13 commented on issue #6623: Fetch offsets optimize
URL: https://github.com/apache/incubator-druid/pull/6623#issuecomment-439290848
 
 
   @QiuMM @asdf2014 We made some simple test as follows.
   
   partitions: 1000,
   kafka.network.RequestMetrics.TotalTimeMs.Offsets.99thPercentile: 500ms
   
   Using KafkaConsumer#endOffsets takes about 300~3000ms, while the previous way takes almost 100s. However, it depends on the state of the kafka cluster which contains many factors.
   
   In the underlying implementation, the previous will execute several times, getting just one partition's offset each time. But KafkaConsumer#endOffsets could fetch all offsets in once.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org