[GitHub] [incubator-pinot] fx19880617 commented on issue #6146: Low maximum limit for batch jobSpec pushParallelism

[GitBox <gi...@apache.org>]

fx19880617 commented on issue #6146:
URL: https://github.com/apache/incubator-pinot/issues/6146#issuecomment-708558434


   I feel is is due to we upload segments to all the controller hosts and the idealStats update requests coming from all controllers will cause the slowness and update conflicts.
   
   This issue wasn't there as controller needs to download the segment tar and untar the metadata then do the update, so it's a costly behavior in controller. With segment metadata only push mode, we may need to rethink this.
   
   cc: @siddharthteotia @mayankshriv @snleee 
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org