[GitHub] [incubator-pinot] mayankshriv commented on issue #4372: Support for single phase and two-phase distributed hash aggregation

[GitBox <gi...@apache.org>]

mayankshriv commented on issue #4372: Support for single phase and two-phase distributed hash aggregation
URL: https://github.com/apache/incubator-pinot/issues/4372#issuecomment-512615638
 
 
   @siddharthteotia Yes, the broker (1 reducer) will perform the final aggregation across nodes.
   
   If the cardinality is low, then the individual server responses will be small and single broker reducer should be able to handle easily. For scalability, we don't want too many nodes to serve the same query, so the number of nodes is also limited.
   
   If the cardinality is high, then we shouldn't shuffle anyways.
   
   So I have the same question as @kishoreg, if we can come up with a case where shuffle would be better than broker side reduce (for limited number of servers per query), then we can discuss in that context.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org