[GitHub] [incubator-pinot] kishoreg commented on issue #4372: Support for single phase and two-phase distributed hash aggregation

[GitBox <gi...@apache.org>]

kishoreg commented on issue #4372: Support for single phase and two-phase distributed hash aggregation
URL: https://github.com/apache/incubator-pinot/issues/4372#issuecomment-506155708
 
 
   What we have in Pinot is very similar to two-phase hash aggregation but we merge groups across segments within the same node. We call this Combine Operator - analogous to the map-combine-reduce concept in M/R. We have S (number of segments) mappers, N (number of servers) Combiners, 1 Reducer.
   
   I think what you are suggesting S mappers, N reducers and 1 accumulator (union operator). I am trying to come up with a scenario where this would be better than the one we currently have in Pinot.
   
   
   
   
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org