[GitHub] [incubator-druid] himanshug commented on issue #8091: groupBy with subtotalsSpec doesn't fully group each set

[GitBox <gi...@apache.org>]

himanshug commented on issue #8091: groupBy with subtotalsSpec doesn't fully group each set
URL: https://github.com/apache/incubator-druid/issues/8091#issuecomment-512387176
 
 
   thanks, yes that sounds correct.
   
   I think current code "works" as long as first dimension from top level is included in the subtotal spec . That could be treated as a special optimized case and could use existing code path.
   
   (2) should work in all cases.
   
   We should do above to fix the bug.
   
   then, following optimizations could be done...
   
   - many users wouldn't enable disk spilling(or there wouldn't be enough data to require disk writes) , we could detect both these cases and use (1) here
   
   - sorting is retained as long as first dim is included . Let say, top level dimensions are [d1, d2, d3, d4] and subtotal specs are..
   
   [d1, d3] ....can be computed from sorted results on [d1,d2,d3,d4]
   [d2, d3] ...can be computed from sorted results on [d2,d3,d4]
   [d2, d4] ...can be computed from sorted results on [d2,d3,d4]
   optimization would be that for computation of 2nd and 3rd subtotal spec we can put data once inside the grouper for [d2,d3,d4] but this one might not be as important because this doesn't reduce the number of required merge buffers.
   
   
   are you working on it or would you like me to send the PR to fix it ?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org