You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Jackie-Jiang (via GitHub)" <gi...@apache.org> on 2023/09/28 20:06:02 UTC
[GitHub] [pinot] Jackie-Jiang opened a new issue, #11706: Group-by without ordering on aggregate
Jackie-Jiang opened a new issue, #11706:
URL: https://github.com/apache/pinot/issues/11706
When a group-by query does not have order-by on the aggregate column, we don't need to keep more groups than the LIMIT because the order-by value won't change. We can maintain a heap (PriorityQueue) of LIMIT values. On the group-key generation side, we should also keep only the relevant keys.
One common query is:
`SELECT COUNT(*) FROM myTable GROUP BY timeCol ORDER BY timeCol DESC LIMIT 10`
Problems to solve:
1. Group-by query with order-by on the key column:
Currently we keep `Math.max(5000, LIMIT * 5)` groups, which is not necessary since only the top `LIMIT` groups are relevant
2. Group-by query without order-by:
Currently we keep random `LIMIT` groups per server, and there is no guarantee the same group is picked across different servers, which can lead to wrong result when there are more than `LIMIT` groups
Solution:
1. To ensure the ordering is deterministic (we need this guarantee to ensure the groups returned from all servers are the same), we should append all non-ordering group keys implicitly. This is one exception: when we want to keep all groups on the server, we don't need this since all groups will be returned anyway.
2. Optimize the execution when all the ordering keys are group key
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Group-by without ordering on aggregate [pinot]
Posted by "anandheritage (via GitHub)" <gi...@apache.org>.
anandheritage commented on issue #11706:
URL: https://github.com/apache/pinot/issues/11706#issuecomment-2374080673
@Jackie-Jiang If no one have picked this up - I can pick it up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Group-by without ordering on aggregate [pinot]
Posted by "anandheritage (via GitHub)" <gi...@apache.org>.
anandheritage commented on issue #11706:
URL: https://github.com/apache/pinot/issues/11706#issuecomment-2893699014
https://github.com/apache/pinot/pull/15844
@Jackie-Jiang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Group-by without ordering on aggregate [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #11706:
URL: https://github.com/apache/pinot/issues/11706#issuecomment-6243694223
This issue has been closed because it has been inactive for 180 days. If you believe this issue is still relevant, please feel free to reopen it or create a new issue. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Group-by without ordering on aggregate [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #11706:
URL: https://github.com/apache/pinot/issues/11706#issuecomment-5706516156
This issue has been automatically marked as stale because it has not had any activity in the last 150 days. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Group-by without ordering on aggregate [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #11706: Group-by without ordering on aggregate
URL: https://github.com/apache/pinot/issues/11706
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org