You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "Dandandan (via GitHub)" <gi...@apache.org> on 2023/06/27 11:38:43 UTC

[GitHub] [arrow-datafusion] Dandandan commented on issue #4973: Improve the performance of `Aggregator`, grouping, aggregation

Dandandan commented on issue #4973:
URL: https://github.com/apache/arrow-datafusion/issues/4973#issuecomment-1609327645

   One earlier suggestion in the comment I found interesting by @tustvold is the ability to store the data inline (in the hashtable) vs storing it in a `Vec`-like container. In some cases (e.g. count/sum) this might have some benefit (perf/memory) over keeping only indices to external array.
   
   There is some overlap with hash join - In https://github.com/apache/arrow-datafusion/pull/6724 added the ability to vectorize the hash collision checks (using `eq`/`take`) .
   If we move away from converting to Row-format and this approach can be re-used here as well.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org