You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2024/07/26 19:48:14 UTC
[I] [Epic] High cardinality aggregation performance wishlist [datafusion]
alamb opened a new issue, #11679:
URL: https://github.com/apache/datafusion/issues/11679
### Is your feature request related to a problem or challenge?
This is my wishlist for improving high cardinality aggregates (ideally for the next blog post in a few months #11631 )
Together with the StringView work in https://github.com/apache/datafusion/issues/10918 that @XiangpengHao @a10y and others are working on, I think it would provide some very compelling overall speedups in ClickBench and TPCH queries
### Describe the solution you'd like
Here is my wishlist:
- [ ] https://github.com/apache/datafusion/issues/6937 (@korowa has a PR up for this one)
- [ ] https://github.com/apache/datafusion/issues/7957 (I have a prototype and some ideas)
- [ ] TBF: reuse the hashed values (working on filing this)
### Describe alternatives you've considered
Do nothing and let DuckDB pass us by ;)
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [Epic] High cardinality aggregation performance wishlist [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #11679:
URL: https://github.com/apache/datafusion/issues/11679#issuecomment-6372565961
I think we have mostly completed this so closing it down and we can plan new epics
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [Epic] High cardinality aggregation performance wishlist [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #11679:
URL: https://github.com/apache/datafusion/issues/11679#issuecomment-2268814899
https://github.com/apache/datafusion/issues/6937 -- bam
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [Epic] High cardinality aggregation performance wishlist [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #11679: [Epic] High cardinality aggregation performance wishlist
URL: https://github.com/apache/datafusion/issues/11679
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [Epic] High cardinality aggregation performance wishlist [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #11679:
URL: https://github.com/apache/datafusion/issues/11679#issuecomment-2258183705
Bonus items:
- [ ] https://github.com/apache/datafusion/issues/11717 from @Rachelint
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [Epic] High cardinality aggregation performance wishlist [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #11679:
URL: https://github.com/apache/datafusion/issues/11679#issuecomment-2457902964
Here is another great improvement: https://github.com/apache/datafusion/pull/12996
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] [Epic] High cardinality aggregation performance wishlist [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #11679:
URL: https://github.com/apache/datafusion/issues/11679#issuecomment-2373862538
Update here
- [x] https://github.com/apache/datafusion/pull/11627 -- improvement from @korowa
- [x] https://github.com/apache/datafusion/pull/12269 -- improvement from @jayzhan211
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org