You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2025/04/03 19:56:35 UTC
[I] Improve time for SortPreservingMerge stream / uninitiated_partitions VecDeque [datafusion]
alamb opened a new issue, #15573:
URL: https://github.com/apache/datafusion/issues/15573
### Is your feature request related to a problem or challenge?
Both @rluvaton and I have seen
<img width="2025" alt="Image" src="https://github.com/user-attachments/assets/cd91c702-51fa-45b7-9214-7913c1281161" />
I saw this when running the following query
```shell
./datafusion-cli-filter-pushdown -c "SELECT \"WatchID\", \"ClientIP\", COUNT(*) AS c, SUM(\"IsRefresh\"), AVG(\"ResolutionWidth\") FROM hits WHERE \"SearchPhrase\" <> '' GROUP BY \"WatchID\", \"ClientIP\" ORDER BY c DESC LIMIT 10;"
````
Using hits.parquet: https://datasets.clickhouse.com/hits_compatible/hits.parquet
I am not sure what @rluvaton was doing when he saw it
### Describe the solution you'd like
Review and see if this is a performance bottleneck we can remove
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve time for SortPreservingMerge stream / uninitiated_partitions VecDeque [datafusion]
Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on issue #15573:
URL: https://github.com/apache/datafusion/issues/15573#issuecomment-2778505956
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve time for SortPreservingMerge stream / uninitiated_partitions VecDeque [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #15573:
URL: https://github.com/apache/datafusion/issues/15573#issuecomment-2776790222
BTW this code was added in
- https://github.com/apache/datafusion/pull/12302
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve time for SortPreservingMerge stream / uninitiated_partitions VecDeque [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #15573:
URL: https://github.com/apache/datafusion/issues/15573#issuecomment-2778821980
FYI @Rachelint I think @rluvaton has a proposal on a ticket as well
- https://github.com/apache/datafusion/pull/15562
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Improve time for SortPreservingMerge stream / uninitiated_partitions VecDeque [datafusion]
Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan closed issue #15573: Improve time for SortPreservingMerge stream / uninitiated_partitions VecDeque<usize>
URL: https://github.com/apache/datafusion/issues/15573
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org