You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "AdamGS (via GitHub)" <gi...@apache.org> on 2026/09/16 13:40:57 UTC
[I] Window sort pushdown below LIMIT/OFFSET prevents early termination [datafusion]
AdamGS opened a new issue, #25378:
URL: https://github.com/apache/datafusion/issues/25378
### Describe the bug
Ran into this issue when working on https://github.com/apache/sedona-spatialbench/pull/146, seems like some change since the 54 release changed the behavior here.
### To Reproduce
Assuming:
```sql
CREATE TABLE t (id INT, kind VARCHAR) AS VALUES
(19, 'a'), (18, 'a'), (17, 'a'), (16, 'a'), (15, 'a'),
(14, 'a'), (13, 'a'), (12, 'a'), (11, 'a'), (10, 'a'),
(9, 'a'), (8, 'a'), (7, 'a'), (6, 'a'), (5, 'a'),
(4, 'a'), (3, 'a'), (2, 'a'), (1, 'a'), (0, 'a');
```
The plan we get in an SLT test is:
```
01)Projection: row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW AS rn, t.id
02)--WindowAggr: windowExpr=[[row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW]]
03)----Projection: t.id
04)------Limit: skip=5, fetch=5
05)--------Filter: t.kind = Utf8View("a")
06)----------TableScan: t projection=[id, kind]
physical_plan
01)ProjectionExec: expr=[row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW@1 as rn, id@0 as id]
02)--BoundedWindowAggExec: wdw=[row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW: Field { "row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW": UInt64 }, frame: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW], mode=[Sorted]
03)----ProjectionExec: expr=[id@0 as id]
04)------GlobalLimitExec: skip=5, fetch=5
05)--------SortExec: TopK(fetch=10), expr=[id@0 ASC NULLS LAST], preserve_partitioning=[false]
06)----------FilterExec: kind@1 = a
07)------------DataSourceExec: partitions=1, partition_sizes=[1]
```
### Expected behavior
On 54, the physical plan is:
```
physical_plan
01)ProjectionExec: expr=[row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW@1 as rn, id@0 as id]
02)--BoundedWindowAggExec: wdw=[row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW: Field { "row_number() ORDER BY [t.id ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW": UInt64 }, frame: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW], mode=[Sorted]
03)----SortExec: expr=[id@0 ASC NULLS LAST], preserve_partitioning=[false]
04)------ProjectionExec: expr=[id@0 as id]
05)--------GlobalLimitExec: skip=5, fetch=5
06)----------FilterExec: kind@1 = a, fetch=10
07)------------DataSourceExec: partitions=1, partition_sizes=[1]
```
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Window sort pushdown below LIMIT/OFFSET prevents early termination [datafusion]
Posted by "AdamGS (via GitHub)" <gi...@apache.org>.
AdamGS commented on issue #25378:
URL: https://github.com/apache/datafusion/issues/25378#issuecomment-5698451736
I saw some similar issues, but I don't think anything is exactly this? or at least they seem different enough.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Window sort pushdown below LIMIT/OFFSET prevents early termination [datafusion]
Posted by "AdamGS (via GitHub)" <gi...@apache.org>.
AdamGS commented on issue #25378:
URL: https://github.com/apache/datafusion/issues/25378#issuecomment-5698445165
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Window sort pushdown below LIMIT/OFFSET prevents early termination [datafusion]
Posted by "blaginin (via GitHub)" <gi...@apache.org>.
blaginin closed issue #25378: Window sort pushdown below LIMIT/OFFSET prevents early termination
URL: https://github.com/apache/datafusion/issues/25378
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org