You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Fly-a-Kite (via GitHub)" <gi...@apache.org> on 2026/07/10 07:27:30 UTC
[I] Wrong result: grouped MAX(NULL) row is dropped with ORDER BY aggregate NULLS FIRST LIMIT [datafusion]
Fly-a-Kite opened a new issue, #23440:
URL: https://github.com/apache/datafusion/issues/23440
### Describe the bug
DataFusion 54.0.0 returns an empty result for a grouped aggregate query where the aggregate value is `NULL`, followed by `ORDER BY ... NULLS FIRST LIMIT`.
The grouped aggregate should still produce one output row. The final `ORDER BY` / `LIMIT` should not remove it.
This looks related to #22190, but I can still reproduce it on the latest `datafusion==54.0.0` with a smaller reproducer.
### To Reproduce
```python
import datafusion
import pyarrow as pa
from datafusion import SessionContext
print("datafusion", datafusion.__version__)
batch = pa.record_batch(
[
pa.array(["gamma"], type=pa.string()),
pa.array([None], type=pa.float64()),
],
names=["s", "y"],
)
ctx = SessionContext()
ctx.register_record_batches("t0", [[batch]])
query = """
SELECT max_y
FROM (
SELECT s, MAX(y) AS max_y
FROM t0
GROUP BY s
)
ORDER BY max_y DESC NULLS FIRST
LIMIT 3
"""
result = ctx.sql(query).to_pandas()
print(result)
print("row count:", len(result))
### Expected behavior
### Expected behavior
The query should return one row:
max_y
0 NULL
Reason: the input contains one group, s = 'gamma'. MAX(y) over a group where all y values are NULL should produce one grouped output row with max_y = NULL.
### Actual behavior
DataFusion 54.0.0 returns zero rows:
Empty DataFrame
Columns: [max_y]
Index: []
row count: 0
### Additional context
### Environment
- datafusion==54.0.0
- pyarrow==24.0.0
- Python 3.12.3
- Linux x86_64
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Wrong result: grouped MAX(NULL) row is dropped with ORDER BY aggregate NULLS FIRST LIMIT [datafusion]
Posted by "u70b3 (via GitHub)" <gi...@apache.org>.
u70b3 commented on issue #23440:
URL: https://github.com/apache/datafusion/issues/23440#issuecomment-5010653365
take — fix in #23684 (this also covers the same root cause as #22190)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Wrong result: grouped MAX(NULL) row is dropped with ORDER BY aggregate NULLS FIRST LIMIT [datafusion]
Posted by "SubhamSinghal (via GitHub)" <gi...@apache.org>.
SubhamSinghal commented on issue #23440:
URL: https://github.com/apache/datafusion/issues/23440#issuecomment-4938361766
Take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Wrong result: grouped MAX(NULL) row is dropped with ORDER BY aggregate NULLS FIRST LIMIT [datafusion]
Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew closed issue #23440: Wrong result: grouped MAX(NULL) row is dropped with ORDER BY aggregate NULLS FIRST LIMIT
URL: https://github.com/apache/datafusion/issues/23440
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org