You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "nuno-faria (via GitHub)" <gi...@apache.org> on 2026/03/03 12:57:49 UTC
[I] Right joins with empty schema return invalid results [datafusion]
nuno-faria opened a new issue, #20669:
URL: https://github.com/apache/datafusion/issues/20669
### Describe the bug
This is a regression found on `branch-53`.
When a physical right join (e.g., from a `RightAnti`) is executed with an empty schema (e.g., from `count(*)` queries), it returns invalid results. The issue comes from the `build_batch_from_indices` function:
https://github.com/apache/datafusion/blob/d1a305821f4d883a2a1e13d0bbc54af43d6d0441/datafusion/physical-plan/src/joins/utils.rs#L913-L925
If the schema is empty, an empty batch with `build_indices.len()` rows is returned. However, if the join type is, e.g., `RightAnti`, the correct number of rows should be retrieved from the probe side.
One way I see of fixing this would be to add the join type to `build_batch_from_indices`, since I don't think that info is available from the current arguments. However this might require a large number of changes.
### To Reproduce
```sql
create table t1 (k int, v int);
create table t2 (k int, v int);
insert into t1 select i as k, i as v from generate_series(1, 100) t(i);
insert into t2 values (1, 1);
-- select * is ok
with t as (
select *
from t1
left anti join t2 on t1.k = t2.k
)
select *
from t;
+----+----+
| k | v |
+----+----+
| 2 | 2 |
| 3 | 3 |
| 4 | 4 |
| 5 | 5 |
...
+----+----+
99 row(s) fetched.
-- select count(*) is wrong
with t as (
select *
from t1
left anti join t2 on t1.k = t2.k
)
select count(*)
from t;
+----------+
| count(*) |
+----------+
| 1 |
+----------+
1 row(s) fetched.
```
### Expected behavior
Return the correct number of rows.
### Additional context
This bug was exposed by the empty project optimization done by #20191, but the behavior of always returning the number of rows from the build side already existed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Right joins with empty schema return invalid results [datafusion]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on issue #20669:
URL: https://github.com/apache/datafusion/issues/20669#issuecomment-3998801327
@nuno-faria is this a hash join?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Right joins with empty schema return invalid results [datafusion]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead closed issue #20669: Right joins with empty schema return invalid results
URL: https://github.com/apache/datafusion/issues/20669
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Right joins with empty schema return invalid results [datafusion]
Posted by "nuno-faria (via GitHub)" <gi...@apache.org>.
nuno-faria commented on issue #20669:
URL: https://github.com/apache/datafusion/issues/20669#issuecomment-4000020439
> [@nuno-faria](https://github.com/nuno-faria) is this a hash join?
Yes, here is the full plan:
```sql
ProjectionExec: expr=[count(Int64(1))@0 as count(*)], metrics=[output_rows=1, elapsed_compute=5.80µs, output_bytes=8.0 B, output_batches=1, expr_0_eval_time=500ns]
AggregateExec: mode=Final, gby=[], aggr=[count(Int64(1))], metrics=[output_rows=1, elapsed_compute=66.50µs, output_bytes=8.0 B, output_batches=1]
CoalescePartitionsExec, metrics=[output_rows=12, elapsed_compute=85.80µs, output_bytes=96.0 B, output_batches=12]
AggregateExec: mode=Partial, gby=[], aggr=[count(Int64(1))], metrics=[output_rows=12, elapsed_compute=474.00µs, output_bytes=96.0 B, output_batches=12]
RepartitionExec: partitioning=RoundRobinBatch(12), input_partitions=1, metrics=[output_rows=1, elapsed_compute=49.80µs, output_bytes=0.0 B, output_batches=1, spill_count=0, spilled_bytes=0.0 B, spilled_rows=0, fetch_time=288.20µs, repartition_time=1ns, send_time=10.11µs]
HashJoinExec: mode=CollectLeft, join_type=RightAnti, on=[(k@0, k@0)], projection=[], metrics=[output_rows=1, elapsed_compute=203.60µs, output_bytes=0.0 B, output_batches=1, array_map_created_count=1, build_input_batches=1, build_input_rows=1, input_batches=1, input_rows=100, build_mem_used=20, build_time=150.80µs, join_time=52.80µs, avg_fanout=100% (1/1), probe_hit_rate=1% (1/100)]
DataSourceExec: partitions=1, partition_sizes=[1], metrics=[]
DataSourceExec: partitions=1, partition_sizes=[1], metrics=[]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org