You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "nuno-faria (via GitHub)" <gi...@apache.org> on 2026/03/03 12:57:49 UTC

[I] Right joins with empty schema return invalid results [datafusion]

nuno-faria opened a new issue, #20669:
URL: https://github.com/apache/datafusion/issues/20669

   ### Describe the bug
   
   This is a regression found on `branch-53`.
   
   When a physical right join (e.g., from a `RightAnti`) is executed with an empty schema (e.g., from `count(*)` queries), it returns invalid results. The issue comes from the `build_batch_from_indices` function:
   https://github.com/apache/datafusion/blob/d1a305821f4d883a2a1e13d0bbc54af43d6d0441/datafusion/physical-plan/src/joins/utils.rs#L913-L925
   
   If the schema is empty, an empty batch with `build_indices.len()` rows is returned. However, if the join type is, e.g., `RightAnti`, the correct number of rows should be retrieved from the probe side.
   
   One way I see of fixing this would be to add the join type to `build_batch_from_indices`, since I don't think that info is available from the current arguments. However this might require a large number of changes.
   
   ### To Reproduce
   
   ```sql
   create table t1 (k int, v int);
   create table t2 (k int, v int);
   insert into t1 select i as k, i as v from generate_series(1, 100) t(i);
   insert into t2 values (1, 1);
   
   -- select * is ok
   with t as (
       select *
       from t1
       left anti join t2 on t1.k = t2.k
   )
   select *
   from t;
   +----+----+
   | k  | v  |
   +----+----+
   | 2  | 2  |
   | 3  | 3  |
   | 4  | 4  |
   | 5  | 5  |
   ...
   +----+----+
   99 row(s) fetched.
   
   -- select count(*) is wrong
   with t as (
       select *
       from t1
       left anti join t2 on t1.k = t2.k
   )
   select count(*)
   from t;
   +----------+
   | count(*) |
   +----------+
   | 1        |
   +----------+
   1 row(s) fetched.
   ```
   
   ### Expected behavior
   
   Return the correct number of rows.
   
   ### Additional context
   
   This bug was exposed by the empty project optimization done by #20191, but the behavior of always returning the number of rows from the build side already existed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Right joins with empty schema return invalid results [datafusion]

Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on issue #20669:
URL: https://github.com/apache/datafusion/issues/20669#issuecomment-3998801327

   @nuno-faria is this a hash join?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Right joins with empty schema return invalid results [datafusion]

Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead closed issue #20669: Right joins with empty schema return invalid results
URL: https://github.com/apache/datafusion/issues/20669


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Right joins with empty schema return invalid results [datafusion]

Posted by "nuno-faria (via GitHub)" <gi...@apache.org>.
nuno-faria commented on issue #20669:
URL: https://github.com/apache/datafusion/issues/20669#issuecomment-4000020439

   > [@nuno-faria](https://github.com/nuno-faria) is this a hash join?
   
   Yes, here is the full plan:
   ```sql
   ProjectionExec: expr=[count(Int64(1))@0 as count(*)], metrics=[output_rows=1, elapsed_compute=5.80µs, output_bytes=8.0 B, output_batches=1, expr_0_eval_time=500ns]
     AggregateExec: mode=Final, gby=[], aggr=[count(Int64(1))], metrics=[output_rows=1, elapsed_compute=66.50µs, output_bytes=8.0 B, output_batches=1]
       CoalescePartitionsExec, metrics=[output_rows=12, elapsed_compute=85.80µs, output_bytes=96.0 B, output_batches=12]
         AggregateExec: mode=Partial, gby=[], aggr=[count(Int64(1))], metrics=[output_rows=12, elapsed_compute=474.00µs, output_bytes=96.0 B, output_batches=12]
           RepartitionExec: partitioning=RoundRobinBatch(12), input_partitions=1, metrics=[output_rows=1, elapsed_compute=49.80µs, output_bytes=0.0 B, output_batches=1, spill_count=0, spilled_bytes=0.0 B, spilled_rows=0, fetch_time=288.20µs, repartition_time=1ns, send_time=10.11µs]
             HashJoinExec: mode=CollectLeft, join_type=RightAnti, on=[(k@0, k@0)], projection=[], metrics=[output_rows=1, elapsed_compute=203.60µs, output_bytes=0.0 B, output_batches=1, array_map_created_count=1, build_input_batches=1, build_input_rows=1, input_batches=1, input_rows=100, build_mem_used=20, build_time=150.80µs, join_time=52.80µs, avg_fanout=100% (1/1), probe_hit_rate=1% (1/100)]
               DataSourceExec: partitions=1, partition_sizes=[1], metrics=[]
               DataSourceExec: partitions=1, partition_sizes=[1], metrics=[]
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org