You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "jackwener (via GitHub)" <gi...@apache.org> on 2023/02/10 04:21:45 UTC
[GitHub] [arrow-datafusion] jackwener commented on a diff in pull request #5156: Always wrapping OnceAsync for the inner table side in NestedLoopJoinExec
jackwener commented on code in PR #5156:
URL: https://github.com/apache/arrow-datafusion/pull/5156#discussion_r1102258731
##########
datafusion/core/src/physical_plan/joins/nested_loop_join.rs:
##########
@@ -50,9 +50,27 @@ use crate::error::Result;
use crate::execution::context::TaskContext;
use crate::physical_plan::coalesce_batches::concat_batches;
-/// Data of the left side
+/// Data of the inner table side
type JoinLeftData = RecordBatch;
+/// NestedLoopJoinExec executes partitions in parallel.
+/// One input will be collected to a single partition, call it inner-table.
+/// The other side of the input is treated as outer-table, and the output Partitioning is from it.
+/// Giving an output partition number x, the execution will be:
+///
+/// ```text
+/// for outer-table-batch in outer-table-partition-x
+/// check-join(outer-table-batch, inner-table-data)
+/// ```
+///
+/// One of the inputs will become inner table, and it is decided by the join type.
+/// Following is the relation table:
+///
+/// ```text
+/// JoinType Distribution Inner-table
+/// Inner/Left/LeftSemi/LeftAnti (UnspecifiedDistribution, SinglePartition) right
+/// Right/RightSemi/RightAnti/Full (SinglePartition, UnspecifiedDistribution) left
+/// ```
Review Comment:
Good summary👍. May we use the `table` format.
```suggestion
/// | JoinType | Distribution (left, right) | Inner-table |
/// |--------------------------------|--------------------------------------------|-------------|
/// | Inner/Left/LeftSemi/LeftAnti | (UnspecifiedDistribution, SinglePartition) | right |
/// | Right/RightSemi/RightAnti/Full | (SinglePartition, UnspecifiedDistribution) | left |
```
##########
datafusion/core/src/physical_plan/joins/nested_loop_join.rs:
##########
@@ -50,9 +50,27 @@ use crate::error::Result;
use crate::execution::context::TaskContext;
use crate::physical_plan::coalesce_batches::concat_batches;
-/// Data of the left side
+/// Data of the inner table side
type JoinLeftData = RecordBatch;
+/// NestedLoopJoinExec executes partitions in parallel.
+/// One input will be collected to a single partition, call it inner-table.
+/// The other side of the input is treated as outer-table, and the output Partitioning is from it.
+/// Giving an output partition number x, the execution will be:
+///
+/// ```text
+/// for outer-table-batch in outer-table-partition-x
+/// check-join(outer-table-batch, inner-table-data)
+/// ```
+///
+/// One of the inputs will become inner table, and it is decided by the join type.
+/// Following is the relation table:
+///
+/// ```text
+/// JoinType Distribution Inner-table
+/// Inner/Left/LeftSemi/LeftAnti (UnspecifiedDistribution, SinglePartition) right
+/// Right/RightSemi/RightAnti/Full (SinglePartition, UnspecifiedDistribution) left
+/// ```
Review Comment:
Good summary👍. May we use the `table` format?
```suggestion
/// | JoinType | Distribution (left, right) | Inner-table |
/// |--------------------------------|--------------------------------------------|-------------|
/// | Inner/Left/LeftSemi/LeftAnti | (UnspecifiedDistribution, SinglePartition) | right |
/// | Right/RightSemi/RightAnti/Full | (SinglePartition, UnspecifiedDistribution) | left |
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org