[jira] [Commented] (CALCITE-2979) Add a block-based nested loop join algorithm

["Stamatis Zampetakis (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809884#comment-16809884 ] 

Stamatis Zampetakis commented on CALCITE-2979:
----------------------------------------------

Implementation wise the first idea that comes to my mind is very similar to how the current single value implementation works. Instead of creating a filter with one correlated variable on the right side, we should create a filter with either:
 * BLK_SIZE correlated variables;
 * a single correlated variable of ARRAY type and BLK_SIZE;

that are combined with an OR predicate with BLK_SIZE disjunctions.

For example, starting with the following pseudo-plan:
{noformat}
Join(A.id > B.id)
  Scan(A)
  Scan(B)
{noformat}
the rules should generate something like the plans below:
{noformat}
NestedLoop(blockSize=3)
  Scan(A)
  Filter(OR(>(cor0_0,B.id), >(cor0_1,B.id), >(cor0_2,B.id))
    Scan(B)
{noformat}
or
{noformat}
NestedLoop(blockSize=3)
  Scan(A)
  Filter(OR(>(cor0[0],B.id), >(cor0[1],B.id), >(cor0[2],B.id))
    Scan(B)
{noformat}
which the implementation of Correlate(blockSize=3) should take into account.

> Add a block-based nested loop join algorithm
> --------------------------------------------
>
>                 Key: CALCITE-2979
>                 URL: https://issues.apache.org/jira/browse/CALCITE-2979
>             Project: Calcite
>          Issue Type: Improvement
>          Components: core
>    Affects Versions: 1.19.0
>            Reporter: Stamatis Zampetakis
>            Priority: Major
>              Labels: performance
>
> Currently Calcite provides a tuple-based nested loop join algorithm implmented through EnumerableCorrelate and EnumerableDefaults.correlateJoin. This means that for each tuple of the outer relation we probe (set variables) in the inner relation. 
> The goal of this issue is to add new algorithm (or extend the correlateJoin method) which first gathers blocks (batches) of tuples from the outer relation and then probes the inner relation once per block. 
> There are cases (eg., indexes) where the inner relation can be accessed  by more than one value which can greatly improve the performance in particular when the outer relation is big. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)