You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kenneth Knowles (Jira)" <ji...@apache.org> on 2022/06/03 22:53:00 UTC

[jira] [Commented] (BEAM-7151) Support conjunction clause when it's only equi-join

    [ https://issues.apache.org/jira/browse/BEAM-7151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17547415#comment-17547415 ] 

Kenneth Knowles commented on BEAM-7151:
---------------------------------------

This issue has been migrated to https://github.com/apache/beam/issues/19176

> Support conjunction clause when it's only equi-join
> ---------------------------------------------------
>
>                 Key: BEAM-7151
>                 URL: https://issues.apache.org/jira/browse/BEAM-7151
>             Project: Beam
>          Issue Type: Sub-task
>          Components: dsl-sql
>            Reporter: Rui Wang
>            Priority: P3
>
> conjunction_clause: function_call(function_parameter, ...) | field_access | column
> function_parameter: function_call | field_access
> In Beam, equi-join is implemented by CoGBK, which requires both join inputs (assume binary join) to build PCollection of KV<Row, Row>, where the key is join key.
> For equi-join, conjunction clause is essentially an equation. In order to build KV<Row, Row>, it requires that columns from different sides of equation should come from different join input. For example, a + b = 2 cannot be used to build join key but a = 2 - b can. So rewriting is required for clauses when it does not satisfy this property. 
> It also implies that not every clause is rewritable. Say the clause is f(a, b) = 3, in which a is from left input and b is from right input. If this function f is not splittable, such that we cannot move a or b to right side of equation, then we cannot support this clause in BeamSQL's  join.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)