You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Rui Wang (JIRA)" <ji...@apache.org> on 2019/08/01 17:39:00 UTC
[jira] [Commented] (BEAM-6114) SQL join selection should be done in
planner, not in expansion to PTransform
[ https://issues.apache.org/jira/browse/BEAM-6114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898224#comment-16898224 ]
Rui Wang commented on BEAM-6114:
--------------------------------
[~rahul8383]
If you just convert the three if else conditions to rules, that should be good enough at the beginning: https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamJoinRel.java#L161
My memory is if either side of PCollection is bounded will go to sideinput join. So only unbounded join unbounded falls into CoGBK join. But my memory might be stale already.
For unsupported join, the better way is to have a rule to match it but the rule will throw unsupported exception(so not in rel). But it could be just easy to throw such exceptions in , etc. SideInputJoin, at the beginning.
Sounds good to handle BEAM-7835 later.
> SQL join selection should be done in planner, not in expansion to PTransform
> ----------------------------------------------------------------------------
>
> Key: BEAM-6114
> URL: https://issues.apache.org/jira/browse/BEAM-6114
> Project: Beam
> Issue Type: Improvement
> Components: dsl-sql
> Reporter: Kenneth Knowles
> Assignee: Rahul Patwari
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Currently Beam SQL joins all go through a single physical operator which has a single PTransform that does all join algorithms based on properties of its input PCollections as well as the relational algebra.
> A first step is to make the needed information part of the relational algebra, so it can choose a PTransform based on that, and the PTransforms can be simpler.
> Second step is to have separate (physical) relational operators for different join algorithms.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)