[jira] [Resolved] (TAJO-1553) Improve broadcast join planning

["Jihoon Son (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TAJO-1553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jihoon Son resolved TAJO-1553.
------------------------------
    Resolution: Fixed

Committed to master.
Thanks for your review!

> Improve broadcast join planning
> -------------------------------
>
>                 Key: TAJO-1553
>                 URL: https://issues.apache.org/jira/browse/TAJO-1553
>             Project: Tajo
>          Issue Type: Improvement
>          Components: distributed query plan, planner/optimizer
>            Reporter: Jihoon Son
>            Assignee: Jihoon Son
>             Fix For: 0.11.0
>
>
> The global engine generates a logical plan, and then marks some parts of the plan as broadcast plan which means that they and their input will be broadcasted to all workers. 
> Currently, broadcast parts are identified according to some rigid and hard-coded rules. This will limit the broadcast opportunities in many cases.
> So, in this issue, I propose refactoring the broadcast planner to be more general.
> Broadcast parts can be identified recursively.
> * A leaf node will be broadcasted if its input size does not exceed the pre-defined threshold.
> * An intermediate node will be broadcasted if it has at least one broadcast child.
> * For outer joins, row-preserved tables must not be broadcasted to avoid input data duplication.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)