[jira] [Commented] (IMPALA-9027) Update planner decisions to factor in mt_dop

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-9027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16967294#comment-16967294 ] 

ASF subversion and git services commented on IMPALA-9027:
---------------------------------------------------------

Commit ad43c873fa5d823a56ba93315a8e76e7cd6a0685 in impala's branch refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ad43c87 ]

IMPALA-9027: planner fixes for mt_dop

* Update the distributed planner to reflect that broadcast join tables
  are replicated in all fragments.
* Did a pass over the planner code looking at call sites of
  getNumNodes() to confirm that they shouldn't be replaced by
  getNumInstances()

Testing:
* Updated affected planner test where PARALLELPLANS had a
  different join strategy.
* Added a targeted test to mem-limit-broadcast-join.test to
  show that mt_dop affects join mode.
* Ran exhaustive tests.

Change-Id: I23395c2dadf6be0e8be99706ca3ab5f4964cbcf9
Reviewed-on: http://gerrit.cloudera.org:8080/14522
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Update planner decisions to factor in mt_dop
> --------------------------------------------
>
>                 Key: IMPALA-9027
>                 URL: https://issues.apache.org/jira/browse/IMPALA-9027
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Frontend
>            Reporter: Tim Armstrong
>            Assignee: Tim Armstrong
>            Priority: Major
>
> We need to make sure all decisions in the planner that currently use number of nodes use number of instances if appropriate.
> E.g.
> * HashJoinNode.computeNodeResourceProfile() has a TODO.
> * I think the broadcast/shuffle decision doesn't take into account mt_dop, which is wrong if we're broadcasting to multiple instances per fragment.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org