[jira] [Commented] (HIVE-16690) Configure Tez cartesian product edge based on LLAP cluster size

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16090585#comment-16090585 ] 

Siddharth Seth commented on HIVE-16690:
---------------------------------------

+1 for the addendum. If cross product does end up getting invoked, is it possible to log the configured value for parallelism? This can be in a follow up jira. Also, I think this clusterstate lookup is very avoidable if cross product is not invoked. Don't need to invoke the ZK registry for each query (even though it does cache records for 2 minutes)

> Configure Tez cartesian product edge based on LLAP cluster size
> ---------------------------------------------------------------
>
>                 Key: HIVE-16690
>                 URL: https://issues.apache.org/jira/browse/HIVE-16690
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhiyuan Yang
>            Assignee: Zhiyuan Yang
>         Attachments: HIVE-16690.1.patch, HIVE-16690.addendum.patch
>
>
> In HIVE-14731 we are using default value for target parallelism of fair cartesian product edge. Ideally this should be set according to cluster size. In case of LLAP it's pretty easy to get cluster size, i.e., number of executors.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)