[jira] [Commented] (HIVE-10937) LLAP: make ObjectCache for plans work properly in the daemon

["Sergey Shelukhin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-10937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010143#comment-15010143 ] 

Sergey Shelukhin commented on HIVE-10937:
-----------------------------------------

On the cluster, with one daemon and 3 threads each running 4 smaller TPCDS 200 queries in parallel (27, 55, 52, 96), with plans obviously not shared between different instances of the same query, I see a plan is used on average 1.93 times. That is actually lower than I would have expected. In any case, this plan cache would only be good for wide queries and/or small and/or busy clusters, where a query would only get few executors per node and thus run sequentially and not in parallel. In parallel the plan cannot be reused it looks like. I actually wonder why...

> LLAP: make ObjectCache for plans work properly in the daemon
> ------------------------------------------------------------
>
>                 Key: HIVE-10937
>                 URL: https://issues.apache.org/jira/browse/HIVE-10937
>             Project: Hive
>          Issue Type: Sub-task
>          Components: llap
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-10937.01.patch, HIVE-10937.02.patch, HIVE-10937.03.patch, HIVE-10937.04.patch, HIVE-10937.05.patch, HIVE-10937.patch
>
>
> There's perf hit otherwise, esp. when stupid planner creates 1009 reducers of 4Mb each.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)