[jira] [Commented] (HIVE-10162) LLAP: Avoid deserializing the plan > 1 times in a single thread

["Gunther Hagleitner (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-10162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14393446#comment-14393446 ] 

Gunther Hagleitner commented on HIVE-10162:
-------------------------------------------

yeah, i broke that. i thought that we won't be able to use thread locals for plan objects anymore. turns out we still use one thread per query fragment. MRLegacyInput makes sure that the same thread is used for both input and processor and apparently we currently start a new thread per child.

long story short - i can switch back to the thread local model for now. however, that's brittle, stupid and inefficient. usually we only allow two of those :-)

> LLAP: Avoid deserializing the plan > 1 times in a single thread
> ---------------------------------------------------------------
>
>                 Key: HIVE-10162
>                 URL: https://issues.apache.org/jira/browse/HIVE-10162
>             Project: Hive
>          Issue Type: Sub-task
>    Affects Versions: llap
>            Reporter: Gopal V
>            Assignee: Gunther Hagleitner
>             Fix For: llap
>
>         Attachments: deserialize-plan-1.png, deserialize-plan-2.png
>
>
> Kryo shows up in the critical hot-path for LLAP when using a plan with a very large filter condition, due to the fact that the plan is deserialized more than once for each task.
> !deserialize-plan-1.png!
> !deserialize-plan-2.png!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)