[jira] [Commented] (HIVE-8760) Pass a copy of HiveConf to hooks

["Gopal V (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200809#comment-14200809 ] 

Gopal V commented on HIVE-8760:
-------------------------------

This is a perf regression for all well-written Hive hooks that exist.

There is previous information to indicate copying Configuration is not a fast operation. In HIVE-4486 we've gone from a query which took 347.66 seconds down to 218 seconds by throwing out unnecessary {{new HiveConf();}} calls.

If this is a thread-safety issue, then the hook spawning its own threads should synchronize - since this is class base config, which is pluggable that is very clearly the minimum impact fix.

{code}
@Override
  public void run(final HookContext hookContext) throws Exception {
    final long currentTime = System.currentTimeMillis();
+  final HiveConf confCopy = new HiveConf(hookContext.getConf());
    executor.submit(new Runnable() {
... // use local value off the closure capture in thread runnable
{code}

> Pass a copy of HiveConf to hooks
> --------------------------------
>
>                 Key: HIVE-8760
>                 URL: https://issues.apache.org/jira/browse/HIVE-8760
>             Project: Hive
>          Issue Type: Bug
>          Components: Configuration
>    Affects Versions: 0.13.0, 0.14.0
>            Reporter: Ashutosh Chauhan
>            Assignee: Ashutosh Chauhan
>         Attachments: HIVE-8760.patch
>
>
> because hadoop's {{Configuration}} is not thread-safe



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)