[jira] [Comment Edited] (HIVE-17585) Improve thread safety when loading dynamic partitions in parallel

["Sergey Shelukhin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-17585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16180102#comment-16180102 ] 

Sergey Shelukhin edited comment on HIVE-17585 at 9/26/17 2:22 AM:
------------------------------------------------------------------

Hmm... wouldn't a simpler solution be to run Hive.get() on the callable (make callable static to make sure it cannot access "this"), given that Hive object is not thread safe and so the original code uses it incorrectly by calling loadPartition from multiple threads?
Right now it's synchronizing a set of methods called by loadPartition; if someone changes what loadPartition calls, this will break again as far as I can tell. And it's not good to change every method to use synchronized MSC, that will just be a perf hit.
Unless I'm missing something.


was (Author: sershe):
Hmm... wouldn't a simpler solution be to run Hive.get() instead of synchronizing a set of methods called by loadPartition, given that Hive object is not thread safe and so the original code uses it incorrectly by calling loadPartition from multiple threads?
If someone changes what loadPartition calls, this will break again as far as I can tell. And it's not good to change every method to use synchronized MSC, that will just be a perf hit.
Unless I'm missing something.

> Improve thread safety when loading dynamic partitions in parallel
> -----------------------------------------------------------------
>
>                 Key: HIVE-17585
>                 URL: https://issues.apache.org/jira/browse/HIVE-17585
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Tao Li
>            Assignee: Tao Li
>             Fix For: 3.0.0
>
>         Attachments: HIVE-17585.1.patch, HIVE-17585.2.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)