[jira] [Commented] (PIG-3978) Container reuse does not across PigServer

["Daniel Dai (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-3978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015723#comment-14015723 ] 

Daniel Dai commented on PIG-3978:
---------------------------------

RB link: https://reviews.apache.org/r/22155/

> Container reuse does not across PigServer
> -----------------------------------------
>
>                 Key: PIG-3978
>                 URL: https://issues.apache.org/jira/browse/PIG-3978
>             Project: Pig
>          Issue Type: Bug
>          Components: tez
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.14.0
>
>         Attachments: PIG-3978-1.patch
>
>
> Container reuse across PigServer does not happen in current code. We use different TezResourceManager per PigServer, and different TezResourceManager will upload LocalResource to different staging directory, so LocalResource is different for different PigServer, thus no container reuse across PigServer. This will affect Pig embedding where multiple PigServer is used for a Pig script. 
> The issue is partly due to PIG-3785 which we switch TezResourceManager to non-singleton. The root cause for PIG-3785 is not the singleton issue. The real issue is TezResourceManager.addTezResource is not synchronized, there is a race condition when one PigServer already upload the resource and use it in container, and the other PigServer overwrite at the same time.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)