[jira] [Comment Edited] (PIG-4944) Reset UDFContext#jobConf in spark mode

["Xianda Ke (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15363799#comment-15363799 ] 

Xianda Ke edited comment on PIG-4944 at 7/6/16 5:27 AM:
--------------------------------------------------------

In MR mode.  LocalJobRunner create a new thread for different UT cases. Thread_local jobConf is set in this new created thread.
In spark mode(Local mode), all the ut cases share the same main thread.  UDFContext's JobConf is reset in PigInputFormat.passLoadSignature()

LGTM.  It is the right place to clean JobConf in end of SparkLauncher.launchPig()
+1



was (Author: kexianda):
In MR mode.  LocalJobRunner create a new thread for different UT cases. Thread_local jobConf is set in this new created thread.
In spark mode(Local mode), all the ut cases share the same main thread.  

LGTM.  It is the right place to clean JobConf in end of SparkLauncher.launchPig()
+1


> Reset UDFContext#jobConf in spark mode
> --------------------------------------
>
>                 Key: PIG-4944
>                 URL: https://issues.apache.org/jira/browse/PIG-4944
>             Project: Pig
>          Issue Type: Bug
>            Reporter: liyunzhang_intel
>            Assignee: liyunzhang_intel
>         Attachments: PIG-4944.patch, TestEvalPipelineLocal.mr, TestEvalPipelineLocal.spark
>
>
> Community gave some comments about TestEvalPipelineLocal unit test:
> https://reviews.apache.org/r/45667/#comment199056
> We can reset "UDFContext.getUDFContext().addJobConf(null)" in other place not  in TestEvalPipelineLocal#testSetLocationCalledInFE



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)