[jira] [Updated] (PIG-4703) TezOperator.stores shall not ship to backend

["Daniel Dai (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai updated PIG-4703:
----------------------------
    Attachment: PIG-4703-1.patch

> TezOperator.stores shall not ship to backend
> --------------------------------------------
>
>                 Key: PIG-4703
>                 URL: https://issues.apache.org/jira/browse/PIG-4703
>             Project: Pig
>          Issue Type: Bug
>          Components: tez
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.16.0, 0.15.1
>
>         Attachments: PIG-4703-1.patch
>
>
> We end up of shipping all physical plan to the backend. One direct result is UDFContext is not set in OutputCommitter. Here is how:
> 1. PigGraceShuffleVertexManager deserialize "pig.tez.plan". TezOperator.stores reference physical plan so physical plan get deserialize.
> 2. When deserialize POUserFunc, POUserFunc.instantiateFunc is invoked. Inside it, POUserFunc.setFuncInputSchema would use UDFContext and leave an entry there (though it is empty map).
> 3. MapRedUtil.setupUDFContext would only deserialize if UDFContext is empty, since we already touch UDFContext with some maps, UDFContext is not deserialized



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)