[jira] [Commented] (PIG-3120) setStoreFuncUDFContextSignature called with null signature

["Joseph Adler (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13553352#comment-13553352 ] 

Joseph Adler commented on PIG-3120:
-----------------------------------

OK, tracked down the issue. It looks like the UDFContextSignature is not getting propagated if there is a LIMIT statement in the pig code.

Very specifically, in org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.LimitAdjuster.adjust, it looks like Pig was creating a new POStore object but not copying the signature. Here is the offending code:

{code}
            // this is line 132...
            POStore st = new POStore(new OperatorKey(scope,nig.getNextNodeId(scope)));
            st.setSFile(oldSpec);
            st.setIsTmpStore(oldIsTmpStore);
            st.setSchema(((POStore)mpLeaf).getSchema());
            
            limitAdjustMROp.reducePlan.addAsLeaf(st);
{code}

This is easily fixable by inserting this statement at line 137:

{code}
            st.setSignature(((POStore)mpLeaf).getSignature());
{code}
I'll follow up with a path for this issue.
                
> setStoreFuncUDFContextSignature called with null signature
> ----------------------------------------------------------
>
>                 Key: PIG-3120
>                 URL: https://issues.apache.org/jira/browse/PIG-3120
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.12
>            Reporter: Joseph Adler
>            Priority: Critical
>             Fix For: 0.12
>
>
> I'm currently working on PIG-3015 and am having trouble passing the UDFContextSignature to the store func. It looks like the store func on the head end is being set to a non-null value, but a null value is being passed to setStoreFuncUDFContextSignature on the back end. I'm opening this ticket to track this issue; I'll follow up with a reproducible test case when I have a clean one.
> I suspect this problem occurs when running on a real cluster, but may not occur in the standard unit tests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira