[jira] [Resolved] (PIG-3492) ColumnPrune dropping used column due to LogicalRelationalOperator.fixDuplicateUids changes not propagating

["Daniel Dai (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai resolved PIG-3492.
-----------------------------

       Resolution: Fixed
    Fix Version/s: 0.12.1
     Hadoop Flags: Reviewed

pig-3492-trunk_04.patch and pig-3492-trunk-delta.patch are committed to both trunk and 0.12 branch. I don't think we will have additional 0.11 release, so skip 0.11 branch.

I will open additional Jira to formalize the validator. 

Thanks Koji, that's a really important fix.

> ColumnPrune dropping used column due to LogicalRelationalOperator.fixDuplicateUids changes not propagating
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-3492
>                 URL: https://issues.apache.org/jira/browse/PIG-3492
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.11.1, 0.12.1, 0.13.0
>            Reporter: Koji Noguchi
>            Assignee: Koji Noguchi
>             Fix For: 0.12.1
>
>         Attachments: pig-3492-trunk_04.patch, PIG-3492-trunk-delta.patch, pig-3492-v0.12_01.patch
>
>
> I don't have a testcase I can upload at the moment, but here's my observation.
> SplitFilter -> schemaResetter -> LOGenerate.getSchema -> LogicalRelationalOperator.fixDuplicateUids() creating a new UID but that UID is not propagated to the entire plan (since SplitFilter.reportChanges only returns subplan).
> As a result, I am seeing ColumnPruning cutting off those used columns.



--
This message was sent by Atlassian JIRA
(v6.1#6144)