[jira] [Commented] (PIG-3492) ColumnPrune dropping used column due to LogicalRelationalOperator.fixDuplicateUids changes not propagating

["Daniel Dai (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13783228#comment-13783228 ] 

Daniel Dai commented on PIG-3492:
---------------------------------

Yes, I also see couple of places fixDuplicateUids is get misused. uid play a vital role in ColumnPruner. So every time we reassign uid, we need to make sure operator has the knowledge how does that uid get generated, and convey it to ColumnPruner.

> ColumnPrune dropping used column due to LogicalRelationalOperator.fixDuplicateUids changes not propagating
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-3492
>                 URL: https://issues.apache.org/jira/browse/PIG-3492
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.11.1, 0.12.1, 0.13.0
>            Reporter: Koji Noguchi
>
> I don't have a testcase I can upload at the moment, but here's my observation.
> SplitFilter -> schemaResetter -> LOGenerate.getSchema -> LogicalRelationalOperator.fixDuplicateUids() creating a new UID but that UID is not propagated to the entire plan (since SplitFilter.reportChanges only returns subplan).
> As a result, I am seeing ColumnPruning cutting off those used columns.



--
This message was sent by Atlassian JIRA
(v6.1#6144)