[jira] [Resolved] (PIG-1892) Bug in new logical plan : No output generated even though there are valid records

["Daniel Dai (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai resolved PIG-1892.
-----------------------------

      Resolution: Fixed
    Hadoop Flags: [Reviewed]

Test-patch:
     [exec] -1 overall.  
     [exec] 
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec] 
     [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
     [exec] 
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec] 
     [exec]     -1 release audit.  The applied patch generated 543 release audit warnings (more than the trunk's current 540 warnings).

All tests pass.

Patch committed to both trunk and 0.8 branch.

> Bug in new logical plan : No output generated even though there are valid records
> ---------------------------------------------------------------------------------
>
>                 Key: PIG-1892
>                 URL: https://issues.apache.org/jira/browse/PIG-1892
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.0, 0.9.0
>            Reporter: Vivek Padmanabhan
>            Assignee: Daniel Dai
>             Fix For: 0.8.0
>
>         Attachments: PIG-1892-1.patch, PIG-1892-2.patch
>
>
> I have the below script which provides me no output even though there are valid records in relation B which is used for the left out join.
> A0 = load 'input' using Maploader()  as ( map1, map2, map3 );
> A = filter A0 by ( (map2#'params'#'prop' == 464)   and (map2#'params'#'query' is not null) );
> B0 = filter A by (map1#'type' == 'c');
> B = filter B0 by ( map2#'info'#'s' matches 'aaaa|bbb|cccc');
> C =  filter A by (map1#'type' == 'p');
> D = join B by map2#'params'#'query' LEFT OUTER , C by map2#'params'#'query';
> store D into 'output';
> This is a bug with the newlogical plan.  From the plan i can see that  map1#'type'  and map2#'info'#'s' is not marked as RequiredKeys ,
> but where as all the fields reffered in the firts filter statement is marked as required.
> For the script to work I have to turn off the coloumn prune optimizer by -t ColumnMapKeyPrune or rearrange the script such that;
> B0 = filter A0 by ( (map2#'params'#'prop' == 464)   and (map2#'params'#'query' is not null) and (map1#'type' == 'c') );
> C =  filter A0 by ( (map2#'params'#'prop' == 464)   and (map2#'params'#'query' is not null) and (map1#'type' == 'p') );

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira