Review Request: Pig 0.8 throws ERROR 1075 while trying to refer a map in the result of eval udf.Works with 0.7

[Daniel Dai <da...@gmail.com>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/353/
-----------------------------------------------------------

Review request for pig and thejas.


Summary
-------

The following script fail:
public static class BagGenerateNoSchema extends EvalFunc<DataBag> {
        @Override
        public DataBag exec(Tuple input) throws IOException {
            DataBag bg = DefaultBagFactory.getInstance().newDefaultBag();
            bg.add(input);
            return bg;
        }
}

a = load '1.txt' as (a0:map[]);
b = foreach a generate BagGenerateNoSchema(*) as b0;
c = foreach b generate flatten(IdentityColumn(b0));
d = foreach c generate $0#'key';
dump d;

Error message:
org.apache.pig.backend.executionengine.ExecException: ERROR 1075: Received a bytearray from the UDF. Cannot determine how to convert the bytearray to map.
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:952)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.processInput(POMapLookUp.java:87)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNext(POMapLookUp.java:98)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POMapLookUp.getNext(POMapLookUp.java:117)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:346)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:291)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:236)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:638)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:314)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1062)
at org.apache.hadoop.mapred.Child.main(Child.java:211)


This addresses bug PIG-1813.
    https://issues.apache.org/jira/browse/PIG-1813


Diffs
-----

  http://svn.apache.org/repos/asf/pig/branches/branch-0.8/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/POUserFunc.java 1062989 
  http://svn.apache.org/repos/asf/pig/branches/branch-0.8/src/org/apache/pig/newplan/logical/relational/LOGenerate.java 1062989 
  http://svn.apache.org/repos/asf/pig/branches/branch-0.8/test/org/apache/pig/test/TestEvalPipeline2.java 1062989 

Diff: https://reviews.apache.org/r/353/diff


Testing
-------

Test-patch:
     [exec] +1 overall.  
     [exec] 
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec] 
     [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
     [exec] 
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec] 
     [exec]     +1 release audit.  The applied patch does not increase the total number of release audit warnings.

Unit test:
    all pass

End-to-end test:
    all pass


Thanks,

Daniel