You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by Daniel Dai <da...@gmail.com> on 2011/01/26 19:58:49 UTC
Review Request: Problem with DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/357/
-----------------------------------------------------------
Review request for pig, thejas and Alan Gates.
Summary
-------
The following script fail:
pkg.txt
a 3 {(123,1.0),(236,2.0)}
a 3 {(236,1.0)}
model.txt
a 123 2 0.33
a 236 2 0.5
My script is listed below:
A = load 'pkg.txt' using PigStorage('\t') as (pkg:chararray, ts:int, cat_bag:{t:(id:chararray, wht:float)});
M = load 'model.txt' using PigStorage('\t') as (pkg:chararray, cat_id:chararray, ts:int, score:double);
B = foreach A generate ts, pkg, flatten(cat_bag.id) as (cat_id:chararray);
B = distinct B;
H1 = cogroup M by (pkg, cat_id) inner, B by (pkg, cat_id);
H2 = foreach H1 { I = order M by ts; J = order B by ts; generate flatten(group) as (pkg:chararray, cat_id:chararray), J.ts as tsorig, I.ts as tsmap; }
dump H2;
Pig Stack Trace
---------------
ERROR 2043: Unexpected error during execution.
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias H2
at org.apache.pig.PigServer.openIterator(PigServer.java:764)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
at org.apache.pig.Main.run(Main.java:500)
at org.apache.pig.Main.main(Main.java:107)
Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias H2
at org.apache.pig.PigServer.storeEx(PigServer.java:888)
at org.apache.pig.PigServer.store(PigServer.java:826)
at org.apache.pig.PigServer.openIterator(PigServer.java:738)
... 7 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:403)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1208)
at org.apache.pig.PigServer.storeEx(PigServer.java:884)
... 9 more
Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad cannot be cast to org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:246)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:41)
at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:498)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:117)
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:378)
... 11 more
This addresses bug PIG-1812.
https://issues.apache.org/jira/browse/PIG-1812
Diffs
-----
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/KeyTypeDiscoveryVisitor.java 1063809
http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MRCompiler.java 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestEvalPipeline2.java 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestMRCompiler.java 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC1.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC10.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC11.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC12.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC13.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC14.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC15.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC16.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC17.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC18.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC19.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC2.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC3.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC5.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC6.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC7.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC8.gld 1063809
http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/MRC9.gld 1063809
Diff: https://reviews.apache.org/r/357/diff
Testing
-------
Test-patch:
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 60 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
Unit-test:
all pass
End-to-end test:
all pass
Thanks,
Daniel