You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "xianyu (JIRA)" <ji...@apache.org> on 2011/01/20 03:06:44 UTC
[jira] Created: (PIG-1812) Problem with
DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
Problem with DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
--------------------------------------------
Key: PIG-1812
URL: https://issues.apache.org/jira/browse/PIG-1812
Project: Pig
Issue Type: Bug
Affects Versions: 0.8.0, 0.7.0
Environment: RHEL, Pig 0.8.0
Reporter: xianyu
Hi,
I have the following input files:
pkg.txt
a 3 {(123,1.0),(236,2.0)}
a 3 {(236,1.0)}
model.txt
a 123 2 0.33
a 236 2 0.5
My script is listed below:
A = load 'pkg.txt' using PigStorage('\t') as (pkg:chararray, ts:int, cat_bag:{t:(id:chararray, wht:float)});
M = load 'model.txt' using PigStorage('\t') as (pkg:chararray, cat_id:chararray, ts:int, score:double);
B = foreach A generate ts, pkg, flatten(cat_bag.id) as (cat_id:chararray);
B = distinct B;
H1 = cogroup M by (pkg, cat_id) inner, B by (pkg, cat_id);
H2 = foreach H1 {
I = order M by ts;
J = order B by ts;
generate flatten(group) as (pkg:chararray, cat_id:chararray), J.ts as tsorig, I.ts as tsmap;
}
dump H2;
When running this script, I got a warning about "Encountered Warning DID_NOT_FIND_LOAD_ONLY_MAP_PLAN 1 time(s)" and pig error log as below:
Pig Stack Trace
---------------
ERROR 2043: Unexpected error during execution.
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias H2
at org.apache.pig.PigServer.openIterator(PigServer.java:764)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
at org.apache.pig.Main.run(Main.java:500)
at org.apache.pig.Main.main(Main.java:107)
Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias H2
at org.apache.pig.PigServer.storeEx(PigServer.java:888)
at org.apache.pig.PigServer.store(PigServer.java:826)
at org.apache.pig.PigServer.openIterator(PigServer.java:738)
... 7 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:403)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1208)
at org.apache.pig.PigServer.storeEx(PigServer.java:884)
... 9 more
Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad cannot be cast to org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:246)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:41)
at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:498)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:117)
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:378)
... 11 more
But, when I removed the DISTINCT statement before COGROUP, i.e. "B = distinct B;" this script can run smoothly. I have also tried other reducer side operations like ORDER, it seems that they will also trigger above error. This is really very confusing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1812) Problem with
DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1812:
----------------------------
Attachment: PIG-1812-2.patch
> Problem with DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
> --------------------------------------------
>
> Key: PIG-1812
> URL: https://issues.apache.org/jira/browse/PIG-1812
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0, 0.8.0
> Environment: RHEL, Pig 0.8.0
> Reporter: xianyu
> Assignee: Daniel Dai
> Attachments: PIG-1812-1.patch, PIG-1812-2.patch
>
>
> Hi,
> I have the following input files:
> pkg.txt
> a 3 {(123,1.0),(236,2.0)}
> a 3 {(236,1.0)}
> model.txt
> a 123 2 0.33
> a 236 2 0.5
> My script is listed below:
> A = load 'pkg.txt' using PigStorage('\t') as (pkg:chararray, ts:int, cat_bag:{t:(id:chararray, wht:float)});
> M = load 'model.txt' using PigStorage('\t') as (pkg:chararray, cat_id:chararray, ts:int, score:double);
> B = foreach A generate ts, pkg, flatten(cat_bag.id) as (cat_id:chararray);
> B = distinct B;
> H1 = cogroup M by (pkg, cat_id) inner, B by (pkg, cat_id);
> H2 = foreach H1 {
> I = order M by ts;
> J = order B by ts;
> generate flatten(group) as (pkg:chararray, cat_id:chararray), J.ts as tsorig, I.ts as tsmap;
> }
> dump H2;
> When running this script, I got a warning about "Encountered Warning DID_NOT_FIND_LOAD_ONLY_MAP_PLAN 1 time(s)" and pig error log as below:
> Pig Stack Trace
> ---------------
> ERROR 2043: Unexpected error during execution.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias H2
> at org.apache.pig.PigServer.openIterator(PigServer.java:764)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> at org.apache.pig.Main.run(Main.java:500)
> at org.apache.pig.Main.main(Main.java:107)
> Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias H2
> at org.apache.pig.PigServer.storeEx(PigServer.java:888)
> at org.apache.pig.PigServer.store(PigServer.java:826)
> at org.apache.pig.PigServer.openIterator(PigServer.java:738)
> ... 7 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:403)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1208)
> at org.apache.pig.PigServer.storeEx(PigServer.java:884)
> ... 9 more
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad cannot be cast to org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:246)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:41)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:498)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:117)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:378)
> ... 11 more
> But, when I removed the DISTINCT statement before COGROUP, i.e. "B = distinct B;" this script can run smoothly. I have also tried other reducer side operations like ORDER, it seems that they will also trigger above error. This is really very confusing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1812) Problem with
DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12987878#action_12987878 ]
Alan Gates commented on PIG-1812:
---------------------------------
+1
> Problem with DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
> --------------------------------------------
>
> Key: PIG-1812
> URL: https://issues.apache.org/jira/browse/PIG-1812
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0, 0.8.0
> Environment: RHEL, Pig 0.8.0
> Reporter: xianyu
> Assignee: Daniel Dai
> Attachments: PIG-1812-1.patch, PIG-1812-2.patch
>
>
> Hi,
> I have the following input files:
> pkg.txt
> a 3 {(123,1.0),(236,2.0)}
> a 3 {(236,1.0)}
> model.txt
> a 123 2 0.33
> a 236 2 0.5
> My script is listed below:
> A = load 'pkg.txt' using PigStorage('\t') as (pkg:chararray, ts:int, cat_bag:{t:(id:chararray, wht:float)});
> M = load 'model.txt' using PigStorage('\t') as (pkg:chararray, cat_id:chararray, ts:int, score:double);
> B = foreach A generate ts, pkg, flatten(cat_bag.id) as (cat_id:chararray);
> B = distinct B;
> H1 = cogroup M by (pkg, cat_id) inner, B by (pkg, cat_id);
> H2 = foreach H1 {
> I = order M by ts;
> J = order B by ts;
> generate flatten(group) as (pkg:chararray, cat_id:chararray), J.ts as tsorig, I.ts as tsmap;
> }
> dump H2;
> When running this script, I got a warning about "Encountered Warning DID_NOT_FIND_LOAD_ONLY_MAP_PLAN 1 time(s)" and pig error log as below:
> Pig Stack Trace
> ---------------
> ERROR 2043: Unexpected error during execution.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias H2
> at org.apache.pig.PigServer.openIterator(PigServer.java:764)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> at org.apache.pig.Main.run(Main.java:500)
> at org.apache.pig.Main.main(Main.java:107)
> Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias H2
> at org.apache.pig.PigServer.storeEx(PigServer.java:888)
> at org.apache.pig.PigServer.store(PigServer.java:826)
> at org.apache.pig.PigServer.openIterator(PigServer.java:738)
> ... 7 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:403)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1208)
> at org.apache.pig.PigServer.storeEx(PigServer.java:884)
> ... 9 more
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad cannot be cast to org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:246)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:41)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:498)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:117)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:378)
> ... 11 more
> But, when I removed the DISTINCT statement before COGROUP, i.e. "B = distinct B;" this script can run smoothly. I have also tried other reducer side operations like ORDER, it seems that they will also trigger above error. This is really very confusing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1812) Problem with
DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-1812:
----------------------------
Attachment: PIG-1812-1.patch
> Problem with DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
> --------------------------------------------
>
> Key: PIG-1812
> URL: https://issues.apache.org/jira/browse/PIG-1812
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0, 0.8.0
> Environment: RHEL, Pig 0.8.0
> Reporter: xianyu
> Assignee: Daniel Dai
> Attachments: PIG-1812-1.patch
>
>
> Hi,
> I have the following input files:
> pkg.txt
> a 3 {(123,1.0),(236,2.0)}
> a 3 {(236,1.0)}
> model.txt
> a 123 2 0.33
> a 236 2 0.5
> My script is listed below:
> A = load 'pkg.txt' using PigStorage('\t') as (pkg:chararray, ts:int, cat_bag:{t:(id:chararray, wht:float)});
> M = load 'model.txt' using PigStorage('\t') as (pkg:chararray, cat_id:chararray, ts:int, score:double);
> B = foreach A generate ts, pkg, flatten(cat_bag.id) as (cat_id:chararray);
> B = distinct B;
> H1 = cogroup M by (pkg, cat_id) inner, B by (pkg, cat_id);
> H2 = foreach H1 {
> I = order M by ts;
> J = order B by ts;
> generate flatten(group) as (pkg:chararray, cat_id:chararray), J.ts as tsorig, I.ts as tsmap;
> }
> dump H2;
> When running this script, I got a warning about "Encountered Warning DID_NOT_FIND_LOAD_ONLY_MAP_PLAN 1 time(s)" and pig error log as below:
> Pig Stack Trace
> ---------------
> ERROR 2043: Unexpected error during execution.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias H2
> at org.apache.pig.PigServer.openIterator(PigServer.java:764)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> at org.apache.pig.Main.run(Main.java:500)
> at org.apache.pig.Main.main(Main.java:107)
> Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias H2
> at org.apache.pig.PigServer.storeEx(PigServer.java:888)
> at org.apache.pig.PigServer.store(PigServer.java:826)
> at org.apache.pig.PigServer.openIterator(PigServer.java:738)
> ... 7 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:403)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1208)
> at org.apache.pig.PigServer.storeEx(PigServer.java:884)
> ... 9 more
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad cannot be cast to org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:246)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:41)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:498)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:117)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:378)
> ... 11 more
> But, when I removed the DISTINCT statement before COGROUP, i.e. "B = distinct B;" this script can run smoothly. I have also tried other reducer side operations like ORDER, it seems that they will also trigger above error. This is really very confusing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1812) Problem with
DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
Posted by "Thejas M Nair (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12988270#action_12988270 ]
Thejas M Nair commented on PIG-1812:
------------------------------------
+1
> Problem with DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
> --------------------------------------------
>
> Key: PIG-1812
> URL: https://issues.apache.org/jira/browse/PIG-1812
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0, 0.8.0
> Environment: RHEL, Pig 0.8.0
> Reporter: xianyu
> Assignee: Daniel Dai
> Attachments: PIG-1812-1.patch, PIG-1812-2.patch
>
>
> Hi,
> I have the following input files:
> pkg.txt
> a 3 {(123,1.0),(236,2.0)}
> a 3 {(236,1.0)}
> model.txt
> a 123 2 0.33
> a 236 2 0.5
> My script is listed below:
> A = load 'pkg.txt' using PigStorage('\t') as (pkg:chararray, ts:int, cat_bag:{t:(id:chararray, wht:float)});
> M = load 'model.txt' using PigStorage('\t') as (pkg:chararray, cat_id:chararray, ts:int, score:double);
> B = foreach A generate ts, pkg, flatten(cat_bag.id) as (cat_id:chararray);
> B = distinct B;
> H1 = cogroup M by (pkg, cat_id) inner, B by (pkg, cat_id);
> H2 = foreach H1 {
> I = order M by ts;
> J = order B by ts;
> generate flatten(group) as (pkg:chararray, cat_id:chararray), J.ts as tsorig, I.ts as tsmap;
> }
> dump H2;
> When running this script, I got a warning about "Encountered Warning DID_NOT_FIND_LOAD_ONLY_MAP_PLAN 1 time(s)" and pig error log as below:
> Pig Stack Trace
> ---------------
> ERROR 2043: Unexpected error during execution.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias H2
> at org.apache.pig.PigServer.openIterator(PigServer.java:764)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> at org.apache.pig.Main.run(Main.java:500)
> at org.apache.pig.Main.main(Main.java:107)
> Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias H2
> at org.apache.pig.PigServer.storeEx(PigServer.java:888)
> at org.apache.pig.PigServer.store(PigServer.java:826)
> at org.apache.pig.PigServer.openIterator(PigServer.java:738)
> ... 7 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:403)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1208)
> at org.apache.pig.PigServer.storeEx(PigServer.java:884)
> ... 9 more
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad cannot be cast to org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:246)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:41)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:498)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:117)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:378)
> ... 11 more
> But, when I removed the DISTINCT statement before COGROUP, i.e. "B = distinct B;" this script can run smoothly. I have also tried other reducer side operations like ORDER, it seems that they will also trigger above error. This is really very confusing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-1812) Problem with
DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai resolved PIG-1812.
-----------------------------
Resolution: Fixed
Fix Version/s: 0.8.0
Hadoop Flags: [Reviewed]
Patch committed to both trunk and 0.8 branch.
> Problem with DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
> --------------------------------------------
>
> Key: PIG-1812
> URL: https://issues.apache.org/jira/browse/PIG-1812
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0, 0.8.0
> Environment: RHEL, Pig 0.8.0
> Reporter: xianyu
> Assignee: Daniel Dai
> Fix For: 0.8.0
>
> Attachments: PIG-1812-1.patch, PIG-1812-2.patch
>
>
> Hi,
> I have the following input files:
> pkg.txt
> a 3 {(123,1.0),(236,2.0)}
> a 3 {(236,1.0)}
> model.txt
> a 123 2 0.33
> a 236 2 0.5
> My script is listed below:
> A = load 'pkg.txt' using PigStorage('\t') as (pkg:chararray, ts:int, cat_bag:{t:(id:chararray, wht:float)});
> M = load 'model.txt' using PigStorage('\t') as (pkg:chararray, cat_id:chararray, ts:int, score:double);
> B = foreach A generate ts, pkg, flatten(cat_bag.id) as (cat_id:chararray);
> B = distinct B;
> H1 = cogroup M by (pkg, cat_id) inner, B by (pkg, cat_id);
> H2 = foreach H1 {
> I = order M by ts;
> J = order B by ts;
> generate flatten(group) as (pkg:chararray, cat_id:chararray), J.ts as tsorig, I.ts as tsmap;
> }
> dump H2;
> When running this script, I got a warning about "Encountered Warning DID_NOT_FIND_LOAD_ONLY_MAP_PLAN 1 time(s)" and pig error log as below:
> Pig Stack Trace
> ---------------
> ERROR 2043: Unexpected error during execution.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias H2
> at org.apache.pig.PigServer.openIterator(PigServer.java:764)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> at org.apache.pig.Main.run(Main.java:500)
> at org.apache.pig.Main.main(Main.java:107)
> Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias H2
> at org.apache.pig.PigServer.storeEx(PigServer.java:888)
> at org.apache.pig.PigServer.store(PigServer.java:826)
> at org.apache.pig.PigServer.openIterator(PigServer.java:738)
> ... 7 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:403)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1208)
> at org.apache.pig.PigServer.storeEx(PigServer.java:884)
> ... 9 more
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad cannot be cast to org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:246)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:41)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:498)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:117)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:378)
> ... 11 more
> But, when I removed the DISTINCT statement before COGROUP, i.e. "B = distinct B;" this script can run smoothly. I have also tried other reducer side operations like ORDER, it seems that they will also trigger above error. This is really very confusing.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (PIG-1812) Problem with
DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai reassigned PIG-1812:
-------------------------------
Assignee: Daniel Dai
> Problem with DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
> --------------------------------------------
>
> Key: PIG-1812
> URL: https://issues.apache.org/jira/browse/PIG-1812
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0, 0.8.0
> Environment: RHEL, Pig 0.8.0
> Reporter: xianyu
> Assignee: Daniel Dai
>
> Hi,
> I have the following input files:
> pkg.txt
> a 3 {(123,1.0),(236,2.0)}
> a 3 {(236,1.0)}
> model.txt
> a 123 2 0.33
> a 236 2 0.5
> My script is listed below:
> A = load 'pkg.txt' using PigStorage('\t') as (pkg:chararray, ts:int, cat_bag:{t:(id:chararray, wht:float)});
> M = load 'model.txt' using PigStorage('\t') as (pkg:chararray, cat_id:chararray, ts:int, score:double);
> B = foreach A generate ts, pkg, flatten(cat_bag.id) as (cat_id:chararray);
> B = distinct B;
> H1 = cogroup M by (pkg, cat_id) inner, B by (pkg, cat_id);
> H2 = foreach H1 {
> I = order M by ts;
> J = order B by ts;
> generate flatten(group) as (pkg:chararray, cat_id:chararray), J.ts as tsorig, I.ts as tsmap;
> }
> dump H2;
> When running this script, I got a warning about "Encountered Warning DID_NOT_FIND_LOAD_ONLY_MAP_PLAN 1 time(s)" and pig error log as below:
> Pig Stack Trace
> ---------------
> ERROR 2043: Unexpected error during execution.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias H2
> at org.apache.pig.PigServer.openIterator(PigServer.java:764)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> at org.apache.pig.Main.run(Main.java:500)
> at org.apache.pig.Main.main(Main.java:107)
> Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias H2
> at org.apache.pig.PigServer.storeEx(PigServer.java:888)
> at org.apache.pig.PigServer.store(PigServer.java:826)
> at org.apache.pig.PigServer.openIterator(PigServer.java:738)
> ... 7 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:403)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1208)
> at org.apache.pig.PigServer.storeEx(PigServer.java:884)
> ... 9 more
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad cannot be cast to org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:246)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:41)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:498)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:117)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:378)
> ... 11 more
> But, when I removed the DISTINCT statement before COGROUP, i.e. "B = distinct B;" this script can run smoothly. I have also tried other reducer side operations like ORDER, it seems that they will also trigger above error. This is really very confusing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1812) Problem with
DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12987164#action_12987164 ]
Daniel Dai commented on PIG-1812:
---------------------------------
Review request:
https://reviews.apache.org/r/357/
> Problem with DID_NOT_FIND_LOAD_ONLY_MAP_PLAN
> --------------------------------------------
>
> Key: PIG-1812
> URL: https://issues.apache.org/jira/browse/PIG-1812
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.7.0, 0.8.0
> Environment: RHEL, Pig 0.8.0
> Reporter: xianyu
> Assignee: Daniel Dai
> Attachments: PIG-1812-1.patch
>
>
> Hi,
> I have the following input files:
> pkg.txt
> a 3 {(123,1.0),(236,2.0)}
> a 3 {(236,1.0)}
> model.txt
> a 123 2 0.33
> a 236 2 0.5
> My script is listed below:
> A = load 'pkg.txt' using PigStorage('\t') as (pkg:chararray, ts:int, cat_bag:{t:(id:chararray, wht:float)});
> M = load 'model.txt' using PigStorage('\t') as (pkg:chararray, cat_id:chararray, ts:int, score:double);
> B = foreach A generate ts, pkg, flatten(cat_bag.id) as (cat_id:chararray);
> B = distinct B;
> H1 = cogroup M by (pkg, cat_id) inner, B by (pkg, cat_id);
> H2 = foreach H1 {
> I = order M by ts;
> J = order B by ts;
> generate flatten(group) as (pkg:chararray, cat_id:chararray), J.ts as tsorig, I.ts as tsmap;
> }
> dump H2;
> When running this script, I got a warning about "Encountered Warning DID_NOT_FIND_LOAD_ONLY_MAP_PLAN 1 time(s)" and pig error log as below:
> Pig Stack Trace
> ---------------
> ERROR 2043: Unexpected error during execution.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias H2
> at org.apache.pig.PigServer.openIterator(PigServer.java:764)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> at org.apache.pig.Main.run(Main.java:500)
> at org.apache.pig.Main.main(Main.java:107)
> Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias H2
> at org.apache.pig.PigServer.storeEx(PigServer.java:888)
> at org.apache.pig.PigServer.store(PigServer.java:826)
> at org.apache.pig.PigServer.openIterator(PigServer.java:738)
> ... 7 more
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 2043: Unexpected error during execution.
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:403)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1208)
> at org.apache.pig.PigServer.storeEx(PigServer.java:884)
> ... 9 more
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad cannot be cast to org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SecondaryKeyOptimizer.visitMROp(SecondaryKeyOptimizer.java:352)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:246)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:41)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:498)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:117)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:378)
> ... 11 more
> But, when I removed the DISTINCT statement before COGROUP, i.e. "B = distinct B;" this script can run smoothly. I have also tried other reducer side operations like ORDER, it seems that they will also trigger above error. This is really very confusing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.