You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Olga Natkovich (JIRA)" <ji...@apache.org> on 2008/08/12 20:39:44 UTC
[jira] Created: (PIG-375) Exception with implicit split
Exception with implicit split
-----------------------------
Key: PIG-375
URL: https://issues.apache.org/jira/browse/PIG-375
Project: Pig
Issue Type: Bug
Reporter: Olga Natkovich
Priority: Critical
Script:
grunt> A = load 'foo';
grunt> B = foreach A generate $0;
grunt> C = foreach A generate $1;
grunt> D = Union B, C;
grunt> dump D;
Error:
08/08/12 11:36:57 ERROR plan.OperatorPlan: Attempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad multiple outputs. This operator does not support multiple outputs.
08/08/12 11:36:57 ERROR physicalLayer.LogToPhyTranslationVisitor: Invalid physical operators in the physical planAttempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad multiple outputs. This operator does not support multiple outputs.
08/08/12 11:36:59 ERROR grunt.GruntParser: java.io.IOException: Unable to open iterator for alias: E [null]
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:268)
at org.apache.pig.PigServer.execute(PigServer.java:519)
at org.apache.pig.PigServer.openIterator(PigServer.java:307)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:258)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:175)
at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:92)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
at org.apache.pig.Main.main(Main.java:278)
Caused by: org.apache.pig.backend.executionengine.ExecException
... 8 more
Caused by: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:162)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:107)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:68)
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:258)
... 7 more
Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLimit
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJobConf(JobControlCompiler.java:203)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:156)
... 10 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-375) Exception with implicit split
Posted by "Shravan Matthur Narayanamurthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shravan Matthur Narayanamurthy updated PIG-375:
-----------------------------------------------
Status: Patch Available (was: Open)
Wrote a walker(SplitIntroducer) which identifies implicit split condition and introduces a split into the plan. This is done while compilation of logicalplan in PigServer.compileLp before type checking. The split has true conditions in the form of constant expressions with true as the value.
Changes:
Added SplitIntroducer. Modified PigServer's compileLp to incorporate the call to splitintroducer. Added a test case to check the same.
Also included in this is the patch for pig-369. Details of the included patch are in the comments of Pig-369.
I have caneled the patch submitted for pig-369. Please verify and close pig-369 along with this.
> Exception with implicit split
> -----------------------------
>
> Key: PIG-375
> URL: https://issues.apache.org/jira/browse/PIG-375
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Olga Natkovich
> Assignee: Shravan Matthur Narayanamurthy
> Priority: Critical
> Fix For: types_branch
>
>
> Script:
> grunt> A = load 'foo';
> grunt> B = foreach A generate $0;
> grunt> C = foreach A generate $1;
> grunt> D = Union B, C;
> grunt> dump D;
> Error:
> 08/08/12 11:36:57 ERROR plan.OperatorPlan: Attempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad multiple outputs. This operator does not support multiple outputs.
> 08/08/12 11:36:57 ERROR physicalLayer.LogToPhyTranslationVisitor: Invalid physical operators in the physical planAttempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad multiple outputs. This operator does not support multiple outputs.
> 08/08/12 11:36:59 ERROR grunt.GruntParser: java.io.IOException: Unable to open iterator for alias: E [null]
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:268)
> at org.apache.pig.PigServer.execute(PigServer.java:519)
> at org.apache.pig.PigServer.openIterator(PigServer.java:307)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:258)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:175)
> at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:92)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:278)
> Caused by: org.apache.pig.backend.executionengine.ExecException
> ... 8 more
> Caused by: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:162)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:107)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:68)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:258)
> ... 7 more
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLimit
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJobConf(JobControlCompiler.java:203)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:156)
> ... 10 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-375) Exception with implicit split
Posted by "Shravan Matthur Narayanamurthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shravan Matthur Narayanamurthy updated PIG-375:
-----------------------------------------------
Attachment: 375.patch
> Exception with implicit split
> -----------------------------
>
> Key: PIG-375
> URL: https://issues.apache.org/jira/browse/PIG-375
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Olga Natkovich
> Assignee: Shravan Matthur Narayanamurthy
> Priority: Critical
> Fix For: types_branch
>
> Attachments: 375.patch
>
>
> Script:
> grunt> A = load 'foo';
> grunt> B = foreach A generate $0;
> grunt> C = foreach A generate $1;
> grunt> D = Union B, C;
> grunt> dump D;
> Error:
> 08/08/12 11:36:57 ERROR plan.OperatorPlan: Attempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad multiple outputs. This operator does not support multiple outputs.
> 08/08/12 11:36:57 ERROR physicalLayer.LogToPhyTranslationVisitor: Invalid physical operators in the physical planAttempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad multiple outputs. This operator does not support multiple outputs.
> 08/08/12 11:36:59 ERROR grunt.GruntParser: java.io.IOException: Unable to open iterator for alias: E [null]
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:268)
> at org.apache.pig.PigServer.execute(PigServer.java:519)
> at org.apache.pig.PigServer.openIterator(PigServer.java:307)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:258)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:175)
> at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:92)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:278)
> Caused by: org.apache.pig.backend.executionengine.ExecException
> ... 8 more
> Caused by: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:162)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:107)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:68)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:258)
> ... 7 more
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLimit
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJobConf(JobControlCompiler.java:203)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:156)
> ... 10 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-375) Exception with implicit split
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich reassigned PIG-375:
----------------------------------
Assignee: Shravan Matthur Narayanamurthy
Shravan, could you take a look, thanks
> Exception with implicit split
> -----------------------------
>
> Key: PIG-375
> URL: https://issues.apache.org/jira/browse/PIG-375
> Project: Pig
> Issue Type: Bug
> Reporter: Olga Natkovich
> Assignee: Shravan Matthur Narayanamurthy
> Priority: Critical
>
> Script:
> grunt> A = load 'foo';
> grunt> B = foreach A generate $0;
> grunt> C = foreach A generate $1;
> grunt> D = Union B, C;
> grunt> dump D;
> Error:
> 08/08/12 11:36:57 ERROR plan.OperatorPlan: Attempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad multiple outputs. This operator does not support multiple outputs.
> 08/08/12 11:36:57 ERROR physicalLayer.LogToPhyTranslationVisitor: Invalid physical operators in the physical planAttempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad multiple outputs. This operator does not support multiple outputs.
> 08/08/12 11:36:59 ERROR grunt.GruntParser: java.io.IOException: Unable to open iterator for alias: E [null]
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:268)
> at org.apache.pig.PigServer.execute(PigServer.java:519)
> at org.apache.pig.PigServer.openIterator(PigServer.java:307)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:258)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:175)
> at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:92)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:278)
> Caused by: org.apache.pig.backend.executionengine.ExecException
> ... 8 more
> Caused by: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:162)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:107)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:68)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:258)
> ... 7 more
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLimit
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJobConf(JobControlCompiler.java:203)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:156)
> ... 10 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-375) Exception with implicit split
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-375:
-------------------------------
Fix Version/s: types_branch
Affects Version/s: types_branch
> Exception with implicit split
> -----------------------------
>
> Key: PIG-375
> URL: https://issues.apache.org/jira/browse/PIG-375
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Olga Natkovich
> Assignee: Shravan Matthur Narayanamurthy
> Priority: Critical
> Fix For: types_branch
>
>
> Script:
> grunt> A = load 'foo';
> grunt> B = foreach A generate $0;
> grunt> C = foreach A generate $1;
> grunt> D = Union B, C;
> grunt> dump D;
> Error:
> 08/08/12 11:36:57 ERROR plan.OperatorPlan: Attempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad multiple outputs. This operator does not support multiple outputs.
> 08/08/12 11:36:57 ERROR physicalLayer.LogToPhyTranslationVisitor: Invalid physical operators in the physical planAttempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad multiple outputs. This operator does not support multiple outputs.
> 08/08/12 11:36:59 ERROR grunt.GruntParser: java.io.IOException: Unable to open iterator for alias: E [null]
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:268)
> at org.apache.pig.PigServer.execute(PigServer.java:519)
> at org.apache.pig.PigServer.openIterator(PigServer.java:307)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:258)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:175)
> at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:92)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:278)
> Caused by: org.apache.pig.backend.executionengine.ExecException
> ... 8 more
> Caused by: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:162)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:107)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:68)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:258)
> ... 7 more
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLimit
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJobConf(JobControlCompiler.java:203)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:156)
> ... 10 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-375) Exception with implicit split
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-375:
-------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
patch committed. thanks shravan for such a quick turnaround!
> Exception with implicit split
> -----------------------------
>
> Key: PIG-375
> URL: https://issues.apache.org/jira/browse/PIG-375
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Olga Natkovich
> Assignee: Shravan Matthur Narayanamurthy
> Priority: Critical
> Fix For: types_branch
>
> Attachments: 375.patch
>
>
> Script:
> grunt> A = load 'foo';
> grunt> B = foreach A generate $0;
> grunt> C = foreach A generate $1;
> grunt> D = Union B, C;
> grunt> dump D;
> Error:
> 08/08/12 11:36:57 ERROR plan.OperatorPlan: Attempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad multiple outputs. This operator does not support multiple outputs.
> 08/08/12 11:36:57 ERROR physicalLayer.LogToPhyTranslationVisitor: Invalid physical operators in the physical planAttempt to give operator of type org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLoad multiple outputs. This operator does not support multiple outputs.
> 08/08/12 11:36:59 ERROR grunt.GruntParser: java.io.IOException: Unable to open iterator for alias: E [null]
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:268)
> at org.apache.pig.PigServer.execute(PigServer.java:519)
> at org.apache.pig.PigServer.openIterator(PigServer.java:307)
> at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:258)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:175)
> at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:92)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:58)
> at org.apache.pig.Main.main(Main.java:278)
> Caused by: org.apache.pig.backend.executionengine.ExecException
> ... 8 more
> Caused by: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:162)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:107)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:68)
> at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:258)
> ... 7 more
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLimit
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJobConf(JobControlCompiler.java:203)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:156)
> ... 10 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.