You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Thejas M Nair (JIRA)" <ji...@apache.org> on 2011/07/14 23:01:00 UTC
[jira] [Created] (PIG-2164) CombinerOptimizer exception with
multi-query
CombinerOptimizer exception with multi-query
--------------------------------------------
Key: PIG-2164
URL: https://issues.apache.org/jira/browse/PIG-2164
Project: Pig
Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Fix For: 0.9.0, 0.10
{code}
l = load 'x' as (a,b,c);
f1 = foreach (group l by (a,b)) generate COUNT_STAR(*);
f2 = foreach (group l by (a,null)) generate COUNT_STAR(*);
u = union f1, f2;
explain u;
Pig Stack Trace
---------------
ERROR 2018: Internal error. Unable to introduce the combiner for optimization.
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1067: Unable to explain alias u
at org.apache.pig.PigServer.explain(PigServer.java:1063)
at org.apache.pig.tools.grunt.GruntParser.explainCurrentBatch(GruntParser.java:393)
at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:325)
at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:288)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.Explain(PigScriptParser.java:665)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:325)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:67)
at org.apache.pig.Main.run(Main.java:487)
at org.apache.pig.Main.main(Main.java:108)
Caused by: org.apache.pig.impl.plan.optimizer.OptimizerException: ERROR 2018: Internal error. Unable to introduce the combiner for optimization.
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer.visitMROp(CombinerOptimizer.java:313)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:252)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:42)
at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:492)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.explain(MapReduceLauncher.java:454)
at org.apache.pig.PigServer.explain(PigServer.java:1055)
... 10 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 2
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer.addAlgebraicFuncToCombineFE(CombinerOptimizer.java:515)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer.visitMROp(CombinerOptimizer.java:227)
... 19 more
================================================================================
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (PIG-2164) CombinerOptimizer exception with
multi-query
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai resolved PIG-2164.
-----------------------------
Resolution: Fixed
It is fixed by PIG-2286. Apply PIG-2286-2.patch and the error go away.
> CombinerOptimizer exception with multi-query
> --------------------------------------------
>
> Key: PIG-2164
> URL: https://issues.apache.org/jira/browse/PIG-2164
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.9.1, 0.10
>
>
> {code}
> l = load 'x' as (a,b,c);
> f1 = foreach (group l by (a,b)) generate COUNT_STAR(*);
> f2 = foreach (group l by (a,null)) generate COUNT_STAR(*);
> u = union f1, f2;
> explain u;
> Pig Stack Trace
> ---------------
> ERROR 2018: Internal error. Unable to introduce the combiner for optimization.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1067: Unable to explain alias u
> at org.apache.pig.PigServer.explain(PigServer.java:1063)
> at org.apache.pig.tools.grunt.GruntParser.explainCurrentBatch(GruntParser.java:393)
> at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:325)
> at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:288)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.Explain(PigScriptParser.java:665)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:325)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:67)
> at org.apache.pig.Main.run(Main.java:487)
> at org.apache.pig.Main.main(Main.java:108)
> Caused by: org.apache.pig.impl.plan.optimizer.OptimizerException: ERROR 2018: Internal error. Unable to introduce the combiner for optimization.
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer.visitMROp(CombinerOptimizer.java:313)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:252)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:42)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:492)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.explain(MapReduceLauncher.java:454)
> at org.apache.pig.PigServer.explain(PigServer.java:1055)
> ... 10 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 2
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer.addAlgebraicFuncToCombineFE(CombinerOptimizer.java:515)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer.visitMROp(CombinerOptimizer.java:227)
> ... 19 more
> ================================================================================
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2164) CombinerOptimizer exception with
multi-query
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-2164:
--------------------------------
Fix Version/s: (was: 0.9.0)
0.9.1
> CombinerOptimizer exception with multi-query
> --------------------------------------------
>
> Key: PIG-2164
> URL: https://issues.apache.org/jira/browse/PIG-2164
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Thejas M Nair
> Assignee: Thejas M Nair
> Fix For: 0.9.1, 0.10
>
>
> {code}
> l = load 'x' as (a,b,c);
> f1 = foreach (group l by (a,b)) generate COUNT_STAR(*);
> f2 = foreach (group l by (a,null)) generate COUNT_STAR(*);
> u = union f1, f2;
> explain u;
> Pig Stack Trace
> ---------------
> ERROR 2018: Internal error. Unable to introduce the combiner for optimization.
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1067: Unable to explain alias u
> at org.apache.pig.PigServer.explain(PigServer.java:1063)
> at org.apache.pig.tools.grunt.GruntParser.explainCurrentBatch(GruntParser.java:393)
> at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:325)
> at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:288)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.Explain(PigScriptParser.java:665)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:325)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:67)
> at org.apache.pig.Main.run(Main.java:487)
> at org.apache.pig.Main.main(Main.java:108)
> Caused by: org.apache.pig.impl.plan.optimizer.OptimizerException: ERROR 2018: Internal error. Unable to introduce the combiner for optimization.
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer.visitMROp(CombinerOptimizer.java:313)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:252)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceOper.visit(MapReduceOper.java:42)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:69)
> at org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:71)
> at org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:52)
> at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(MapReduceLauncher.java:492)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.explain(MapReduceLauncher.java:454)
> at org.apache.pig.PigServer.explain(PigServer.java:1055)
> ... 10 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 2
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer.addAlgebraicFuncToCombineFE(CombinerOptimizer.java:515)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.CombinerOptimizer.visitMROp(CombinerOptimizer.java:227)
> ... 19 more
> ================================================================================
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira