You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Haitao Yao <ya...@gmail.com> on 2012/08/10 05:20:23 UTC
ColumnMapKeyPrune bug?

hey, all
	I got this error on my pig script: 

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2000: Error processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPrune
        at org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:122)
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:287)
        at org.apache.pig.PigServer.compilePp(PigServer.java:1360)
        at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1297)
        at org.apache.pig.PigServer.execute(PigServer.java:1289)
        at org.apache.pig.PigServer.executeBatch(PigServer.java:360)
        at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:131)
        at org.apache.pig.tools.grunt.GruntParser.processFsCommand(GruntParser.java:972)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:117)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
        at he.pig.PigWorker.call(BatchPig.java:426)
        at he.pig.PigWorker.call(BatchPig.java:400)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.IndexOutOfBoundsException: Index: 6, Size: 6
        at java.util.ArrayList.rangeCheck(ArrayList.java:604)
        at java.util.ArrayList.get(ArrayList.java:382)
        at org.apache.pig.newplan.logical.relational.LogicalSchema.getField(LogicalSchema.java:651)
        at org.apache.pig.newplan.logical.expression.DereferenceExpression.getFieldSchema(DereferenceExpression.java:153)
        at org.apache.pig.newplan.logical.optimizer.FieldSchemaResetter.execute(SchemaResetter.java:192)
        at org.apache.pig.newplan.logical.expression.AllSameExpressionVisitor.visit(AllSameExpressionVisitor.java:148)
        at org.apache.pig.newplan.logical.expression.DereferenceExpression.accept(DereferenceExpression.java:84)
        at org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70)
        at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
        at org.apache.pig.newplan.logical.optimizer.SchemaResetter.visit(SchemaResetter.java:104)
        at org.apache.pig.newplan.logical.relational.LOGenerate.accept(LOGenerate.java:240)
        at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
        at org.apache.pig.newplan.logical.optimizer.SchemaResetter.visit(SchemaResetter.java:93)
        at org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:74)
        at org.apache.pig.newplan.DependencyOrderWalker.walk(DependencyOrderWalker.java:75)
        at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
        at org.apache.pig.newplan.logical.optimizer.SchemaPatcher.transformed(SchemaPatcher.java:43)
        at org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:113)
        ... 18 more


In order by continue running the script, I have to use -t ColumnMapKeyPrune to turn off the optimization. It seems like there's a simple bug java.lang.IndexOutOfBoundsException  here? 

Thanks


Haitao Yao
yao.erix@gmail.com
weibo: @haitao_yao
Skype:  haitao.yao.final