You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Koji Noguchi (JIRA)" <ji...@apache.org> on 2012/10/12 21:03:03 UTC
[jira] [Commented] (PIG-2968) ColumnMapKeyPrune fails to prune a
subtree inside foreach
[ https://issues.apache.org/jira/browse/PIG-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475247#comment-13475247 ]
Koji Noguchi commented on PIG-2968:
-----------------------------------
Log showing
{noformat}
$ cat /Users/knoguchi/git/pig/pig_1350068049281.logPig Stack Trace
---------------
ERROR 2000: Error processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPrune
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1067: Unable to explain alias another at org.apache.pig.PigServer.explain(PigServer.java:999)
at org.apache.pig.tools.grunt.GruntParser.explainCurrentBatch(GruntParser.java:398)
at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:330)
at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:293)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.Explain(PigScriptParser.java:715) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:342)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:193) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:169)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84) at org.apache.pig.Main.run(Main.java:604)
at org.apache.pig.Main.main(Main.java:154)
Caused by: org.apache.pig.impl.logicalLayer.FrontendException: ERROR 2000: Error processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPrune at org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:122)
at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:277)
at org.apache.pig.PigServer.compilePp(PigServer.java:1322) at org.apache.pig.PigServer.explain(PigServer.java:984)
... 10 more
Caused by: java.util.ConcurrentModificationException
at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
at java.util.AbstractList$Itr.next(AbstractList.java:343)
at org.apache.pig.newplan.logical.rules.ColumnPruneVisitor.removeSubTree(ColumnPruneVisitor.java:451)
at org.apache.pig.newplan.logical.rules.ColumnPruneVisitor.removeSubTree(ColumnPruneVisitor.java:452)
at org.apache.pig.newplan.logical.rules.ColumnPruneVisitor.visit(ColumnPruneVisitor.java:431)
at org.apache.pig.newplan.logical.relational.LOForEach.accept(LOForEach.java:76)
at org.apache.pig.newplan.ReverseDependencyOrderWalker.walk(ReverseDependencyOrderWalker.java:70)
at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
at org.apache.pig.newplan.logical.rules.ColumnMapKeyPrune$ColumnMapKeyPruneTransformer.transform(ColumnMapKeyPrune.java:141)
at org.apache.pig.newplan.optimizer.PlanOptimizer.optimize(PlanOptimizer.java:110)
... 13 more
================================================================================
{noformat}
> ColumnMapKeyPrune fails to prune a subtree inside foreach
> ---------------------------------------------------------
>
> Key: PIG-2968
> URL: https://issues.apache.org/jira/browse/PIG-2968
> Project: Pig
> Issue Type: Bug
> Reporter: Koji Noguchi
> Assignee: Koji Noguchi
> Priority: Minor
>
> Sample code
> {noformat}
> $ cat test/foreach.pig
> daily = load 'nyse' as (exchange, symbol);
> grpd = group daily by exchange;
> uniquecnt = foreach grpd {
> sym = daily.symbol;
> uniq_sym = distinct sym;
> generate group, uniq_sym;
> };
> another = FOREACH uniquecnt GENERATE group;
> explain another;
> {noformat}
> This breaks when it tries to prune uniq_sym->sym->innerload_daily
> bq. 2012-10-12 14:54:11,031 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2000: Error processing rule ColumnMapKeyPrune. Try -t ColumnMapKeyPrune
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira