You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Haijun Cao (JIRA)" <ji...@apache.org> on 2008/06/08 23:21:44 UTC
[jira] Commented: (PIG-261) explain exception
[ https://issues.apache.org/jira/browse/PIG-261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12603438#action_12603438 ]
Haijun Cao commented on PIG-261:
--------------------------------
I should add that the script runs fine when I store d, it just throws exception during explain.
I would like to use explain to see if distinct is in combiner phase.
> explain exception
> -----------------
>
> Key: PIG-261
> URL: https://issues.apache.org/jira/browse/PIG-261
> Project: Pig
> Issue Type: Bug
> Components: grunt
> Environment: ubuntu 7.10
> jdk1.6.0_06
> ant 1.7.0
> hadoop-0.17.0
> Reporter: Haijun Cao
>
> grunt> a = load 'whparsed_20071231';
> grunt> b = filter a by $0 neq 'SE' AND ARITY (*) >= 6;
> grunt> c = group b by ($4, $5);
> grunt> d = foreach c {
> >> e = distinct b.$1;
> >> generate flatten (group), COUNT (b), COUNT (e);
> >> }
> grunt> explain d
> Logical Plan:
> |---LOEval ( GENERATE {[FLATTEN PROJECT $0],[COUNT(GENERATE {[PROJECT $1]})],[COUNT(GENERATE {[PROJECT $1]->[PROJECT $1]->[DISTINCT ]})]} )
> |---LOCogroup ( GENERATE {[PROJECT $4],[PROJECT $5],[*]} )
> |---LOEval ( [FILTER BY (([PROJECT $0] neq ['SE']) AND ([ARITY(GENERATE {[*]})] >= ['6']))] )
> |---LOLoad ( file = whparsed_20071231 )
> -----------------------------------------------
> Physical Plan:
> |---POMapreduce
> Map : Composite(*,Filter: AND )
> Reduce : Generate(Project(0),FuncEval(COUNT(Generate(Project(1)))),FuncEval(COUNT(Generate(Composite(Project(1),Project(1),Sort(Distinct(148045 [main] ERROR org.apache.pig.tools.grunt.GruntParser - java.lang.NullPointerException
> at org.apache.pig.impl.eval.EvalSpecTreePrinter.visitSortDistinct(EvalSpecTreePrinter.java:42)
> at org.apache.pig.impl.eval.SortDistinctSpec.visit(SortDistinctSpec.java:130)
> at org.apache.pig.impl.eval.EvalSpecTreePrinter.visitCompositeEval(EvalSpecTreePrinter.java:111)
> at org.apache.pig.impl.eval.CompositeEvalSpec.visit(CompositeEvalSpec.java:116)
> at org.apache.pig.impl.eval.EvalSpecTreePrinter.visitGenerate(EvalSpecTreePrinter.java:61)
> at org.apache.pig.impl.eval.GenerateSpec.visit(GenerateSpec.java:366)
> at org.apache.pig.impl.eval.EvalSpecTreePrinter.visitFuncEval(EvalSpecTreePrinter.java:95)
> at org.apache.pig.impl.eval.FuncEvalSpec.visit(FuncEvalSpec.java:249)
> at org.apache.pig.impl.eval.EvalSpecTreePrinter.visitGenerate(EvalSpecTreePrinter.java:61)
> at org.apache.pig.impl.eval.GenerateSpec.visit(GenerateSpec.java:366)
> at org.apache.pig.impl.physicalLayer.POTreePrinter.visitMapreduce(POTreePrinter.java:79)
> at org.apache.pig.backend.hadoop.executionengine.POMapreduce.visit(POMapreduce.java:281)
> at org.apache.pig.backend.hadoop.executionengine.MapRedPhysicalPlan.explain(MapRedPhysicalPlan.java:41)
> at org.apache.pig.PigServer.explain(PigServer.java:442)
> at org.apache.pig.tools.grunt.GruntParser.processExplain(GruntParser.java:136)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:177)
> at org.apache.pig.tools.grunt.GruntParser.parseContOnError(GruntParser.java:73)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:54)
> at org.apache.pig.Main.main(Main.java:270)
> 148046 [main] ERROR org.apache.pig.tools.grunt.GruntParser - java.lang.NullPointerException
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.