You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Ming-Hay Luk <ma...@gmail.com> on 2010/03/17 09:57:38 UTC
NULL pointer when using JOIN and ILLUSTRATE
I'm using the 0.6.0 build and I was trying out the ILLUSTRATE command the
example given in the Pig Latin Reference Manual 2 for JOIN (inner).
I have the following pig script:
A = LOAD 'data1' AS (a1:int,a2:int,a3:int);
B = LOAD 'data2' AS (b1:int,b2:int);
X = JOIN A BY a1, B BY b1;
ILLUSTRATE X;
Which produces one of two null pointer exceptions:
1)
Pig Stack Trace
---------------
ERROR 2999: Unexpected internal error. null
java.lang.NullPointerException
at
org.apache.pig.pen.LineageTrimmingVisitor.PruneBaseDataConstrainedCoverage(LineageTrimmingVisitor.java:297)
at
org.apache.pig.pen.LineageTrimmingVisitor.checkCompleteness(LineageTrimmingVisitor.java:466)
at
org.apache.pig.pen.LineageTrimmingVisitor.processOperator(LineageTrimmingVisitor.java:424)
at
org.apache.pig.pen.LineageTrimmingVisitor.visit(LineageTrimmingVisitor.java:255)
at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:130)
at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:45)
at
org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:70)
at
org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:72)
at
org.apache.pig.pen.util.PreOrderDepthFirstWalker.walk(PreOrderDepthFirstWalker.java:55)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at
org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:108)
at org.apache.pig.PigServer.getExamples(PigServer.java:731)
at
org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:549)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:246)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
at org.apache.pig.Main.main(Main.java:386)
================================================================================
2)
java.lang.NullPointerException
at
org.apache.pig.pen.util.DisplayExamples.MakeArray(DisplayExamples.java:183)
at
org.apache.pig.pen.util.DisplayExamples.printTabular(DisplayExamples.java:86)
at
org.apache.pig.pen.util.DisplayExamples.printTabular(DisplayExamples.java:69)
at
org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:144)
at org.apache.pig.PigServer.getExamples(PigServer.java:731)
at
org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:549)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:246)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
at org.apache.pig.Main.main(Main.java:386)
Any insight is greatly appreciated.
- Ming-Hay Luk
Re: NULL pointer when using JOIN and ILLUSTRATE
Posted by Rekha Joshi <re...@yahoo-inc.com>.
If you get the same after filtering possible null values, you might hitting the known issues with illustrate - PIG-1066, PIG-534 to name a few.
I think the fix would be coming in later versions, the workaround is to do without illustrate after join..describe/explain work fine.
Cheers,
/R
On 3/17/10 2:27 PM, "Ming-Hay Luk" <ma...@gmail.com> wrote:
I'm using the 0.6.0 build and I was trying out the ILLUSTRATE command the
example given in the Pig Latin Reference Manual 2 for JOIN (inner).
I have the following pig script:
A = LOAD 'data1' AS (a1:int,a2:int,a3:int);
B = LOAD 'data2' AS (b1:int,b2:int);
X = JOIN A BY a1, B BY b1;
ILLUSTRATE X;
Which produces one of two null pointer exceptions:
1)
Pig Stack Trace
---------------
ERROR 2999: Unexpected internal error. null
java.lang.NullPointerException
at
org.apache.pig.pen.LineageTrimmingVisitor.PruneBaseDataConstrainedCoverage(LineageTrimmingVisitor.java:297)
at
org.apache.pig.pen.LineageTrimmingVisitor.checkCompleteness(LineageTrimmingVisitor.java:466)
at
org.apache.pig.pen.LineageTrimmingVisitor.processOperator(LineageTrimmingVisitor.java:424)
at
org.apache.pig.pen.LineageTrimmingVisitor.visit(LineageTrimmingVisitor.java:255)
at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:130)
at org.apache.pig.impl.logicalLayer.LOForEach.visit(LOForEach.java:45)
at
org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:70)
at
org.apache.pig.pen.util.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:72)
at
org.apache.pig.pen.util.PreOrderDepthFirstWalker.walk(PreOrderDepthFirstWalker.java:55)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at
org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:108)
at org.apache.pig.PigServer.getExamples(PigServer.java:731)
at
org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:549)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:246)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
at org.apache.pig.Main.main(Main.java:386)
================================================================================
2)
java.lang.NullPointerException
at
org.apache.pig.pen.util.DisplayExamples.MakeArray(DisplayExamples.java:183)
at
org.apache.pig.pen.util.DisplayExamples.printTabular(DisplayExamples.java:86)
at
org.apache.pig.pen.util.DisplayExamples.printTabular(DisplayExamples.java:69)
at
org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:144)
at org.apache.pig.PigServer.getExamples(PigServer.java:731)
at
org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:549)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:246)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:144)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
at org.apache.pig.Main.main(Main.java:386)
Any insight is greatly appreciated.
- Ming-Hay Luk