You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by James Kebinger <jk...@gmail.com> on 2009/11/15 20:38:24 UTC
NullPointerException calling illustrate
Hello all,
I'm encountering an error when I call illustrate - it blows up with a null
pointer and the following exception:
java.lang.NullPointerException
at
org.apache.pig.pen.util.DisplayExamples.ShortenField(DisplayExamples.java:205)
at
org.apache.pig.pen.util.DisplayExamples.MakeArray(DisplayExamples.java:190)
at
org.apache.pig.pen.util.DisplayExamples.PrintTabular(DisplayExamples.java:86)
at
org.apache.pig.pen.util.DisplayExamples.PrintTabular(DisplayExamples.java:69)
at
org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:144)
at org.apache.pig.PigServer.getExamples(PigServer.java:678)
at
org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:369)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:198)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:99)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.main(Main.java:321)
This happens with even a 2 step load then illustrate sequence like this:
grunt> data = load '/Users/jkk/projects/weblogdumps/weblogs_2005_11' using
PigStorage('\t') AS (session_id, user_id, page_name, date, params,
total_time, render_time, db_time, ip, cookies, user_agent, referrer, ajax,
http_verb);
grunt> illustrate data;
This trace is from an older pig, but I get a very similar stack dump from
pig 0.5 where the error also occurs.
Is anyone else seeing this? I didn't get a lot of hits in google.
thanks!
james
Problems with Illustrate
Posted by Sam Rash <sa...@ning.com>.
Hi,
A while back I emailed about some issues with ILLUSTRATE and a custom
LoadFunc impl. I've since fixed those and am now getting a new
error. Does this look familiar to anyone?
2009-11-16 10:03:25,570 [main] INFO org.apache.pig.Main - Logging
error messages to: /Users/samr/pig/pig_1258394605570.log
2009-11-16 10:03:25,595 [main] INFO
org.apache.pig.tools.parameters.PreprocessorContext - Executing
command : date +'%Y-%m-%dT%H.%M.%S'
2009-11-16 10:03:25,627 [main] INFO
org.apache.pig.tools.parameters.PreprocessorContext - Executing
command : whoami
2009-11-16 10:03:25,838 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to hadoop file system at: ...
2009-11-16 10:03:26,158 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to map-reduce job tracker at: ...
2009-11-16 10:03:32,375 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to hadoop file system at: ...
2009-11-16 10:03:32,392 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to map-reduce job tracker at: jobtracker.ninginc.com:9001
2009-11-16 10:03:32,846 [main] INFO ning.dsp.pig.ThriftStorage -
binding to: file1 0 9223372036854775807
2009-11-16 10:03:32,934 [main] WARN
org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
2009-11-16 10:03:32,935 [main] INFO
org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor
2009-11-16 10:03:32,936 [main] INFO
org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor
2009-11-16 10:03:32,936 [main] INFO
org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor
2009-11-16 10:03:32,936 [main] INFO
org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor
2009-11-16 10:04:26,488 [main] INFO ning.dsp.pig.ThriftStorage -
binding to: file2 0 9223372036854775807
2009-11-16 10:04:28,533 [main] INFO
org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor
2009-11-16 10:04:28,533 [main] INFO
org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor
2009-11-16 10:04:28,533 [main] INFO
org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor
2009-11-16 10:04:28,534 [main] INFO
org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor
2009-11-16 10:05:05,524 [main] INFO ning.dsp.pig.ThriftStorage -
binding to: file3 0 9223372036854775807
2009-11-16 10:05:05,704 [main] INFO
org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor
2009-11-16 10:05:05,704 [main] INFO
org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor
2009-11-16 10:05:05,704 [main] INFO
org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor
2009-11-16 10:05:05,705 [main] INFO
org.apache.hadoop.io.compress.CodecPool - Got brand-new decompressor
2009-11-16 10:05:11,624 [main] ERROR org.apache.pig.tools.grunt.Grunt
- ERROR 2999: Unexpected internal error. null
Details at logfile: /Users/samr/pig/pig_1258394605570.log
Pig Stack Trace
---------------
ERROR 2999: Unexpected internal error. null
java.lang.NullPointerException
at
org
.apache
.pig
.pen
.LineageTrimmingVisitor
.PruneBaseDataConstrainedCoverage(LineageTrimmingVisitor.java:297)
at
org
.apache
.pig
.pen
.LineageTrimmingVisitor.checkCompleteness(LineageTrimmingVisitor.java:
467)
at
org
.apache
.pig
.pen
.LineageTrimmingVisitor.processOperator(LineageTrimmingVisitor.java:425)
at
org
.apache
.pig.pen.LineageTrimmingVisitor.visit(LineageTrimmingVisitor.java:249)
at org.apache.pig.impl.logicalLayer.LOFilter.visit(LOFilter.java:110)
at org.apache.pig.impl.logicalLayer.LOFilter.visit(LOFilter.java:41)
at
org
.apache
.pig
.pen
.util
.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:69)
at
org
.apache
.pig
.pen
.util
.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:71)
at
org
.apache
.pig
.pen
.util
.PreOrderDepthFirstWalker.depthFirst(PreOrderDepthFirstWalker.java:71)
at
org
.apache
.pig
.pen.util.PreOrderDepthFirstWalker.walk(PreOrderDepthFirstWalker.java:
55)
at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
at
org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:
108)
at org.apache.pig.PigServer.getExamples(PigServer.java:723)
at
org
.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:
545)
at
org
.apache
.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:
246)
at
org
.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:
168)
at
org
.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:
144)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:89)
at org.apache.pig.Main.main(Main.java:320)
Sam Rash
samr@ning.com