You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Alex McLintock <al...@gmail.com> on 2011/02/07 20:39:22 UTC
Weird stack trace NullableBytesWritable vs NullableText
Can anyone give me any hints on why a JOIN may be failing with this weird
error....
I can DESCRIBE the two tables it is joining
justurls: {tweetid: bytearray,userid: bytearray,url: bytearray}
userdb: {ouruserid: bytearray,friendid: bytearray}
and the join itself is
urlspertimeline = JOIN userdb BY friendid,
justurls BY userid;
ERROR 1066: Unable to open iterator for alias urlspertimeline. Backend error
: Type mismatch in key from map: expected
org.apache.pig.impl.io.NullableBytesWritable, recieved
org.apache.pig.impl.io.NullableText
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
open iterator for alias urlspertimeline. Backend error : Type mismatch in
key from map: expected org.apache.pig.impl.io.NullableBytesWritable,
recieved org.apache.pig.impl.io.NullableText
at org.apache.pig.PigServer.openIterator(PigServer.java:742)
at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
at org.apache.pig.Main.run(Main.java:510)
at org.apache.pig.Main.main(Main.java:107)
Caused by: java.io.IOException: Type mismatch in key from map: expected
org.apache.pig.impl.io.NullableBytesWritable, recieved
org.apache.pig.impl.io.NullableText
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:845)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:541)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:116)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:238)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
Re: Weird stack trace NullableBytesWritable vs NullableText
Posted by Daniel Dai <ji...@yahoo-inc.com>.
In Pig 0.9, we will detect group/join key type dynamically (PIG-1277),
and will provide typed map. This will solve the map value type problem.
Daniel
Alex McLintock wrote:
> I am using maps a lot so I guess this is related to PIG-919 which is closed
> but not really fixed.
>
> https://issues.apache.org/jira/browse/PIG-919
>
> This suggested that I force the relevant types to (chararray) and that seems
> to have worked as a work around.
>
> Alex
>
> On 7 February 2011 19:39, Alex McLintock <al...@gmail.com> wrote:
>
>
>> Can anyone give me any hints on why a JOIN may be failing with this weird
>> error....
>>
>>
>>
>> I can DESCRIBE the two tables it is joining
>>
>> justurls: {tweetid: bytearray,userid: bytearray,url: bytearray}
>> userdb: {ouruserid: bytearray,friendid: bytearray}
>>
>> and the join itself is
>>
>>
>> urlspertimeline = JOIN userdb BY friendid,
>> justurls BY userid;
>>
>>
>> ERROR 1066: Unable to open iterator for alias urlspertimeline. Backend
>> error : Type mismatch in key from map: expected
>> org.apache.pig.impl.io.NullableBytesWritable, recieved
>> org.apache.pig.impl.io.NullableText
>>
>> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
>> open iterator for alias urlspertimeline. Backend error : Type mismatch in
>> key from map: expected org.apache.pig.impl.io.NullableBytesWritable,
>> recieved org.apache.pig.impl.io.NullableText
>> at org.apache.pig.PigServer.openIterator(PigServer.java:742)
>> at
>> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
>> at
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
>> at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>> at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
>> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
>> at org.apache.pig.Main.run(Main.java:510)
>> at org.apache.pig.Main.main(Main.java:107)
>> Caused by: java.io.IOException: Type mismatch in key from map: expected
>> org.apache.pig.impl.io.NullableBytesWritable, recieved
>> org.apache.pig.impl.io.NullableText
>> at
>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:845)
>> at
>> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:541)
>> at
>> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:116)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:238)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
>> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>>
>>
>>
>>
Re: Weird stack trace NullableBytesWritable vs NullableText
Posted by Alex McLintock <al...@gmail.com>.
I am using maps a lot so I guess this is related to PIG-919 which is closed
but not really fixed.
https://issues.apache.org/jira/browse/PIG-919
This suggested that I force the relevant types to (chararray) and that seems
to have worked as a work around.
Alex
On 7 February 2011 19:39, Alex McLintock <al...@gmail.com> wrote:
> Can anyone give me any hints on why a JOIN may be failing with this weird
> error....
>
>
>
> I can DESCRIBE the two tables it is joining
>
> justurls: {tweetid: bytearray,userid: bytearray,url: bytearray}
> userdb: {ouruserid: bytearray,friendid: bytearray}
>
> and the join itself is
>
>
> urlspertimeline = JOIN userdb BY friendid,
> justurls BY userid;
>
>
> ERROR 1066: Unable to open iterator for alias urlspertimeline. Backend
> error : Type mismatch in key from map: expected
> org.apache.pig.impl.io.NullableBytesWritable, recieved
> org.apache.pig.impl.io.NullableText
>
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias urlspertimeline. Backend error : Type mismatch in
> key from map: expected org.apache.pig.impl.io.NullableBytesWritable,
> recieved org.apache.pig.impl.io.NullableText
> at org.apache.pig.PigServer.openIterator(PigServer.java:742)
> at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> at org.apache.pig.Main.run(Main.java:510)
> at org.apache.pig.Main.main(Main.java:107)
> Caused by: java.io.IOException: Type mismatch in key from map: expected
> org.apache.pig.impl.io.NullableBytesWritable, recieved
> org.apache.pig.impl.io.NullableText
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:845)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:541)
> at
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:116)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:238)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:231)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:53)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>
>
>