You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by venkat <ve...@fifthgentech.com> on 2011/06/23 13:26:45 UTC
Unexpected data type -1 found in stream in PIG-0.8.1-src and Cassandra-0.8-src
Hi,
Previously we had cassandra-0.7.6-2 setup installed in a cluster and
used PIG Scripts for reading data from cassandra DB.
Right now we changed our Cassandra version from Cassandra-0.7.6-2 to
Cassandra-0.8.0 and the cassandra cluster working fine now, but while
reading from cassandra using Pig-0.8.1 is throwing below error.
"2011-06-23 03:06:58,629 [Thread-13] WARN
org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
java.lang.RuntimeException: Unexpected data type -1 found in stream.
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:478)
at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541)
at org.apache.pig.data.BinInterSedes.writeBag(BinInterSedes.java:522)
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:361)
at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:541)
at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:357)
at
org.apache.pig.impl.io.InterRecordWriter.write(InterRecordWriter.java:73)
at org.apache.pig.impl.io.InterStorage.putNext(InterStorage.java:87)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:138)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:97)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:498)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:242)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:58)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)"
please advice.
thanks,
venkat
Re: Unexpected data type -1 found in stream in PIG-0.8.1-src and
Cassandra-0.8-src
Posted by Daniel Dai <ji...@yahoo-inc.com>.
One possibility is that you produced some unknown data type in your UDF.
Check https://issues.apache.org/jira/browse/PIG-1826.
Daniel
On 06/23/2011 06:08 AM, Matt Davies wrote:
> I've seen this before when some data was a null, and I wasn't checking for
> it. Have you tried stepping through the pig script by
>
> 1. pig -r ...... (generates script with any variables replaced)
> 2. copy / paste lines into a grunt shell
>
> This will tell you (most of the time) where the problem lies in your
> script.
>
> -Matt
>
> On Thu, Jun 23, 2011 at 5:26 AM, venkat<ve...@fifthgentech.com>wrote:
>
>> Hi,
>>
>> Previously we had cassandra-0.7.6-2 setup installed in a cluster and used
>> PIG Scripts for reading data from cassandra DB.
>>
>> Right now we changed our Cassandra version from Cassandra-0.7.6-2 to
>> Cassandra-0.8.0 and the cassandra cluster working fine now, but while
>> reading from cassandra using Pig-0.8.1 is throwing below error.
>>
>> "2011-06-23 03:06:58,629 [Thread-13] WARN org.apache.hadoop.mapred.**LocalJobRunner
>> - job_local_0001
>> java.lang.RuntimeException: Unexpected data type -1 found in stream.
>> at org.apache.pig.data.**BinInterSedes.writeDatum(**
>> BinInterSedes.java:478)
>> at org.apache.pig.data.**BinInterSedes.writeTuple(**
>> BinInterSedes.java:541)
>> at org.apache.pig.data.**BinInterSedes.writeBag(**
>> BinInterSedes.java:522)
>> at org.apache.pig.data.**BinInterSedes.writeDatum(**
>> BinInterSedes.java:361)
>> at org.apache.pig.data.**BinInterSedes.writeTuple(**
>> BinInterSedes.java:541)
>> at org.apache.pig.data.**BinInterSedes.writeDatum(**
>> BinInterSedes.java:357)
>> at org.apache.pig.impl.io.**InterRecordWriter.write(**
>> InterRecordWriter.java:73)
>> at org.apache.pig.impl.io.**InterStorage.putNext(**
>> InterStorage.java:87)
>> at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.**
>> PigOutputFormat$**PigRecordWriter.write(**PigOutputFormat.java:138)
>> at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.**
>> PigOutputFormat$**PigRecordWriter.write(**PigOutputFormat.java:97)
>> at org.apache.hadoop.mapred.**MapTask$**NewDirectOutputCollector.**
>> write(MapTask.java:498)
>> at org.apache.hadoop.mapreduce.**TaskInputOutputContext.write(**
>> TaskInputOutputContext.java:**80)
>> at org.apache.pig.backend.hadoop.**executionengine.**
>> mapReduceLayer.PigMapOnly$Map.**collect(PigMapOnly.java:48)
>> at org.apache.pig.backend.hadoop.**executionengine.**
>> mapReduceLayer.PigMapBase.map(**PigMapBase.java:242)
>> at org.apache.pig.backend.hadoop.**executionengine.**
>> mapReduceLayer.PigMapBase.map(**PigMapBase.java:58)
>> at org.apache.hadoop.mapreduce.**Mapper.run(Mapper.java:144)
>> at org.apache.hadoop.mapred.**MapTask.runNewMapper(MapTask.**java:621)
>> at org.apache.hadoop.mapred.**MapTask.run(MapTask.java:305)
>> at org.apache.hadoop.mapred.**LocalJobRunner$Job.run(**
>> LocalJobRunner.java:177)"
>>
>> please advice.
>>
>> thanks,
>> venkat
>>
Re: Unexpected data type -1 found in stream in PIG-0.8.1-src and Cassandra-0.8-src
Posted by Matt Davies <ma...@mattdavies.net>.
I've seen this before when some data was a null, and I wasn't checking for
it. Have you tried stepping through the pig script by
1. pig -r ...... (generates script with any variables replaced)
2. copy / paste lines into a grunt shell
This will tell you (most of the time) where the problem lies in your
script.
-Matt
On Thu, Jun 23, 2011 at 5:26 AM, venkat <ve...@fifthgentech.com>wrote:
> Hi,
>
> Previously we had cassandra-0.7.6-2 setup installed in a cluster and used
> PIG Scripts for reading data from cassandra DB.
>
> Right now we changed our Cassandra version from Cassandra-0.7.6-2 to
> Cassandra-0.8.0 and the cassandra cluster working fine now, but while
> reading from cassandra using Pig-0.8.1 is throwing below error.
>
> "2011-06-23 03:06:58,629 [Thread-13] WARN org.apache.hadoop.mapred.**LocalJobRunner
> - job_local_0001
> java.lang.RuntimeException: Unexpected data type -1 found in stream.
> at org.apache.pig.data.**BinInterSedes.writeDatum(**
> BinInterSedes.java:478)
> at org.apache.pig.data.**BinInterSedes.writeTuple(**
> BinInterSedes.java:541)
> at org.apache.pig.data.**BinInterSedes.writeBag(**
> BinInterSedes.java:522)
> at org.apache.pig.data.**BinInterSedes.writeDatum(**
> BinInterSedes.java:361)
> at org.apache.pig.data.**BinInterSedes.writeTuple(**
> BinInterSedes.java:541)
> at org.apache.pig.data.**BinInterSedes.writeDatum(**
> BinInterSedes.java:357)
> at org.apache.pig.impl.io.**InterRecordWriter.write(**
> InterRecordWriter.java:73)
> at org.apache.pig.impl.io.**InterStorage.putNext(**
> InterStorage.java:87)
> at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.**
> PigOutputFormat$**PigRecordWriter.write(**PigOutputFormat.java:138)
> at org.apache.pig.backend.hadoop.**executionengine.**mapReduceLayer.**
> PigOutputFormat$**PigRecordWriter.write(**PigOutputFormat.java:97)
> at org.apache.hadoop.mapred.**MapTask$**NewDirectOutputCollector.**
> write(MapTask.java:498)
> at org.apache.hadoop.mapreduce.**TaskInputOutputContext.write(**
> TaskInputOutputContext.java:**80)
> at org.apache.pig.backend.hadoop.**executionengine.**
> mapReduceLayer.PigMapOnly$Map.**collect(PigMapOnly.java:48)
> at org.apache.pig.backend.hadoop.**executionengine.**
> mapReduceLayer.PigMapBase.map(**PigMapBase.java:242)
> at org.apache.pig.backend.hadoop.**executionengine.**
> mapReduceLayer.PigMapBase.map(**PigMapBase.java:58)
> at org.apache.hadoop.mapreduce.**Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.**MapTask.runNewMapper(MapTask.**java:621)
> at org.apache.hadoop.mapred.**MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.**LocalJobRunner$Job.run(**
> LocalJobRunner.java:177)"
>
> please advice.
>
> thanks,
> venkat
>