You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by zhang jianfeng <zj...@gmail.com> on 2009/06/09 09:14:35 UTC

Group does not work in mapreduce mode ?

Hi all,

I run a pig script in local mode successfully, but it fails in mapreduce
mode.

This is the error message I found in the jobtracker:

java.io.IOException: Type mismatch in key from map: expected
org.apache.pig.impl.io.NullableBytesWritable, recieved
org.apache.pig.impl.io.NullableText
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:415)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:103)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:247)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)




I doubt this is caused by the group, because this error is always come when
I use the group statement.

Thank you for any help.


Jeff Zhang

Re: Group does not work in mapreduce mode ?

Posted by zhang jianfeng <zj...@gmail.com>.
I sovled this problem, because I can not put String into Tuple, I have to
use the DataByteArray.

But the String works fine in local mode.



On Tue, Jun 9, 2009 at 3:14 PM, zhang jianfeng <zj...@gmail.com> wrote:

> Hi all,
>
> I run a pig script in local mode successfully, but it fails in mapreduce
> mode.
>
> This is the error message I found in the jobtracker:
>
> java.io.IOException: Type mismatch in key from map: expected org.apache.pig.impl.io.NullableBytesWritable, recieved org.apache.pig.impl.io.NullableText
>
> 	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:415)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:103)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:247)
>
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:236)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:88)
> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
>
>
>
>
> I doubt this is caused by the group, because this error is always come when
> I use the group statement.
>
> Thank you for any help.
>
>
> Jeff Zhang
>
>
>