You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Alan Gates (JIRA)" <ji...@apache.org> on 2009/12/11 00:24:18 UTC
[jira] Commented: (PIG-919) Type mismatch in key from map: expected
org.apache.pig.impl.io.NullableBytesWritable, recieved
org.apache.pig.impl.io.NullableText when doing simple group
[ https://issues.apache.org/jira/browse/PIG-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12789016#action_12789016 ]
Alan Gates commented on PIG-919:
--------------------------------
This was closed Fixed when it should have been closed Won't Fix or Later. No patch was checked in for it. Where are you seeing this error? In general Pig Latin does not allow users to declare the type of map values, so you have to explicitly cast map values if you want to group by them so that Pig knows how to set the key in the MR job.
> Type mismatch in key from map: expected org.apache.pig.impl.io.NullableBytesWritable, recieved org.apache.pig.impl.io.NullableText when doing simple group
> ----------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-919
> URL: https://issues.apache.org/jira/browse/PIG-919
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Viraj Bhat
> Assignee: Viraj Bhat
> Fix For: 0.3.0
>
> Attachments: GenHashList.java, mapscript.pig, mymapudf.jar
>
>
> I have a Pig script, which takes in a student file and generates a bag of maps. I later want to group on the value of the key "name0" which corresponds to the first name of the student.
> {code}
> register mymapudf.jar;
> data = LOAD '/user/viraj/studenttab10k' AS (somename:chararray,age:long,marks:float);
> genmap = foreach data generate flatten(mymapudf.GenHashList(somename,' ')) as bp:map[], age, marks;
> getfirstnames = foreach genmap generate bp#'name0' as firstname, age, marks;
> filternonnullfirstnames = filter getfirstnames by firstname is not null;
> groupgenmap = group filternonnullfirstnames by firstname;
> dump groupgenmap;
> {code}
> When I execute this code, I get an error in the Map Phase:
> ===========================================================================================================
> java.io.IOException: Type mismatch in key from map: expected org.apache.pig.impl.io.NullableBytesWritable, recieved org.apache.pig.impl.io.NullableText
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:415)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:108)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:253)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:242)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:93)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)
> ===========================================================================================================
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.