You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by tim robertson <ti...@gmail.com> on 2008/12/01 18:43:54 UTC
Reduce can't change type of "Key" class?
Hi all,
I am trying to do a map which emits:
- Key<MyKeyType>
- Value<Text>
and then reduce that emits
- Key<Text>
- Values of type<Text>
It blows up after a few reductions with:
java.io.IOException: wrong key class: class org.apache.hadoop.io.Text
is not class com.blah.blah.MyKeyType
I can understand why... but is there a work around for this?
(I do want to do this since the conversion is an expensive operation,
and by doing it in the reduction, I can do it once for the group...)
Maybe a Combiner that runs exactly once and does the conversion and
then a different reducer? Is it possible?
Thanks for any help / insights
Tim