You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@hadoop.apache.org by himanshu chandola <hi...@yahoo.com> on 2009/09/06 21:52:49 UTC

questions on map and reduce

Hi Everyone,
What would be the best way to get map to output values  in different formats (reduce needs to add the integer ones) . I realize that writing an ObjectWritable (not of the primitive type that comes with hadoop) might be the best. I basically am worried that if I output integers to Text and convert them back to IntWritable, it would be a performance overhead. Would it be significant enough to worry about ? And can you think of anything other than ObjectWritable or having everything as Text to do this ?

And another question was is there a way to get reduce to set values in JobConf or do I just flush that into disk and let the sequential job to read that. Basically , am I reading too much into the performance overhead thing ?



Thanks

 Morpheus: Do you believe in fate, Neo?
Neo: No.
Morpheus: Why Not?
Neo: Because I don't like the idea that I'm not in control of my life.