You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Superymk <su...@hotmail.com> on 2012/04/28 06:13:51 UTC

Node-wide Combiner

Hi all,

I am a newbie in Hadoop and I like the system. I have one question: Is 
there a node-wide combiner or something similar in Hadoop? I think it 
can reduce the number of intermediate results in further. Any hint?

Thanks a lot!

Superymk

Re: Node-wide Combiner

Posted by Robert Evans <ev...@YAHOO-INC.COM>.

Do you mean that when multiple map jobs run on the same node, that there is a combiner that will run across all of that code.  There is nothing for that right now.  It seems like it could be somewhat difficult to get right given the current architecture.

--Bobby Evans


On 4/27/12 11:13 PM, "Superymk" <su...@hotmail.com> wrote:

Hi all,

I am a newbie in Hadoop and I like the system. I have one question: Is
there a node-wide combiner or something similar in Hadoop? I think it
can reduce the number of intermediate results in further. Any hint?

Thanks a lot!

Superymk