You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Stefan Groschupf <sg...@media-style.com> on 2006/03/09 01:27:14 UTC
map tasks < reduce task
Hi developers,
It looks like hadoop hangs for ever when there are less map tasks
than reduce tasks.
I had generate a nutch segment using a setup that configured 12 map
and 6 reducers but fetched this segment with a setup running with 18
map tasks and 9 reduces. (on 9 boxes),
Now hadoop hangs with reduce > copy > task_m_22fc8c@box09:50040.
Is that a known problem? Should I open a bug or there may something
other that froze my hadoop?
Thanks for any hints
Stefan
---------------------------------------------
blog: http://www.find23.org
company: http://www.media-style.com
Re: map tasks < reduce task
Posted by Stefan Groschupf <sg...@media-style.com>.
May be interesting for you is that there is a heavy io on my
"box09", 329megabit/sec out and 36megabit/sec in.
A kind of ping pong effect? ...
Am 09.03.2006 um 01:38 schrieb Doug Cutting:
> Stefan Groschupf wrote:
>> It looks like hadoop hangs for ever when there are less map
>> tasks than reduce tasks.
>> I had generate a nutch segment using a setup that configured 12
>> map and 6 reducers but fetched this segment with a setup running
>> with 18 map tasks and 9 reduces. (on 9 boxes),
>
> That should work fine.
>
>> Now hadoop hangs with reduce > copy > task_m_22fc8c@box09:50040.
>
> That sounds like the same problem Michael Stack reported earlier
> today.
>
>> Is that a known problem? Should I open a bug or there may
>> something other that froze my hadoop?
>
> Sounds like a bug.
>
> Doug
>
---------------------------------------------
blog: http://www.find23.org
company: http://www.media-style.com
Re: map tasks < reduce task
Posted by Doug Cutting <cu...@apache.org>.
Stefan Groschupf wrote:
> It looks like hadoop hangs for ever when there are less map tasks than
> reduce tasks.
> I had generate a nutch segment using a setup that configured 12 map and
> 6 reducers but fetched this segment with a setup running with 18 map
> tasks and 9 reduces. (on 9 boxes),
That should work fine.
> Now hadoop hangs with reduce > copy > task_m_22fc8c@box09:50040.
That sounds like the same problem Michael Stack reported earlier today.
> Is that a known problem? Should I open a bug or there may something
> other that froze my hadoop?
Sounds like a bug.
Doug