You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by jamal sasha <ja...@gmail.com> on 2012/11/20 17:52:47 UTC
reducer not starting
I am not sure whats happening, but I wrote a simple mapper and reducer
script.
And I am testing it against a small dataset (like few lines long).
For some reason reducer is just not starting.. and mapper is executing
again and again?
12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
Let me know if you want the code also.
Any clues of where I am going wrong?
Thanks
Re: reducer not starting
Posted by bharath vissapragada <bh...@gmail.com>.
As harsh suggested, you might want to check the task logs on slaves (you
can do it though web UI by clicking on map/reduce task links) and see if
there are any exceptions .
On Wed, Nov 21, 2012 at 8:06 PM, jamal sasha <ja...@gmail.com> wrote:
> Hi
> Thanks for the insights.
> I noticed that these restarts of mappers were because in the shebang i had
> Usr/env/bin instead of usr/env/bin python
> Any clue of what was going on with reducers not starting but mappers being
> executed again and again.
> Probably a very naive question but i am newbie you see :)
>
>
>
> On Wednesday, November 21, 2012, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
> > Just FYI, you don't need to stop the job, update the host, and retry.
> >
> > Just update the host while the job is running and it should retry and
> restart.
> >
> > I had a similar issue with one of my node where the hosts file were
> > not updated. After the updated it has automatically resume the work...
> >
> > JM
> >
> > 2012/11/21, praveenesh kumar <pr...@gmail.com>:
> >> Sometimes its network issue, reducers are not able to find hostnames or
> IPs
> >> of the other machines. Make sure your /etc/hosts entries and hostnames
> are
> >> correct.
> >>
> >> Regards,
> >> Praveenesh
> >>
> >> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
> >>
> >>> Your mappers are failing (possibly a user-side error or an
> >>> environmental one) and are being reattempted by the framework (default
> >>> behavior, attempts 4 times to avoid transient failure scenario).
> >>>
> >>> Visit your job's logs in the JobTracker web UI, to find more
> >>> information on why your tasks fail.
> >>>
> >>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
> >>> wrote:
> >>> >
> >>> >
> >>> >
> >>> > I am not sure whats happening, but I wrote a simple mapper and
> reducer
> >>> > script.
> >>> >
> >>> >
> >>> >
> >>> > And I am testing it against a small dataset (like few lines long).
> >>> >
> >>> >
> >>> >
> >>> > For some reason reducer is just not starting.. and mapper is
> executing
> >>> again
> >>> > and again?
> >>> >
> >>> >
> >>> >
> >>> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
> >>> >
> >>> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > Let me know if you want the code also.
> >>> >
> >>> > Any clues of where I am going wrong?
> >>> >
> >>> > Thanks
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Harsh J
> >>>
> >>
> >
>
--
Regards,
Bharath .V
w:http://researchweb.iiit.ac.in/~bharath.v
Re: reducer not starting
Posted by bharath vissapragada <bh...@gmail.com>.
As harsh suggested, you might want to check the task logs on slaves (you
can do it though web UI by clicking on map/reduce task links) and see if
there are any exceptions .
On Wed, Nov 21, 2012 at 8:06 PM, jamal sasha <ja...@gmail.com> wrote:
> Hi
> Thanks for the insights.
> I noticed that these restarts of mappers were because in the shebang i had
> Usr/env/bin instead of usr/env/bin python
> Any clue of what was going on with reducers not starting but mappers being
> executed again and again.
> Probably a very naive question but i am newbie you see :)
>
>
>
> On Wednesday, November 21, 2012, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
> > Just FYI, you don't need to stop the job, update the host, and retry.
> >
> > Just update the host while the job is running and it should retry and
> restart.
> >
> > I had a similar issue with one of my node where the hosts file were
> > not updated. After the updated it has automatically resume the work...
> >
> > JM
> >
> > 2012/11/21, praveenesh kumar <pr...@gmail.com>:
> >> Sometimes its network issue, reducers are not able to find hostnames or
> IPs
> >> of the other machines. Make sure your /etc/hosts entries and hostnames
> are
> >> correct.
> >>
> >> Regards,
> >> Praveenesh
> >>
> >> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
> >>
> >>> Your mappers are failing (possibly a user-side error or an
> >>> environmental one) and are being reattempted by the framework (default
> >>> behavior, attempts 4 times to avoid transient failure scenario).
> >>>
> >>> Visit your job's logs in the JobTracker web UI, to find more
> >>> information on why your tasks fail.
> >>>
> >>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
> >>> wrote:
> >>> >
> >>> >
> >>> >
> >>> > I am not sure whats happening, but I wrote a simple mapper and
> reducer
> >>> > script.
> >>> >
> >>> >
> >>> >
> >>> > And I am testing it against a small dataset (like few lines long).
> >>> >
> >>> >
> >>> >
> >>> > For some reason reducer is just not starting.. and mapper is
> executing
> >>> again
> >>> > and again?
> >>> >
> >>> >
> >>> >
> >>> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
> >>> >
> >>> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > Let me know if you want the code also.
> >>> >
> >>> > Any clues of where I am going wrong?
> >>> >
> >>> > Thanks
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Harsh J
> >>>
> >>
> >
>
--
Regards,
Bharath .V
w:http://researchweb.iiit.ac.in/~bharath.v
Re: reducer not starting
Posted by bharath vissapragada <bh...@gmail.com>.
As harsh suggested, you might want to check the task logs on slaves (you
can do it though web UI by clicking on map/reduce task links) and see if
there are any exceptions .
On Wed, Nov 21, 2012 at 8:06 PM, jamal sasha <ja...@gmail.com> wrote:
> Hi
> Thanks for the insights.
> I noticed that these restarts of mappers were because in the shebang i had
> Usr/env/bin instead of usr/env/bin python
> Any clue of what was going on with reducers not starting but mappers being
> executed again and again.
> Probably a very naive question but i am newbie you see :)
>
>
>
> On Wednesday, November 21, 2012, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
> > Just FYI, you don't need to stop the job, update the host, and retry.
> >
> > Just update the host while the job is running and it should retry and
> restart.
> >
> > I had a similar issue with one of my node where the hosts file were
> > not updated. After the updated it has automatically resume the work...
> >
> > JM
> >
> > 2012/11/21, praveenesh kumar <pr...@gmail.com>:
> >> Sometimes its network issue, reducers are not able to find hostnames or
> IPs
> >> of the other machines. Make sure your /etc/hosts entries and hostnames
> are
> >> correct.
> >>
> >> Regards,
> >> Praveenesh
> >>
> >> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
> >>
> >>> Your mappers are failing (possibly a user-side error or an
> >>> environmental one) and are being reattempted by the framework (default
> >>> behavior, attempts 4 times to avoid transient failure scenario).
> >>>
> >>> Visit your job's logs in the JobTracker web UI, to find more
> >>> information on why your tasks fail.
> >>>
> >>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
> >>> wrote:
> >>> >
> >>> >
> >>> >
> >>> > I am not sure whats happening, but I wrote a simple mapper and
> reducer
> >>> > script.
> >>> >
> >>> >
> >>> >
> >>> > And I am testing it against a small dataset (like few lines long).
> >>> >
> >>> >
> >>> >
> >>> > For some reason reducer is just not starting.. and mapper is
> executing
> >>> again
> >>> > and again?
> >>> >
> >>> >
> >>> >
> >>> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
> >>> >
> >>> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > Let me know if you want the code also.
> >>> >
> >>> > Any clues of where I am going wrong?
> >>> >
> >>> > Thanks
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Harsh J
> >>>
> >>
> >
>
--
Regards,
Bharath .V
w:http://researchweb.iiit.ac.in/~bharath.v
Re: reducer not starting
Posted by bharath vissapragada <bh...@gmail.com>.
As harsh suggested, you might want to check the task logs on slaves (you
can do it though web UI by clicking on map/reduce task links) and see if
there are any exceptions .
On Wed, Nov 21, 2012 at 8:06 PM, jamal sasha <ja...@gmail.com> wrote:
> Hi
> Thanks for the insights.
> I noticed that these restarts of mappers were because in the shebang i had
> Usr/env/bin instead of usr/env/bin python
> Any clue of what was going on with reducers not starting but mappers being
> executed again and again.
> Probably a very naive question but i am newbie you see :)
>
>
>
> On Wednesday, November 21, 2012, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
> > Just FYI, you don't need to stop the job, update the host, and retry.
> >
> > Just update the host while the job is running and it should retry and
> restart.
> >
> > I had a similar issue with one of my node where the hosts file were
> > not updated. After the updated it has automatically resume the work...
> >
> > JM
> >
> > 2012/11/21, praveenesh kumar <pr...@gmail.com>:
> >> Sometimes its network issue, reducers are not able to find hostnames or
> IPs
> >> of the other machines. Make sure your /etc/hosts entries and hostnames
> are
> >> correct.
> >>
> >> Regards,
> >> Praveenesh
> >>
> >> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
> >>
> >>> Your mappers are failing (possibly a user-side error or an
> >>> environmental one) and are being reattempted by the framework (default
> >>> behavior, attempts 4 times to avoid transient failure scenario).
> >>>
> >>> Visit your job's logs in the JobTracker web UI, to find more
> >>> information on why your tasks fail.
> >>>
> >>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
> >>> wrote:
> >>> >
> >>> >
> >>> >
> >>> > I am not sure whats happening, but I wrote a simple mapper and
> reducer
> >>> > script.
> >>> >
> >>> >
> >>> >
> >>> > And I am testing it against a small dataset (like few lines long).
> >>> >
> >>> >
> >>> >
> >>> > For some reason reducer is just not starting.. and mapper is
> executing
> >>> again
> >>> > and again?
> >>> >
> >>> >
> >>> >
> >>> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
> >>> >
> >>> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
> >>> >
> >>> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > Let me know if you want the code also.
> >>> >
> >>> > Any clues of where I am going wrong?
> >>> >
> >>> > Thanks
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Harsh J
> >>>
> >>
> >
>
--
Regards,
Bharath .V
w:http://researchweb.iiit.ac.in/~bharath.v
Re: reducer not starting
Posted by jamal sasha <ja...@gmail.com>.
Hi
Thanks for the insights.
I noticed that these restarts of mappers were because in the shebang i had
Usr/env/bin instead of usr/env/bin python
Any clue of what was going on with reducers not starting but mappers being
executed again and again.
Probably a very naive question but i am newbie you see :)
On Wednesday, November 21, 2012, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> Just FYI, you don't need to stop the job, update the host, and retry.
>
> Just update the host while the job is running and it should retry and
restart.
>
> I had a similar issue with one of my node where the hosts file were
> not updated. After the updated it has automatically resume the work...
>
> JM
>
> 2012/11/21, praveenesh kumar <pr...@gmail.com>:
>> Sometimes its network issue, reducers are not able to find hostnames or
IPs
>> of the other machines. Make sure your /etc/hosts entries and hostnames
are
>> correct.
>>
>> Regards,
>> Praveenesh
>>
>> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
>>
>>> Your mappers are failing (possibly a user-side error or an
>>> environmental one) and are being reattempted by the framework (default
>>> behavior, attempts 4 times to avoid transient failure scenario).
>>>
>>> Visit your job's logs in the JobTracker web UI, to find more
>>> information on why your tasks fail.
>>>
>>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
>>> wrote:
>>> >
>>> >
>>> >
>>> > I am not sure whats happening, but I wrote a simple mapper and reducer
>>> > script.
>>> >
>>> >
>>> >
>>> > And I am testing it against a small dataset (like few lines long).
>>> >
>>> >
>>> >
>>> > For some reason reducer is just not starting.. and mapper is executing
>>> again
>>> > and again?
>>> >
>>> >
>>> >
>>> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
>>> >
>>> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
>>> >
>>> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
>>> >
>>> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
>>> >
>>> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
>>> >
>>> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > Let me know if you want the code also.
>>> >
>>> > Any clues of where I am going wrong?
>>> >
>>> > Thanks
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>
>
Re: reducer not starting
Posted by jamal sasha <ja...@gmail.com>.
Hi
Thanks for the insights.
I noticed that these restarts of mappers were because in the shebang i had
Usr/env/bin instead of usr/env/bin python
Any clue of what was going on with reducers not starting but mappers being
executed again and again.
Probably a very naive question but i am newbie you see :)
On Wednesday, November 21, 2012, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> Just FYI, you don't need to stop the job, update the host, and retry.
>
> Just update the host while the job is running and it should retry and
restart.
>
> I had a similar issue with one of my node where the hosts file were
> not updated. After the updated it has automatically resume the work...
>
> JM
>
> 2012/11/21, praveenesh kumar <pr...@gmail.com>:
>> Sometimes its network issue, reducers are not able to find hostnames or
IPs
>> of the other machines. Make sure your /etc/hosts entries and hostnames
are
>> correct.
>>
>> Regards,
>> Praveenesh
>>
>> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
>>
>>> Your mappers are failing (possibly a user-side error or an
>>> environmental one) and are being reattempted by the framework (default
>>> behavior, attempts 4 times to avoid transient failure scenario).
>>>
>>> Visit your job's logs in the JobTracker web UI, to find more
>>> information on why your tasks fail.
>>>
>>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
>>> wrote:
>>> >
>>> >
>>> >
>>> > I am not sure whats happening, but I wrote a simple mapper and reducer
>>> > script.
>>> >
>>> >
>>> >
>>> > And I am testing it against a small dataset (like few lines long).
>>> >
>>> >
>>> >
>>> > For some reason reducer is just not starting.. and mapper is executing
>>> again
>>> > and again?
>>> >
>>> >
>>> >
>>> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
>>> >
>>> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
>>> >
>>> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
>>> >
>>> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
>>> >
>>> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
>>> >
>>> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > Let me know if you want the code also.
>>> >
>>> > Any clues of where I am going wrong?
>>> >
>>> > Thanks
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>
>
Re: reducer not starting
Posted by jamal sasha <ja...@gmail.com>.
Hi
Thanks for the insights.
I noticed that these restarts of mappers were because in the shebang i had
Usr/env/bin instead of usr/env/bin python
Any clue of what was going on with reducers not starting but mappers being
executed again and again.
Probably a very naive question but i am newbie you see :)
On Wednesday, November 21, 2012, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> Just FYI, you don't need to stop the job, update the host, and retry.
>
> Just update the host while the job is running and it should retry and
restart.
>
> I had a similar issue with one of my node where the hosts file were
> not updated. After the updated it has automatically resume the work...
>
> JM
>
> 2012/11/21, praveenesh kumar <pr...@gmail.com>:
>> Sometimes its network issue, reducers are not able to find hostnames or
IPs
>> of the other machines. Make sure your /etc/hosts entries and hostnames
are
>> correct.
>>
>> Regards,
>> Praveenesh
>>
>> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
>>
>>> Your mappers are failing (possibly a user-side error or an
>>> environmental one) and are being reattempted by the framework (default
>>> behavior, attempts 4 times to avoid transient failure scenario).
>>>
>>> Visit your job's logs in the JobTracker web UI, to find more
>>> information on why your tasks fail.
>>>
>>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
>>> wrote:
>>> >
>>> >
>>> >
>>> > I am not sure whats happening, but I wrote a simple mapper and reducer
>>> > script.
>>> >
>>> >
>>> >
>>> > And I am testing it against a small dataset (like few lines long).
>>> >
>>> >
>>> >
>>> > For some reason reducer is just not starting.. and mapper is executing
>>> again
>>> > and again?
>>> >
>>> >
>>> >
>>> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
>>> >
>>> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
>>> >
>>> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
>>> >
>>> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
>>> >
>>> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
>>> >
>>> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > Let me know if you want the code also.
>>> >
>>> > Any clues of where I am going wrong?
>>> >
>>> > Thanks
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>
>
Re: reducer not starting
Posted by jamal sasha <ja...@gmail.com>.
Hi
Thanks for the insights.
I noticed that these restarts of mappers were because in the shebang i had
Usr/env/bin instead of usr/env/bin python
Any clue of what was going on with reducers not starting but mappers being
executed again and again.
Probably a very naive question but i am newbie you see :)
On Wednesday, November 21, 2012, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:
> Just FYI, you don't need to stop the job, update the host, and retry.
>
> Just update the host while the job is running and it should retry and
restart.
>
> I had a similar issue with one of my node where the hosts file were
> not updated. After the updated it has automatically resume the work...
>
> JM
>
> 2012/11/21, praveenesh kumar <pr...@gmail.com>:
>> Sometimes its network issue, reducers are not able to find hostnames or
IPs
>> of the other machines. Make sure your /etc/hosts entries and hostnames
are
>> correct.
>>
>> Regards,
>> Praveenesh
>>
>> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
>>
>>> Your mappers are failing (possibly a user-side error or an
>>> environmental one) and are being reattempted by the framework (default
>>> behavior, attempts 4 times to avoid transient failure scenario).
>>>
>>> Visit your job's logs in the JobTracker web UI, to find more
>>> information on why your tasks fail.
>>>
>>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
>>> wrote:
>>> >
>>> >
>>> >
>>> > I am not sure whats happening, but I wrote a simple mapper and reducer
>>> > script.
>>> >
>>> >
>>> >
>>> > And I am testing it against a small dataset (like few lines long).
>>> >
>>> >
>>> >
>>> > For some reason reducer is just not starting.. and mapper is executing
>>> again
>>> > and again?
>>> >
>>> >
>>> >
>>> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
>>> >
>>> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
>>> >
>>> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
>>> >
>>> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
>>> >
>>> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
>>> >
>>> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
>>> >
>>> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > Let me know if you want the code also.
>>> >
>>> > Any clues of where I am going wrong?
>>> >
>>> > Thanks
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Harsh J
>>>
>>
>
Re: reducer not starting
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Just FYI, you don't need to stop the job, update the host, and retry.
Just update the host while the job is running and it should retry and restart.
I had a similar issue with one of my node where the hosts file were
not updated. After the updated it has automatically resume the work...
JM
2012/11/21, praveenesh kumar <pr...@gmail.com>:
> Sometimes its network issue, reducers are not able to find hostnames or IPs
> of the other machines. Make sure your /etc/hosts entries and hostnames are
> correct.
>
> Regards,
> Praveenesh
>
> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Your mappers are failing (possibly a user-side error or an
>> environmental one) and are being reattempted by the framework (default
>> behavior, attempts 4 times to avoid transient failure scenario).
>>
>> Visit your job's logs in the JobTracker web UI, to find more
>> information on why your tasks fail.
>>
>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
>> wrote:
>> >
>> >
>> >
>> > I am not sure whats happening, but I wrote a simple mapper and reducer
>> > script.
>> >
>> >
>> >
>> > And I am testing it against a small dataset (like few lines long).
>> >
>> >
>> >
>> > For some reason reducer is just not starting.. and mapper is executing
>> again
>> > and again?
>> >
>> >
>> >
>> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
>> >
>> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
>> >
>> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
>> >
>> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
>> >
>> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
>> >
>> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
>> >
>> >
>> >
>> >
>> >
>> > Let me know if you want the code also.
>> >
>> > Any clues of where I am going wrong?
>> >
>> > Thanks
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
Re: reducer not starting
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Just FYI, you don't need to stop the job, update the host, and retry.
Just update the host while the job is running and it should retry and restart.
I had a similar issue with one of my node where the hosts file were
not updated. After the updated it has automatically resume the work...
JM
2012/11/21, praveenesh kumar <pr...@gmail.com>:
> Sometimes its network issue, reducers are not able to find hostnames or IPs
> of the other machines. Make sure your /etc/hosts entries and hostnames are
> correct.
>
> Regards,
> Praveenesh
>
> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Your mappers are failing (possibly a user-side error or an
>> environmental one) and are being reattempted by the framework (default
>> behavior, attempts 4 times to avoid transient failure scenario).
>>
>> Visit your job's logs in the JobTracker web UI, to find more
>> information on why your tasks fail.
>>
>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
>> wrote:
>> >
>> >
>> >
>> > I am not sure whats happening, but I wrote a simple mapper and reducer
>> > script.
>> >
>> >
>> >
>> > And I am testing it against a small dataset (like few lines long).
>> >
>> >
>> >
>> > For some reason reducer is just not starting.. and mapper is executing
>> again
>> > and again?
>> >
>> >
>> >
>> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
>> >
>> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
>> >
>> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
>> >
>> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
>> >
>> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
>> >
>> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
>> >
>> >
>> >
>> >
>> >
>> > Let me know if you want the code also.
>> >
>> > Any clues of where I am going wrong?
>> >
>> > Thanks
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
Re: reducer not starting
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Just FYI, you don't need to stop the job, update the host, and retry.
Just update the host while the job is running and it should retry and restart.
I had a similar issue with one of my node where the hosts file were
not updated. After the updated it has automatically resume the work...
JM
2012/11/21, praveenesh kumar <pr...@gmail.com>:
> Sometimes its network issue, reducers are not able to find hostnames or IPs
> of the other machines. Make sure your /etc/hosts entries and hostnames are
> correct.
>
> Regards,
> Praveenesh
>
> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Your mappers are failing (possibly a user-side error or an
>> environmental one) and are being reattempted by the framework (default
>> behavior, attempts 4 times to avoid transient failure scenario).
>>
>> Visit your job's logs in the JobTracker web UI, to find more
>> information on why your tasks fail.
>>
>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
>> wrote:
>> >
>> >
>> >
>> > I am not sure whats happening, but I wrote a simple mapper and reducer
>> > script.
>> >
>> >
>> >
>> > And I am testing it against a small dataset (like few lines long).
>> >
>> >
>> >
>> > For some reason reducer is just not starting.. and mapper is executing
>> again
>> > and again?
>> >
>> >
>> >
>> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
>> >
>> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
>> >
>> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
>> >
>> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
>> >
>> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
>> >
>> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
>> >
>> >
>> >
>> >
>> >
>> > Let me know if you want the code also.
>> >
>> > Any clues of where I am going wrong?
>> >
>> > Thanks
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
Re: reducer not starting
Posted by Jean-Marc Spaggiari <je...@spaggiari.org>.
Just FYI, you don't need to stop the job, update the host, and retry.
Just update the host while the job is running and it should retry and restart.
I had a similar issue with one of my node where the hosts file were
not updated. After the updated it has automatically resume the work...
JM
2012/11/21, praveenesh kumar <pr...@gmail.com>:
> Sometimes its network issue, reducers are not able to find hostnames or IPs
> of the other machines. Make sure your /etc/hosts entries and hostnames are
> correct.
>
> Regards,
> Praveenesh
>
> On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Your mappers are failing (possibly a user-side error or an
>> environmental one) and are being reattempted by the framework (default
>> behavior, attempts 4 times to avoid transient failure scenario).
>>
>> Visit your job's logs in the JobTracker web UI, to find more
>> information on why your tasks fail.
>>
>> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
>> wrote:
>> >
>> >
>> >
>> > I am not sure whats happening, but I wrote a simple mapper and reducer
>> > script.
>> >
>> >
>> >
>> > And I am testing it against a small dataset (like few lines long).
>> >
>> >
>> >
>> > For some reason reducer is just not starting.. and mapper is executing
>> again
>> > and again?
>> >
>> >
>> >
>> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
>> >
>> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
>> >
>> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
>> >
>> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
>> >
>> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
>> >
>> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
>> >
>> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
>> >
>> >
>> >
>> >
>> >
>> > Let me know if you want the code also.
>> >
>> > Any clues of where I am going wrong?
>> >
>> > Thanks
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
Re: reducer not starting
Posted by praveenesh kumar <pr...@gmail.com>.
Sometimes its network issue, reducers are not able to find hostnames or IPs
of the other machines. Make sure your /etc/hosts entries and hostnames are
correct.
Regards,
Praveenesh
On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
> Your mappers are failing (possibly a user-side error or an
> environmental one) and are being reattempted by the framework (default
> behavior, attempts 4 times to avoid transient failure scenario).
>
> Visit your job's logs in the JobTracker web UI, to find more
> information on why your tasks fail.
>
> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
> wrote:
> >
> >
> >
> > I am not sure whats happening, but I wrote a simple mapper and reducer
> > script.
> >
> >
> >
> > And I am testing it against a small dataset (like few lines long).
> >
> >
> >
> > For some reason reducer is just not starting.. and mapper is executing
> again
> > and again?
> >
> >
> >
> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
> >
> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
> >
> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
> >
> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
> >
> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
> >
> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
> >
> >
> >
> >
> >
> > Let me know if you want the code also.
> >
> > Any clues of where I am going wrong?
> >
> > Thanks
> >
> >
> >
> >
> >
> >
>
>
>
> --
> Harsh J
>
Re: reducer not starting
Posted by praveenesh kumar <pr...@gmail.com>.
Sometimes its network issue, reducers are not able to find hostnames or IPs
of the other machines. Make sure your /etc/hosts entries and hostnames are
correct.
Regards,
Praveenesh
On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
> Your mappers are failing (possibly a user-side error or an
> environmental one) and are being reattempted by the framework (default
> behavior, attempts 4 times to avoid transient failure scenario).
>
> Visit your job's logs in the JobTracker web UI, to find more
> information on why your tasks fail.
>
> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
> wrote:
> >
> >
> >
> > I am not sure whats happening, but I wrote a simple mapper and reducer
> > script.
> >
> >
> >
> > And I am testing it against a small dataset (like few lines long).
> >
> >
> >
> > For some reason reducer is just not starting.. and mapper is executing
> again
> > and again?
> >
> >
> >
> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
> >
> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
> >
> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
> >
> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
> >
> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
> >
> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
> >
> >
> >
> >
> >
> > Let me know if you want the code also.
> >
> > Any clues of where I am going wrong?
> >
> > Thanks
> >
> >
> >
> >
> >
> >
>
>
>
> --
> Harsh J
>
Re: reducer not starting
Posted by praveenesh kumar <pr...@gmail.com>.
Sometimes its network issue, reducers are not able to find hostnames or IPs
of the other machines. Make sure your /etc/hosts entries and hostnames are
correct.
Regards,
Praveenesh
On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
> Your mappers are failing (possibly a user-side error or an
> environmental one) and are being reattempted by the framework (default
> behavior, attempts 4 times to avoid transient failure scenario).
>
> Visit your job's logs in the JobTracker web UI, to find more
> information on why your tasks fail.
>
> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
> wrote:
> >
> >
> >
> > I am not sure whats happening, but I wrote a simple mapper and reducer
> > script.
> >
> >
> >
> > And I am testing it against a small dataset (like few lines long).
> >
> >
> >
> > For some reason reducer is just not starting.. and mapper is executing
> again
> > and again?
> >
> >
> >
> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
> >
> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
> >
> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
> >
> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
> >
> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
> >
> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
> >
> >
> >
> >
> >
> > Let me know if you want the code also.
> >
> > Any clues of where I am going wrong?
> >
> > Thanks
> >
> >
> >
> >
> >
> >
>
>
>
> --
> Harsh J
>
Re: reducer not starting
Posted by praveenesh kumar <pr...@gmail.com>.
Sometimes its network issue, reducers are not able to find hostnames or IPs
of the other machines. Make sure your /etc/hosts entries and hostnames are
correct.
Regards,
Praveenesh
On Tue, Nov 20, 2012 at 10:46 PM, Harsh J <ha...@cloudera.com> wrote:
> Your mappers are failing (possibly a user-side error or an
> environmental one) and are being reattempted by the framework (default
> behavior, attempts 4 times to avoid transient failure scenario).
>
> Visit your job's logs in the JobTracker web UI, to find more
> information on why your tasks fail.
>
> On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com>
> wrote:
> >
> >
> >
> > I am not sure whats happening, but I wrote a simple mapper and reducer
> > script.
> >
> >
> >
> > And I am testing it against a small dataset (like few lines long).
> >
> >
> >
> > For some reason reducer is just not starting.. and mapper is executing
> again
> > and again?
> >
> >
> >
> > 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
> >
> > 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
> >
> > 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
> >
> > 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
> >
> > 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
> >
> > 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
> >
> > 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
> >
> >
> >
> >
> >
> > Let me know if you want the code also.
> >
> > Any clues of where I am going wrong?
> >
> > Thanks
> >
> >
> >
> >
> >
> >
>
>
>
> --
> Harsh J
>
Re: reducer not starting
Posted by Harsh J <ha...@cloudera.com>.
Your mappers are failing (possibly a user-side error or an
environmental one) and are being reattempted by the framework (default
behavior, attempts 4 times to avoid transient failure scenario).
Visit your job's logs in the JobTracker web UI, to find more
information on why your tasks fail.
On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com> wrote:
>
>
>
> I am not sure whats happening, but I wrote a simple mapper and reducer
> script.
>
>
>
> And I am testing it against a small dataset (like few lines long).
>
>
>
> For some reason reducer is just not starting.. and mapper is executing again
> and again?
>
>
>
> 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
>
> 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
>
> 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
>
> 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
>
> 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
>
> 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
>
>
>
>
>
> Let me know if you want the code also.
>
> Any clues of where I am going wrong?
>
> Thanks
>
>
>
>
>
>
--
Harsh J
Re: reducer not starting
Posted by Harsh J <ha...@cloudera.com>.
Your mappers are failing (possibly a user-side error or an
environmental one) and are being reattempted by the framework (default
behavior, attempts 4 times to avoid transient failure scenario).
Visit your job's logs in the JobTracker web UI, to find more
information on why your tasks fail.
On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com> wrote:
>
>
>
> I am not sure whats happening, but I wrote a simple mapper and reducer
> script.
>
>
>
> And I am testing it against a small dataset (like few lines long).
>
>
>
> For some reason reducer is just not starting.. and mapper is executing again
> and again?
>
>
>
> 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
>
> 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
>
> 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
>
> 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
>
> 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
>
> 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
>
>
>
>
>
> Let me know if you want the code also.
>
> Any clues of where I am going wrong?
>
> Thanks
>
>
>
>
>
>
--
Harsh J
Re: reducer not starting
Posted by Harsh J <ha...@cloudera.com>.
Your mappers are failing (possibly a user-side error or an
environmental one) and are being reattempted by the framework (default
behavior, attempts 4 times to avoid transient failure scenario).
Visit your job's logs in the JobTracker web UI, to find more
information on why your tasks fail.
On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com> wrote:
>
>
>
> I am not sure whats happening, but I wrote a simple mapper and reducer
> script.
>
>
>
> And I am testing it against a small dataset (like few lines long).
>
>
>
> For some reason reducer is just not starting.. and mapper is executing again
> and again?
>
>
>
> 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
>
> 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
>
> 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
>
> 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
>
> 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
>
> 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
>
>
>
>
>
> Let me know if you want the code also.
>
> Any clues of where I am going wrong?
>
> Thanks
>
>
>
>
>
>
--
Harsh J
Re: reducer not starting
Posted by Harsh J <ha...@cloudera.com>.
Your mappers are failing (possibly a user-side error or an
environmental one) and are being reattempted by the framework (default
behavior, attempts 4 times to avoid transient failure scenario).
Visit your job's logs in the JobTracker web UI, to find more
information on why your tasks fail.
On Tue, Nov 20, 2012 at 10:22 PM, jamal sasha <ja...@gmail.com> wrote:
>
>
>
> I am not sure whats happening, but I wrote a simple mapper and reducer
> script.
>
>
>
> And I am testing it against a small dataset (like few lines long).
>
>
>
> For some reason reducer is just not starting.. and mapper is executing again
> and again?
>
>
>
> 12/11/20 09:21:18 INFO streaming.StreamJob: map 0% reduce 0%
>
> 12/11/20 09:22:05 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:22:10 INFO streaming.StreamJob: map 100% reduce 0%
>
> 12/11/20 09:32:05 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:32:11 INFO streaming.StreamJob: map 0% reduce 0%
>
> 12/11/20 09:32:20 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:32:31 INFO streaming.StreamJob: map 100% reduce 0%
>
> 12/11/20 09:42:20 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:42:31 INFO streaming.StreamJob: map 0% reduce 0%
>
> 12/11/20 09:42:32 INFO streaming.StreamJob: map 50% reduce 0%
>
> 12/11/20 09:42:50 INFO streaming.StreamJob: map 100% reduce 0%
>
>
>
>
>
> Let me know if you want the code also.
>
> Any clues of where I am going wrong?
>
> Thanks
>
>
>
>
>
>
--
Harsh J