You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Mohit Anchlia <mo...@gmail.com> on 2013/03/07 06:51:03 UTC
mapred.max.tracker.failures
I am wondering what the correct behaviour is of this parameter? If it's set
to 4 does it mean job should fail if a job has more than 4 failures?
Re: mapred.max.tracker.failures
Posted by Abdelrhman Shettia <as...@hortonworks.com>.
Hi Mohit ,
This is The number of failed tasks in specified job after which the job will not run on the task tracker. The job's tasks will no longer be assigned to the task tracker. However, If the same task failed more than 4 times , The job will fail regardless.
Hope this helps.
Thanks
-Abdelrahman
On Mar 6, 2013, at 9:51 PM, Mohit Anchlia <mo...@gmail.com> wrote:
> I am wondering what the correct behaviour is of this parameter? If it's set to 4 does it mean job should fail if a job has more than 4 failures?
>
Re: mapred.max.tracker.failures
Posted by Harsh J <ha...@cloudera.com>.
It is a per-job config which controls the automatic job-level
blacklist: If, for a single job, a specific tracker has failed 4 (or
X) total tasks, then as prevent scheduling anymore of the job's tasks
to that tracker (but we don't eliminate more than 25% of the available
trackers this way, as for a bad logic job causing failures, that'd
make the job simply hang).
On Thu, Mar 7, 2013 at 11:21 AM, Mohit Anchlia <mo...@gmail.com> wrote:
> I am wondering what the correct behaviour is of this parameter? If it's set
> to 4 does it mean job should fail if a job has more than 4 failures?
--
Harsh J
Re: mapred.max.tracker.failures
Posted by Harsh J <ha...@cloudera.com>.
It is a per-job config which controls the automatic job-level
blacklist: If, for a single job, a specific tracker has failed 4 (or
X) total tasks, then as prevent scheduling anymore of the job's tasks
to that tracker (but we don't eliminate more than 25% of the available
trackers this way, as for a bad logic job causing failures, that'd
make the job simply hang).
On Thu, Mar 7, 2013 at 11:21 AM, Mohit Anchlia <mo...@gmail.com> wrote:
> I am wondering what the correct behaviour is of this parameter? If it's set
> to 4 does it mean job should fail if a job has more than 4 failures?
--
Harsh J
Re: mapred.max.tracker.failures
Posted by Mohit Anchlia <mo...@gmail.com>.
Thanks this is very helpful.
On Wed, Mar 6, 2013 at 10:03 PM, bharath vissapragada <
bharathvissapragada1990@gmail.com> wrote:
> No, its the number of task failures in a job after which that
> particular tasktracker can be blacklisted *for that job*! Note that it
> can take tasks from other jobs!
>
> On Thu, Mar 7, 2013 at 11:21 AM, Mohit Anchlia <mo...@gmail.com>
> wrote:
> > I am wondering what the correct behaviour is of this parameter? If
> it's set
> > to 4 does it mean job should fail if a job has more than 4 failures?
>
Re: mapred.max.tracker.failures
Posted by Mohit Anchlia <mo...@gmail.com>.
Thanks this is very helpful.
On Wed, Mar 6, 2013 at 10:03 PM, bharath vissapragada <
bharathvissapragada1990@gmail.com> wrote:
> No, its the number of task failures in a job after which that
> particular tasktracker can be blacklisted *for that job*! Note that it
> can take tasks from other jobs!
>
> On Thu, Mar 7, 2013 at 11:21 AM, Mohit Anchlia <mo...@gmail.com>
> wrote:
> > I am wondering what the correct behaviour is of this parameter? If
> it's set
> > to 4 does it mean job should fail if a job has more than 4 failures?
>
Re: mapred.max.tracker.failures
Posted by Mohit Anchlia <mo...@gmail.com>.
Thanks this is very helpful.
On Wed, Mar 6, 2013 at 10:03 PM, bharath vissapragada <
bharathvissapragada1990@gmail.com> wrote:
> No, its the number of task failures in a job after which that
> particular tasktracker can be blacklisted *for that job*! Note that it
> can take tasks from other jobs!
>
> On Thu, Mar 7, 2013 at 11:21 AM, Mohit Anchlia <mo...@gmail.com>
> wrote:
> > I am wondering what the correct behaviour is of this parameter? If
> it's set
> > to 4 does it mean job should fail if a job has more than 4 failures?
>
Re: mapred.max.tracker.failures
Posted by Mohit Anchlia <mo...@gmail.com>.
Thanks this is very helpful.
On Wed, Mar 6, 2013 at 10:03 PM, bharath vissapragada <
bharathvissapragada1990@gmail.com> wrote:
> No, its the number of task failures in a job after which that
> particular tasktracker can be blacklisted *for that job*! Note that it
> can take tasks from other jobs!
>
> On Thu, Mar 7, 2013 at 11:21 AM, Mohit Anchlia <mo...@gmail.com>
> wrote:
> > I am wondering what the correct behaviour is of this parameter? If
> it's set
> > to 4 does it mean job should fail if a job has more than 4 failures?
>
Re: mapred.max.tracker.failures
Posted by bharath vissapragada <bh...@gmail.com>.
No, its the number of task failures in a job after which that
particular tasktracker can be blacklisted *for that job*! Note that it
can take tasks from other jobs!
On Thu, Mar 7, 2013 at 11:21 AM, Mohit Anchlia <mo...@gmail.com> wrote:
> I am wondering what the correct behaviour is of this parameter? If it's set
> to 4 does it mean job should fail if a job has more than 4 failures?
Re: mapred.max.tracker.failures
Posted by Harsh J <ha...@cloudera.com>.
It is a per-job config which controls the automatic job-level
blacklist: If, for a single job, a specific tracker has failed 4 (or
X) total tasks, then as prevent scheduling anymore of the job's tasks
to that tracker (but we don't eliminate more than 25% of the available
trackers this way, as for a bad logic job causing failures, that'd
make the job simply hang).
On Thu, Mar 7, 2013 at 11:21 AM, Mohit Anchlia <mo...@gmail.com> wrote:
> I am wondering what the correct behaviour is of this parameter? If it's set
> to 4 does it mean job should fail if a job has more than 4 failures?
--
Harsh J
Re: mapred.max.tracker.failures
Posted by Harsh J <ha...@cloudera.com>.
It is a per-job config which controls the automatic job-level
blacklist: If, for a single job, a specific tracker has failed 4 (or
X) total tasks, then as prevent scheduling anymore of the job's tasks
to that tracker (but we don't eliminate more than 25% of the available
trackers this way, as for a bad logic job causing failures, that'd
make the job simply hang).
On Thu, Mar 7, 2013 at 11:21 AM, Mohit Anchlia <mo...@gmail.com> wrote:
> I am wondering what the correct behaviour is of this parameter? If it's set
> to 4 does it mean job should fail if a job has more than 4 failures?
--
Harsh J
Re: mapred.max.tracker.failures
Posted by bharath vissapragada <bh...@gmail.com>.
No, its the number of task failures in a job after which that
particular tasktracker can be blacklisted *for that job*! Note that it
can take tasks from other jobs!
On Thu, Mar 7, 2013 at 11:21 AM, Mohit Anchlia <mo...@gmail.com> wrote:
> I am wondering what the correct behaviour is of this parameter? If it's set
> to 4 does it mean job should fail if a job has more than 4 failures?
Re: mapred.max.tracker.failures
Posted by Abdelrhman Shettia <as...@hortonworks.com>.
Hi Mohit ,
This is The number of failed tasks in specified job after which the job will not run on the task tracker. The job's tasks will no longer be assigned to the task tracker. However, If the same task failed more than 4 times , The job will fail regardless.
Hope this helps.
Thanks
-Abdelrahman
On Mar 6, 2013, at 9:51 PM, Mohit Anchlia <mo...@gmail.com> wrote:
> I am wondering what the correct behaviour is of this parameter? If it's set to 4 does it mean job should fail if a job has more than 4 failures?
>
Re: mapred.max.tracker.failures
Posted by bharath vissapragada <bh...@gmail.com>.
No, its the number of task failures in a job after which that
particular tasktracker can be blacklisted *for that job*! Note that it
can take tasks from other jobs!
On Thu, Mar 7, 2013 at 11:21 AM, Mohit Anchlia <mo...@gmail.com> wrote:
> I am wondering what the correct behaviour is of this parameter? If it's set
> to 4 does it mean job should fail if a job has more than 4 failures?
Re: mapred.max.tracker.failures
Posted by Abdelrhman Shettia <as...@hortonworks.com>.
Hi Mohit ,
This is The number of failed tasks in specified job after which the job will not run on the task tracker. The job's tasks will no longer be assigned to the task tracker. However, If the same task failed more than 4 times , The job will fail regardless.
Hope this helps.
Thanks
-Abdelrahman
On Mar 6, 2013, at 9:51 PM, Mohit Anchlia <mo...@gmail.com> wrote:
> I am wondering what the correct behaviour is of this parameter? If it's set to 4 does it mean job should fail if a job has more than 4 failures?
>
Re: mapred.max.tracker.failures
Posted by bharath vissapragada <bh...@gmail.com>.
No, its the number of task failures in a job after which that
particular tasktracker can be blacklisted *for that job*! Note that it
can take tasks from other jobs!
On Thu, Mar 7, 2013 at 11:21 AM, Mohit Anchlia <mo...@gmail.com> wrote:
> I am wondering what the correct behaviour is of this parameter? If it's set
> to 4 does it mean job should fail if a job has more than 4 failures?
Re: mapred.max.tracker.failures
Posted by Abdelrhman Shettia <as...@hortonworks.com>.
Hi Mohit ,
This is The number of failed tasks in specified job after which the job will not run on the task tracker. The job's tasks will no longer be assigned to the task tracker. However, If the same task failed more than 4 times , The job will fail regardless.
Hope this helps.
Thanks
-Abdelrahman
On Mar 6, 2013, at 9:51 PM, Mohit Anchlia <mo...@gmail.com> wrote:
> I am wondering what the correct behaviour is of this parameter? If it's set to 4 does it mean job should fail if a job has more than 4 failures?
>