You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Dhanasekaran Anbalagan <bu...@gmail.com> on 2013/01/28 13:16:31 UTC
jobTracker Blacklisted Nodes
Hi Guys,
Difference between Job tracker cluster level blacklisted nodes and job
level blacklist node.
In my case My jobtacker page says no Blacklisted nodes,
When i open job link it's says Blacklist node entry.
please guide me how to understand.
-Dhanasekaran.
Did I learn something today? If not, I wasted it.
Re: jobTracker Blacklisted Nodes
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Overall tasks can fail because of the job itself (bugs or erroneous condition in job's/user's code) or issues on the node (hardware/system software).
Job level blacklisting accounts for the first type of problems i.e it avoids nodes on which many tasks of this particular job failed for whatever reason. It is used to make sure that more tasks of this job don't get launched on the same node.
Cluster level blacklisting accounts for cumulative failures on nodes across all jobs. Used to heuristically determine generic node issues independent of the jobs.
Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/
On Jan 28, 2013, at 4:16 AM, Dhanasekaran Anbalagan wrote:
> Hi Guys,
>
> Difference between Job tracker cluster level blacklisted nodes and job level blacklist node.
>
> In my case My jobtacker page says no Blacklisted nodes,
> When i open job link it's says Blacklist node entry.
>
> please guide me how to understand.
>
> -Dhanasekaran.
> Did I learn something today? If not, I wasted it.
Re: jobTracker Blacklisted Nodes
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Overall tasks can fail because of the job itself (bugs or erroneous condition in job's/user's code) or issues on the node (hardware/system software).
Job level blacklisting accounts for the first type of problems i.e it avoids nodes on which many tasks of this particular job failed for whatever reason. It is used to make sure that more tasks of this job don't get launched on the same node.
Cluster level blacklisting accounts for cumulative failures on nodes across all jobs. Used to heuristically determine generic node issues independent of the jobs.
Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/
On Jan 28, 2013, at 4:16 AM, Dhanasekaran Anbalagan wrote:
> Hi Guys,
>
> Difference between Job tracker cluster level blacklisted nodes and job level blacklist node.
>
> In my case My jobtacker page says no Blacklisted nodes,
> When i open job link it's says Blacklist node entry.
>
> please guide me how to understand.
>
> -Dhanasekaran.
> Did I learn something today? If not, I wasted it.
Re: jobTracker Blacklisted Nodes
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Overall tasks can fail because of the job itself (bugs or erroneous condition in job's/user's code) or issues on the node (hardware/system software).
Job level blacklisting accounts for the first type of problems i.e it avoids nodes on which many tasks of this particular job failed for whatever reason. It is used to make sure that more tasks of this job don't get launched on the same node.
Cluster level blacklisting accounts for cumulative failures on nodes across all jobs. Used to heuristically determine generic node issues independent of the jobs.
Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/
On Jan 28, 2013, at 4:16 AM, Dhanasekaran Anbalagan wrote:
> Hi Guys,
>
> Difference between Job tracker cluster level blacklisted nodes and job level blacklist node.
>
> In my case My jobtacker page says no Blacklisted nodes,
> When i open job link it's says Blacklist node entry.
>
> please guide me how to understand.
>
> -Dhanasekaran.
> Did I learn something today? If not, I wasted it.
Re: jobTracker Blacklisted Nodes
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Overall tasks can fail because of the job itself (bugs or erroneous condition in job's/user's code) or issues on the node (hardware/system software).
Job level blacklisting accounts for the first type of problems i.e it avoids nodes on which many tasks of this particular job failed for whatever reason. It is used to make sure that more tasks of this job don't get launched on the same node.
Cluster level blacklisting accounts for cumulative failures on nodes across all jobs. Used to heuristically determine generic node issues independent of the jobs.
Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/
On Jan 28, 2013, at 4:16 AM, Dhanasekaran Anbalagan wrote:
> Hi Guys,
>
> Difference between Job tracker cluster level blacklisted nodes and job level blacklist node.
>
> In my case My jobtacker page says no Blacklisted nodes,
> When i open job link it's says Blacklist node entry.
>
> please guide me how to understand.
>
> -Dhanasekaran.
> Did I learn something today? If not, I wasted it.