You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Hemanth Yamijala (JIRA)" <ji...@apache.org> on 2009/10/13 08:14:31 UTC
[jira] Created: (MAPREDUCE-1099) Setup and cleanup tasks could
affect job latency if they are caught running on bad nodes
Setup and cleanup tasks could affect job latency if they are caught running on bad nodes
----------------------------------------------------------------------------------------
Key: MAPREDUCE-1099
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1099
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Affects Versions: 0.20.1
Reporter: Hemanth Yamijala
We found cases on our clusters where a setup task got scheduled on a bad node and took upwards of several minutes to run, adversely affecting job runtimes. Speculation did not help here as speculation is not used for setup tasks. I suspect the same could happen for cleanup tasks as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1099) Setup and cleanup tasks could
affect job latency if they are caught running on bad nodes
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12764968#action_12764968 ]
Hemanth Yamijala commented on MAPREDUCE-1099:
---------------------------------------------
Note that I am not suggesting we speculate setup and cleanup tasks. In an offline discussion with Amareshwari and Devaraj we realized there are potential difficulties in speculating these attempts. Firstly there's no good way to measure the progress of these attempts against other attempts. I am just opening this issue to open for discussion.
> Setup and cleanup tasks could affect job latency if they are caught running on bad nodes
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1099
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1099
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Hemanth Yamijala
>
> We found cases on our clusters where a setup task got scheduled on a bad node and took upwards of several minutes to run, adversely affecting job runtimes. Speculation did not help here as speculation is not used for setup tasks. I suspect the same could happen for cleanup tasks as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1099) Setup and cleanup tasks could
affect job latency if they are caught running on bad nodes
Posted by "ZhuGuanyin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784067#action_12784067 ]
ZhuGuanyin commented on MAPREDUCE-1099:
---------------------------------------
We have encountered the same problem, so We just remove the setup and cleanup task (inport the patch https://issues.apache.org/jira/browse/MAPREDUCE-463 )
> Setup and cleanup tasks could affect job latency if they are caught running on bad nodes
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1099
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1099
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.1
> Reporter: Hemanth Yamijala
>
> We found cases on our clusters where a setup task got scheduled on a bad node and took upwards of several minutes to run, adversely affecting job runtimes. Speculation did not help here as speculation is not used for setup tasks. I suspect the same could happen for cleanup tasks as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.