You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Amareshwari Sri Ramadasu (JIRA)" <ji...@apache.org> on 2007/10/18 12:07:51 UTC
[jira] Issue Comment Edited: (HADOOP-1900) the heartbeat and task
event queries interval should be set dynamically by the JobTracker
[ https://issues.apache.org/jira/browse/HADOOP-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12535840 ]
amareshwari edited comment on HADOOP-1900 at 10/18/07 3:07 AM:
----------------------------------------------------------------------------
bq. I wonder if instead we should just make it clusterSize/50+1? That way, small clusters will get a heartbeat of just one second, which should make them more responsive.
Agreed.
bq. Now, MapEventsFetcherThread polls jobtracker for completed map tasks for every 5 secs (MIN_POLL_INTERVAL). Shall we change polling interval also in the similar fashion as heartbeat interval? But, here some reduce tasks could be idle for longer time.
We can have MapEventsFetcherThread polling jobtracker for completed map tasks in the similar fashion as heartbeat interval with MIN_POLL_INTERVAL=5secs and MAX_POLL_INTERVAL=30secs. And Whenever tasktracker finds there are no mapevents and reduce task is waiting for the map events, it will wakeup the thread to fetch map events from job tracker.
was (Author: amareshwari):
bq. I wonder if instead we should just make it clusterSize/50+1? That way, small clusters will get a heartbeat of just one second, which should make them more responsive.
Agreed.
bq. Now, MapEventsFetcherThread polls jobtracker for completed map tasks for every 5 secs (MIN_POLL_INTERVAL). Shall we change polling interval also in the similar fashion as heartbeat interval? But, here some reduce tasks could be idle for longer time.
We can have MapEventsFetcherThread polling jobtracker for completed map tasks in the similar fashion as heartbeat interval with MIN_POLL_INTERVAL=5secs and MIN_POLL_INTERVAL_MAX=30secs. And Whenever tasktracker finds there are no mapevents and reduce task is waiting for the map events, it will wakeup the thread to fetch map events from job tracker.
> the heartbeat and task event queries interval should be set dynamically by the JobTracker
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-1900
> URL: https://issues.apache.org/jira/browse/HADOOP-1900
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
>
> The JobTracker should scale the intervals that the TaskTrackers use to contact it dynamically, based on how the busy it is and the size of the cluster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.