You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Owen O'Malley (JIRA)" <ji...@apache.org> on 2007/09/14 20:50:32 UTC

[jira] Created: (HADOOP-1899) the metrics system in the job tracker is running too often

the metrics system in the job tracker is running too often
----------------------------------------------------------

                 Key: HADOOP-1899
                 URL: https://issues.apache.org/jira/browse/HADOOP-1899
             Project: Hadoop
          Issue Type: Improvement
          Components: metrics
            Reporter: Owen O'Malley
            Assignee: Owen O'Malley
             Fix For: 0.15.0


The metrics system in the JobTracker is defaulting to every 5 seconds computing all of the counters for all of the jobs. This work is a substantial amount of work showing up as running in 20% of the snapshots that I've seen. I'd like to lower the default interval to once every 60 seconds and make it a low priority thread.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-1899) the metrics system in the job tracker is running too often

Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/HADOOP-1899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12527617 ] 

Sameer Paranjpye commented on HADOOP-1899:
------------------------------------------

Could this be somehow optimized by ignoring tasks that have succeeded and whose counters are already rolled up, and doing deltas for the ones in progress? It'll make managing individual task metrics more complex, but might be a win overall.

 

> the metrics system in the job tracker is running too often
> ----------------------------------------------------------
>
>                 Key: HADOOP-1899
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1899
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: metrics
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.15.0
>
>
> The metrics system in the JobTracker is defaulting to every 5 seconds computing all of the counters for all of the jobs. This work is a substantial amount of work showing up as running in 20% of the snapshots that I've seen. I'd like to lower the default interval to once every 60 seconds and make it a low priority thread.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.