You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-issues@hadoop.apache.org by "Ray Chiang (JIRA)" <ji...@apache.org> on 2015/06/11 00:27:00 UTC

[jira] [Commented] (MAPREDUCE-6394) Speed up Task processing loop in HsTasksBlock#render()

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14581158#comment-14581158 ] 

Ray Chiang commented on MAPREDUCE-6394:
---------------------------------------

In doing some profiling, it looks as though the bulk of the runtime occurs copying the YARN counters into the TaskReport.  Since this value isn't used in the web display for the tasks page, turn it off for webapp rendering.

For 404k map tasks, this reduces the loop from about 50 seconds down to about 5 seconds.

> Speed up Task processing loop in HsTasksBlock#render()
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-6394
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6394
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: jobhistoryserver
>    Affects Versions: 2.7.0
>            Reporter: Ray Chiang
>            Assignee: Ray Chiang
>              Labels: supportability
>         Attachments: MAPREDUCE-6394.001.patch
>
>
> In HsTasksBlock#render(), there is a loop to create a Javascript table which slows down immensely on jobs with a large number of tasks (200k or more).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)