[jira] [Commented] (TEZ-1652) Large job support for Swimlane view

["Jonathan Eagles (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14167515#comment-14167515 ] 

Jonathan Eagles commented on TEZ-1652:
--------------------------------------

[~pramachandran], this is the first part to getting swimlanes to work with 10,000 task attempts. First issue is that only 100 entries are returned. Second issue is that loading 10,000 tasks is slow.

This patch addresses the first issues. It reads the counters to determine number of task attempts to return, defaulting to a very large number is the total tasks cant be found. Please review and suggest code improvements. Should this logic be moved into the controller as part of a getQueryParams method?

The 2 part of why 10,000 tasks is slow is due to Task attempt serialization as part of counter normalization. Do you want to look at improving counter normalization speed?

> Large job support for Swimlane view
> -----------------------------------
>
>                 Key: TEZ-1652
>                 URL: https://issues.apache.org/jira/browse/TEZ-1652
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Jonathan Eagles
>            Assignee: Jonathan Eagles
>         Attachments: TEZ-1652-v1.patch
>
>
> Issue is that by default, Timeline Server only returns 100 max entity results per query. For query for all task attempts for dag. A limit should be high enough to get all entries and yet provide responsiveness.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)