[jira] [Commented] (TEZ-3611) Create lightweight summary events for ATS.

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-3611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15898339#comment-15898339 ] 

Siddharth Seth commented on TEZ-3611:
-------------------------------------

Minor comments on the patch.
- Should we have HistoryEventTimelineConversion return a list from convertToTimelineEntity - instead of exposing summary vs non-summary events in this class?
- Nothing invokes convertToTimelineEntity with a value of false? (That was for backward compatibility of the UI I believe?)

Question: This puts DAG_EXTRA_INFO into the regular details file, correct? Does it make sense to put DAG_EXTRA_INFO into the summary as well, (except not part of DAG_ID - This is really just to get past the inability to retrieve specific fields from ATS).

Other than this, changes look good to me. 

Think we need to wait for the UI part of the change to be in place before committing this?

Unrelated question: I didn't realize there's additional information like Vertex list (without task count) logged as part of TEZ_DAG_ID. Assuming all the events - recovery, dag initialized, etc - all pubish to ATS summry, since they all update TEZ_DAG_ID?

> Create lightweight summary events for ATS.
> ------------------------------------------
>
>                 Key: TEZ-3611
>                 URL: https://issues.apache.org/jira/browse/TEZ-3611
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Harish Jaiprakash
>            Assignee: Harish Jaiprakash
>         Attachments: TEZ-3611.01.patch, TEZ-3611.02.patch, TEZ-3611.03.patch
>
>
> All DAG*Event are summary logs. DAGSubmittedEvent has dagPlan in it and DAGFinishedEvent has counters in it. And hence these events very large, which makes the dag landing page on the tez-ui very expensive.
> Split these events into two parts:
> * One which contains only data used by the tez-ui "All Dags" page which will go into summary log.
> * Another which will contain the current object, which will be part of the appropriate TimelineEntityGroupId.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)