[jira] [Commented] (TEZ-3609) Improve ATSv15 performance for DAG entities read calls.

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15857312#comment-15857312 ] 

Siddharth Seth commented on TEZ-3609:
-------------------------------------

Looks good to me. I believe ATS uses this to determine where to look for data. Since all DAG data is in a single file - we should be ok not needing the App file for DAG_ID, VERTEX_ID, TASK_ID, TASK_ATTEMPT_ID

Question: Does this look correct primaryFilter.getName, primaryFilter.getValue as parameters - and the entityType is ignored?
{code}
  public Set<TimelineEntityGroupId> getTimelineEntityGroupId(String entityType,
      NameValuePair primaryFilter,
      Collection<NameValuePair> secondaryFilters) {
    if (!knownEntityTypes.contains(entityType)
        || primaryFilter == null
        || !knownEntityTypes.contains(primaryFilter.getName())
        || summaryEntityTypes.contains(entityType)) {
      return null;
    }
    return convertToTimelineEntityGroupIds(primaryFilter.getName(), primaryFilter.getValue().toString());
  }
{code}

> Improve ATSv15 performance for DAG entities read calls.
> -------------------------------------------------------
>
>                 Key: TEZ-3609
>                 URL: https://issues.apache.org/jira/browse/TEZ-3609
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Harish Jaiprakash
>            Assignee: Harish Jaiprakash
>         Attachments: TEZ-3609.01.patch
>
>
> TimelineCachePluginImpl generates TimelineEntityGroupId.newInstance(appId, appId.toString()) as an EntityGroupId for dag entities. This results entities in file based on application EntityGroupId to loaded by ATS during a read operation. This file can become very large in long running AM and causes timeout and delays in ATS.
> Removing this id, reduces this parsing time and prevents the application entities from being loaded into cache.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)