You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Xiaomin Zhang (Jira)" <ji...@apache.org> on 2021/07/16 05:55:00 UTC
[jira] [Commented] (YARN-6221) Entities missing from ATS when
summary log file info got returned to the ATS before the domain log
[ https://issues.apache.org/jira/browse/YARN-6221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17381788#comment-17381788 ]
Xiaomin Zhang commented on YARN-6221:
-------------------------------------
Greetings, [~gtcarrera9]. May I know if you are still working on this jira?
We observed this issue when running Isilon. For some reasons, getListing output is in fixed order from HDFS, but it's not true for Isilon. For most cases, domain log is not listed as the first from Isilon's getListing call. And when this happens, domain log will be after other summary files to be parsed by ATS.
We have a patch to make sure the domain log be parsed before others, which is supposed to fix this issue. Can you please help assign this Jira so I can upload the patch? Thanks.
> Entities missing from ATS when summary log file info got returned to the ATS before the domain log
> --------------------------------------------------------------------------------------------------
>
> Key: YARN-6221
> URL: https://issues.apache.org/jira/browse/YARN-6221
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Reporter: Sushmitha Sreenivasan
> Assignee: Li Lu
> Priority: Critical
>
> Events data missing for the following entities:
> curl -k --negotiate -u: http://<timeline_host>:8188/ws/v1/timeline/TEZ_APPLICATION_ATTEMPT/tez_appattempt_1487706062210_0012_000001
> {"events":[],"entitytype":"TEZ_APPLICATION_ATTEMPT","entity":"tez_appattempt_1487706062210_0012_000001","starttime":1487711606077,"domain":"Tez_ATS_application_1487706062210_0012","relatedentities":{"TEZ_DAG_ID":["dag_1487706062210_0012_2","dag_1487706062210_0012_1"]},"primaryfilters":{},"otherinfo":{}}
> {code:title=Timeline Server log entry}
> WARN timeline.TimelineDataManager (TimelineDataManager.java:doPostEntities(366)) - Skip the timeline entity: { id: tez_application_1487706062210_0012, type: TEZ_APPLICATION }
> org.apache.hadoop.yarn.exceptions.YarnException: Domain information of the timeline entity { id: tez_application_1487706062210_0012, type: TEZ_APPLICATION } doesn't exist.
> at org.apache.hadoop.yarn.server.timeline.security.TimelineACLsManager.checkAccess(TimelineACLsManager.java:122)
> at org.apache.hadoop.yarn.server.timeline.TimelineDataManager.doPostEntities(TimelineDataManager.java:356)
> at org.apache.hadoop.yarn.server.timeline.TimelineDataManager.postEntities(TimelineDataManager.java:316)
> at org.apache.hadoop.yarn.server.timeline.EntityLogInfo.doParse(LogInfo.java:204)
> at org.apache.hadoop.yarn.server.timeline.LogInfo.parsePath(LogInfo.java:156)
> at org.apache.hadoop.yarn.server.timeline.LogInfo.parseForStore(LogInfo.java:113)
> at org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:682)
> at org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$AppLogs.parseSummaryLogs(EntityGroupFSTimelineStore.java:657)
> at org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore$ActiveLogParser.run(EntityGroupFSTimelineStore.java:870)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org