You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Rakesh Radhakrishnan (Jira)" <ji...@apache.org> on 2023/07/24 06:31:00 UTC
[jira] [Commented] (YARN-10975) EntityGroupFSTimelineStore#ActiveLogParser parses already processed files
[ https://issues.apache.org/jira/browse/YARN-10975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746217#comment-17746217 ]
Rakesh Radhakrishnan commented on YARN-10975:
---------------------------------------------
[~prabhujoseph] I have updated the fix version to 3.4.0 as this is merged only to the trunk.
> EntityGroupFSTimelineStore#ActiveLogParser parses already processed files
> --------------------------------------------------------------------------
>
> Key: YARN-10975
> URL: https://issues.apache.org/jira/browse/YARN-10975
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
> Affects Versions: 3.3.0
> Reporter: Prabhu Joseph
> Assignee: Ravuri Sushma sree
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: YARN-10975.001.patch, YARN-10975.002.patch, YARN-10975.003.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> EntityGroupFSTimelineStore#ActiveLogParser parses already processed files again and again even though there is no change in the file. This leads to unnecessary load on DFS where summary files resides and Timeline Store where timeline entities present.
> {code}
> 2021-10-10 19:20:43,940 INFO timeline.LogInfo - Parsed 6 entities from hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_000001/summarylog-appattempt_1631559260564_0009_000001_2331123893 in 275 msec
> 2021-10-10 19:21:44,079 INFO timeline.LogInfo - Parsed 6 entities from hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_000001/summarylog-appattempt_1631559260564_0009_000001_2331123893 in 341 msec
> 2021-10-10 19:22:44,065 INFO timeline.LogInfo - Parsed 6 entities from hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_000001/summarylog-appattempt_1631559260564_0009_000001_2331123893 in 335 msec
> 2021-10-10 19:23:44,038 INFO timeline.LogInfo - Parsed 6 entities from hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_000001/summarylog-appattempt_1631559260564_0009_000001_2331123893 in 370 msec
> 2021-10-10 19:24:44,087 INFO timeline.LogInfo - Parsed 6 entities from hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_000001/summarylog-appattempt_1631559260564_0009_000001_2331123893 in 317 msec
> 2021-10-10 19:25:44,092 INFO timeline.LogInfo - Parsed 6 entities from hdfs:/prabhuJoseph/atshistory/active/application_1631559260564_0009/appattempt_1631559260564_0009_000001/summarylog-appattempt_1631559260564_0009_000001_2331123893 in 336 msec
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org