You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Akash (Jira)" <ji...@apache.org> on 2022/06/03 07:49:00 UTC
[jira] [Updated] (YARN-11170) EntityGroupFSTimelineStore cleaner thread stops if it fails with error
[ https://issues.apache.org/jira/browse/YARN-11170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Akash updated YARN-11170:
-------------------------
Description:
timeline server v1.5 hdfs data cleanup thread stops cleaning when it finds a file missing
{code:java}
2022-06-01 08:00:25,861 ERROR org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore: Error cleaning files
java.io.FileNotFoundException: File hdfs://<namenode-nameservice>/ats-cluster2/entity-file-history/done/1646401932991/0000/650/application_1646401932991_650010 does not exist.
at org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:1144)
{code}
We found that older files are present
{code:java}
drwxrwxr-x - hadoop supergroup 0 2022-03-16 01:40 hdfs://<namenode-nameservice>/ats-cluster2/entity-file-history/done/1646401932991
drwxrwxr-x - hadoop supergroup 0 2022-05-17 19:43 hdfs://<namenode-nameservice>/ats-cluster2/entity-file-history/done/1648056692622
{code}
our configs are
{code:java}
yarn.timeline-service.ttl-enable=ture
yarn.timeline-service.client.internal-timers-ttl-secs=420
yarn.timeline-service.ttl-ms=432000000{code}
This thread should ignore the error and should continue cleaning up further directories
was:
timeline server v1.5 hdfs data cleanup thread stops cleaning when it finds a file missing
```2022-06-01 08:00:25,861 ERROR org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore: Error cleaning files
java.io.FileNotFoundException: File hdfs://<namenode-nameservice>/ats-cluster2/entity-file-history/done/1646401932991/0000/650/application_1646401932991_650010 does not exist.
at org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:1144)```
We found that older files are present
```
drwxrwxr-x - hadoop supergroup 0 2022-03-16 01:40 hdfs://<namenode-nameservice>/ats-cluster2/entity-file-history/done/1646401932991
drwxrwxr-x - hadoop supergroup 0 2022-05-17 19:43 hdfs://<namenode-nameservice>/ats-cluster2/entity-file-history/done/1648056692622```
our configs are
yarn.timeline-service.ttl-enable=ture
yarn.timeline-service.client.internal-timers-ttl-secs=420
yarn.timeline-service.ttl-ms=432000000
This thread should ignore the error and should continue cleaning up further directories
> EntityGroupFSTimelineStore cleaner thread stops if it fails with error
> -----------------------------------------------------------------------
>
> Key: YARN-11170
> URL: https://issues.apache.org/jira/browse/YARN-11170
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
> Affects Versions: 2.10.0
> Reporter: Akash
> Priority: Major
>
> timeline server v1.5 hdfs data cleanup thread stops cleaning when it finds a file missing
>
> {code:java}
> 2022-06-01 08:00:25,861 ERROR org.apache.hadoop.yarn.server.timeline.EntityGroupFSTimelineStore: Error cleaning files
> java.io.FileNotFoundException: File hdfs://<namenode-nameservice>/ats-cluster2/entity-file-history/done/1646401932991/0000/650/application_1646401932991_650010 does not exist.
> at org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:1144)
> {code}
>
> We found that older files are present
>
> {code:java}
> drwxrwxr-x - hadoop supergroup 0 2022-03-16 01:40 hdfs://<namenode-nameservice>/ats-cluster2/entity-file-history/done/1646401932991
> drwxrwxr-x - hadoop supergroup 0 2022-05-17 19:43 hdfs://<namenode-nameservice>/ats-cluster2/entity-file-history/done/1648056692622
> {code}
> our configs are
> {code:java}
> yarn.timeline-service.ttl-enable=ture
> yarn.timeline-service.client.internal-timers-ttl-secs=420
> yarn.timeline-service.ttl-ms=432000000{code}
> This thread should ignore the error and should continue cleaning up further directories
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org