[jira] [Commented] (YARN-8617) Aggregated Application Logs accumulates for long running jobs

["Tarun Parimi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-8617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790508#comment-16790508 ] 

Tarun Parimi commented on YARN-8617:
------------------------------------

Hi [~bibinchundatt],

I was also facing this issue and on testing in my local cluster I observed the follows:

{quote}1. limit number of files per node
public static final String NM_LOG_AGGREGATION_NUM_LOG_FILES_SIZE_PER_APP
= NM_PREFIX + "log-aggregation.num-log-files-per-app";{quote}
This doesn't seem to work currently for IndexedFileFormat. After the file exceeds LOG_ROLL_OVER_MAX_FILE_SIZE_GB, a new file is created. But the older node files can keep on accumulating as long as the app is running. Should we implement this config for IndexedFileFormat also as a fix?

{quote}For long running service the application folder eg :user/logs/application_1234 modification time gets updated on every upload cycle.
This could cause nodefile to remain in hdfs if no new containers are allocated to same node.{quote}
Should we check and delete nodefiles in AggrgeatedLogDeletionService  for RUNNING apps without the checking the condition appDir.getModificationTime() < cutoffMillis ? 
Doing so will delete the older node files and fix the problem of old node files getting accumulated.



> Aggregated Application Logs accumulates for long running jobs
> -------------------------------------------------------------
>
>                 Key: YARN-8617
>                 URL: https://issues.apache.org/jira/browse/YARN-8617
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: log-aggregation
>    Affects Versions: 2.7.4
>            Reporter: Prabhu Joseph
>            Priority: Major
>
> Currently AggregationDeletionService will delete older aggregated log files once when they are complete. This will cause logs to accumulate for Long Running Jobs like Llap, Spark Streaming.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org