You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "zlzhang0122 (Jira)" <ji...@apache.org> on 2021/09/02 03:20:00 UTC

[jira] [Created] (FLINK-24122) Add support to do clean in history server

zlzhang0122 created FLINK-24122:
-----------------------------------

             Summary: Add support to do clean in history server
                 Key: FLINK-24122
                 URL: https://issues.apache.org/jira/browse/FLINK-24122
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / REST
    Affects Versions: 1.13.2, 1.12.3
            Reporter: zlzhang0122
             Fix For: 1.14.1


Now, the history server can clean history jobs by two means:
 # if users have configured 
{code:java}
historyserver.web.ssl.enabled: true{code}
, then compare the files in hdfs over two clean interval and find the delete and clean the local cache file.
 # if users have configured the 
{code:java}
historyserver.archive.retained-jobs{code}
a positive number, then clean the oldest files in hdfs and local.

But it can't clean the job history which was no longer in hdfs but still cached in local filesystem and these files will store forever and can't be cleaned unless users manually do this. Maybe we can give a option and do this clean if the option says true.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)