You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "zlzhang0122 (Jira)" <ji...@apache.org> on 2021/08/04 13:03:00 UTC

[jira] [Created] (FLINK-23622) Default filesystem scheme improvement

zlzhang0122 created FLINK-23622:
-----------------------------------

             Summary: Default filesystem scheme improvement
                 Key: FLINK-23622
                 URL: https://issues.apache.org/jira/browse/FLINK-23622
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Checkpointing
    Affects Versions: 1.13.1, 1.12.2
            Reporter: zlzhang0122
             Fix For: 1.14.0


Now, when flink try to get the filesystem scheme(when doing checkpoint, savepoint and so on), it will according to the steps as following:

1.try to get it from the given path, if it is null, then 2

2.try to read the flink configuration of "fs.default-scheme", if it is null, then 3

3.default to LocalFileSystem

Since flink is mostly running on hadoop, and I think that maybe we should also try to get the scheme from hadoop default configuration file core-site.xml and load thevalue of the  key "fs.defaultFS". This step can be added between step 2 and 3.

So after this change, the steps should like this:

1.try to get it from the given path, if it is null, then 2

2.try to read the flink configuration of "fs.default-scheme", if it is null, then 3

3.try to read the hadoop configuration of "fs.defaultFS", if it is null, then 4

4.default to LocalFileSystem.

I think that it also doesn't break any kind of consistent of flink and can simplify the user behavior.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)