[jira] [Created] (FLINK-9626) Possible resource leak in FileSystem

["Piotr Nowojski (JIRA)" <ji...@apache.org>]

Piotr Nowojski created FLINK-9626:
-------------------------------------

             Summary: Possible resource leak in FileSystem
                 Key: FLINK-9626
                 URL: https://issues.apache.org/jira/browse/FLINK-9626
             Project: Flink
          Issue Type: Bug
          Components: FileSystem
    Affects Versions: 1.5.0
            Reporter: Piotr Nowojski


There is a potential resource leak in org.apache.flink.core.fs.FileSystem#getUnguardedFileSystem.

Inside it there is a code:

 
{code:java}
// this "default" initialization makes sure that the FileSystem class works
// even when not configured with an explicit Flink configuration, like on
// JobManager or TaskManager setup
if (FS_FACTORIES.isEmpty()) {
   initialize(new Configuration());
}

{code}
which is executed on each cache miss. However this initialize method is also doing

 

 
{code:java}
CACHE.clear();
{code}
without closing file systems in CACHE (this could be problematic for HadoopFileSystem which is a wrapper around org.apache.hadoop.fs.FileSystem which is closable).

Now if for example we are constantly accessing two file systems initialized from FALLBACK_FACTORY, each time we call getUnguardedFileSystem for one of them, that call will clear from CACHE entry for the other one. Thus we will constantly be creating new FileSystems without closing them.

Solution could be to either not clear the CACHE or make sure that FileSystems are properly closed.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)