You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@metamodel.apache.org by "Tomasz Guzialek (JIRA)" <ji...@apache.org> on 2015/08/21 12:49:45 UTC

[jira] [Created] (METAMODEL-179) FileSystem instance in HdfsResource should not be closed

Tomasz Guzialek created METAMODEL-179:
-----------------------------------------

             Summary: FileSystem instance in HdfsResource should not be closed
                 Key: METAMODEL-179
                 URL: https://issues.apache.org/jira/browse/METAMODEL-179
             Project: Apache MetaModel
          Issue Type: Bug
            Reporter: Tomasz Guzialek
            Priority: Critical


In HdfsResource, a FileSystem instance is obtained via get() method. According to [1] this method does not return a new instance every time, but caches it for the same configuration. Closing such instance may result in IOException: Filesystem closed in other, possibly unrelated, places in the code also using this FileSystem instance.

Possible solutions: 
1) Despite a Java convention to always close Closable objects after use, we should leave without closing. Apparently, Hive does it this way (see [1])
2) Create instances of FileSystem with newInstance() method, but this is an expensive operation.

[1] http://stackoverflow.com/questions/20057881/hadoop-filesystem-closed-exception-when-doing-bufferedreader-close/20061797#20061797



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)