You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Pete Wyckoff (JIRA)" <ji...@apache.org> on 2008/10/04 02:43:44 UTC

[jira] Created: (HADOOP-4341) fuse-dfs leaks FileSystem handles as it never disconnects them because the FileSystem.Cache does not do reference counting

fuse-dfs leaks FileSystem handles as it never disconnects them because the FileSystem.Cache does not do reference counting
--------------------------------------------------------------------------------------------------------------------------

                 Key: HADOOP-4341
                 URL: https://issues.apache.org/jira/browse/HADOOP-4341
             Project: Hadoop Core
          Issue Type: Bug
          Components: contrib/fuse-dfs, fs
    Affects Versions: 0.19.0
            Reporter: Pete Wyckoff
            Priority: Minor


since users may be doing multiple file operations at the same time, a single task in fuse, can never call close() on a filesystem (ie libhdfs::hdfsDisconnect) because there may be another thread for the same user.

as such, either fuse-dfs needs to do reference counting or FileSystem.Cache needs to or maybe enable a mode where one can turn off the Cache??

I currently am not seeing any problems in production, but I am still running 0.18 version which keeps only one connection as root.


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (HADOOP-4341) fuse-dfs leaks FileSystem handles as it never disconnects them because the FileSystem.Cache does not do reference counting

Posted by "Craig Macdonald (JIRA)" <ji...@apache.org>.

    [ https://issues.apache.org/jira/browse/HADOOP-4341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637664#action_12637664 ] 

Craig Macdonald commented on HADOOP-4341:
-----------------------------------------

Reference counting or a GC pthread, which calls close() on filesystems that havent been used for a while?

C

> fuse-dfs leaks FileSystem handles as it never disconnects them because the FileSystem.Cache does not do reference counting
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-4341
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4341
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/fuse-dfs, fs
>    Affects Versions: 0.19.0
>            Reporter: Pete Wyckoff
>            Priority: Minor
>
> since users may be doing multiple file operations at the same time, a single task in fuse, can never call close() on a filesystem (ie libhdfs::hdfsDisconnect) because there may be another thread for the same user.
> as such, either fuse-dfs needs to do reference counting or FileSystem.Cache needs to or maybe enable a mode where one can turn off the Cache??
> I currently am not seeing any problems in production, but I am still running 0.18 version which keeps only one connection as root.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.