[jira] [Resolved] (HADOOP-12707) key of FileSystem inner class Cache contains UGI.hascode which uses the defualt hascode method, leading to the memory leak

["Chris Nauroth (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-12707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Nauroth resolved HADOOP-12707.
------------------------------------
    Resolution: Won't Fix

Hello, [~haitao-tony].  Thank you for filing the issue, but the current implementation of {{UserGroupInformation#hashCode}} and {{UserGroupInformation#equals}} is by design.  Please see issue HADOOP-6670 if you're interested in more background.

I have 2 suggestions to offer for the memory leak problem that you noticed:
# Consider using the {{FileSystem#closeAllForUGI}} method, which would close all cached instances for a user.
# If it's not possible to change code, then consider disabling the cache by setting configuration property {{fs.hdfs.impl.disable.cache}} to {{true}} in core-site.xml.  This would bypass the cache completely, which might be appropriate depending on your application.

> key of FileSystem inner class Cache contains UGI.hascode which uses the defualt hascode method, leading to the memory leak
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-12707
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12707
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.7.1
>            Reporter: sunhaitao
>            Assignee: sunhaitao
>
> FileSystem.get(conf) method,By default it will get the fs object from CACHE,But the key of the CACHE  constains ugi.hashCode, which uses the default hascode method of subject instead of the hascode method overwritten by subject.
>    @Override
>       public int hashCode() {
>         return (scheme + authority).hashCode() + ugi.hashCode() + (int)unique;
>       }
> In this case, even if same user, if the calll FileSystem.get(conf) twice, two different key will be created. In long duartion, this will lead to memory leak.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)