You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Wei-Chiu Chuang (JIRA)" <ji...@apache.org> on 2018/04/17 15:35:00 UTC
[jira] [Comment Edited] (HBASE-20433) HBase Export Snapshot utility
does not close FileSystem instances
[ https://issues.apache.org/jira/browse/HBASE-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441029#comment-16441029 ]
Wei-Chiu Chuang edited comment on HBASE-20433 at 4/17/18 3:34 PM:
------------------------------------------------------------------
Hi Voyta,
The reason for not caching FileSystem object is explained in HADOOP-13971. The internal design causes memory leak and has resulted in quite a few tickets in other projects.
Yes, therefore the responsibility falls on to the application (like HBase) to manage the FileSystem object & close when appropriate.
Do you have a heapdump or profiler that shows fs object occupies memory in your ExportSnapshot?
was (Author: jojochuang):
Hi Voyta,
The reason for not caching FileSystem object is explained in HADOOP-13971. The internal design causes memory leak and has resulted in quite a few tickets in other projects.
Yes, therefore the responsibility falls on to the application (like HBase) to manage the FileSystem object & close when appropriate.
> HBase Export Snapshot utility does not close FileSystem instances
> -----------------------------------------------------------------
>
> Key: HBASE-20433
> URL: https://issues.apache.org/jira/browse/HBASE-20433
> Project: HBase
> Issue Type: Bug
> Components: Client, fs, snapshots
> Affects Versions: 1.2.6, 1.4.3
> Reporter: Voyta
> Priority: Major
>
> It seems org.apache.hadoop.hbase.snapshot.ExportSnapshot disallows FileSystem instance caching.
> When verifySnapshot method is being run it calls often methods like org.apache.hadoop.hbase.util.FSUtils#getRootDir that instantiate FileSystem but never calls org.apache.hadoop.fs.FileSystem#close method. This behaviour allows allocation of unwanted objects potentially causing memory leaks.
> Related issue: https://issues.apache.org/jira/browse/HADOOP-15392
>
> Expectation:
> * HBase should properly release/close all objects, especially FileSystem instances.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)