[jira] [Comment Edited] (HBASE-21959) CompactionTool should close the store it uses for compacting files, in order to properly archive compacted files.

["Wellington Chevreuil (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-21959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16783213#comment-16783213 ] 

Wellington Chevreuil edited comment on HBASE-21959 at 3/4/19 10:23 AM:
-----------------------------------------------------------------------

Thanks for reviewing this, [~xucang]! Sorry for the delay in responding, as I was on a short break for the last 4 days.

So, both *close()* and *closeAndArchiveCompactedFiles()* will rely on private method *HStore.removeCompactedFiles()*, which will _archive_ the files. However, *closeAndArchiveCompactedFiles()* seems simpler, it only move compacted files to archive, whilst *close()* will also try to gracefully close read streams from the *HStore* instance in use, which I guess it's what it should be done at that point (we can safely close the store here, as either in client or mapred mode, there will be an individual *HStore* instance per *CompactionWorker*).

Regarding tests, seems *CompactionTool* currently has no UTs for it. Maybe worth address that on a separate jira?  


was (Author: wchevreuil):
Thanks for reviewing this, [~xucang]! Sorry for the delay in responding, as I was on a short break for the last 4 days.

So, both *close()* and *closeAndArchiveCompactedFiles()* will rely on private method *HStore.removeCompactedFiles()*, which will _archive_ the files. However, *closeAndArchiveCompactedFiles()* seems simpler, it only move compacted files to archive, whilst *close()* will also try to gracefully close read streams from the *HStore* instance in use, which I guess it's what it should be done at that point (we can safely close the store here, as either in client or mapred mode, there will be an individual *HStore* instance per *CompactionWorker*).

Regarding tests, **I guess *CompactionTool* currently has no UTs for it. Maybe worth address that on a separate jira?  

> CompactionTool should close the store it uses for compacting files, in order to properly archive compacted files.
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-21959
>                 URL: https://issues.apache.org/jira/browse/HBASE-21959
>             Project: HBase
>          Issue Type: Bug
>          Components: tooling
>            Reporter: Wellington Chevreuil
>            Assignee: Wellington Chevreuil
>            Priority: Major
>         Attachments: HBASE-21959-master-001.patch
>
>
> While using CompactionTool to offload RSes, noticed compacted files were never archived from original region dir, causing the space used by the region to actually double. Going through its compaction related code on HStore, which is used by CompactionTool for performing compactions, found out what that compacted files archiving happens mainly while closing the HStore instance. CompactionTool is never explicitly closing its HStore instance, so adding a simple patch that properly close the store.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)