You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2009/07/16 00:47:14 UTC
[jira] Created: (HBASE-1662) Tool to run major compaction on
catalog regions when hbase is shutdown
Tool to run major compaction on catalog regions when hbase is shutdown
----------------------------------------------------------------------
Key: HBASE-1662
URL: https://issues.apache.org/jira/browse/HBASE-1662
Project: Hadoop HBase
Issue Type: New Feature
Reporter: stack
Fix For: 0.20.0
We need this tool in 0.19. Mildly useful in 0.20.
To migrate (HBASE-1215), all tables need to have had a major compaction done. This should work fine for user-space tables as long as no writes coming in. With catalog tables its a little different. They will always have some edits and its tricky getting cluster to go down with it flushing -- adding more files to region.
Am adding an HRegion#main that will list catalog region content by default -- may help debugging -- and then if you add the major_compact option, will major_compact passed region.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1662) Tool to run major compaction on
catalog regions when hbase is shutdown
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-1662:
-------------------------
Attachment: 1662.patch
Adds some utility to FSUtil -- a DirFilter for FileSystem.getListStatus.
Adds a main to Region. Here is what usage looks like:
{code}
durruti:cleantrunk stack$ ./bin/hbase org.apache.hadoop.hbase.regionserver.HRegion
Usage: HRegion CATLALOG_TABLE_DIR [major_compact]
Options:
major_compact Pass this option to major compact passed region.
Default outputs scan of passed region.
{code}
Here is what happens if you major compact:
{code}
durruti:cleantrunk stack$ ./bin/hbase org.apache.hadoop.hbase.regionserver.HRegion /tmp/hbase-stack/hbase/.META. major_compact
09/07/15 15:45:58 INFO regionserver.HLog: HLog configuration: blocksize=33554432, rollsize=31876710, enabled=true, flushlogentries=100, optionallogflushinternal=10000ms
09/07/15 15:45:59 INFO regionserver.HLog: New hlog /tmp/hbase-stack/hlog.META.1247697958961/hlog.dat.1247697958971
09/07/15 15:45:59 DEBUG regionserver.HRegion: Opening region .META.,,1, encoded=1028785192
09/07/15 15:45:59 DEBUG regionserver.Store: loaded /tmp/hbase-stack/hbase/.META./1028785192/historian/4170498238110389438, isReference=false, sequence id=28, length=987, majorCompaction=true
09/07/15 15:45:59 INFO regionserver.StoreFile: Allocating LruBlockCache with maximum size 199.0m
09/07/15 15:45:59 DEBUG regionserver.Store: loaded /tmp/hbase-stack/hbase/.META./1028785192/info/2332831811545144948, isReference=false, sequence id=28, length=918, majorCompaction=true
09/07/15 15:45:59 INFO regionserver.HRegion: region .META.,,1/1028785192 available; sequence id is 29
09/07/15 15:45:59 INFO regionserver.HRegion: Starting major compaction on region .META.,,1
09/07/15 15:45:59 DEBUG regionserver.Store: Started compaction of 1 file(s) into /tmp/hbase-stack/hbase/.META./compaction.dir/1028785192/5547240455794549466
09/07/15 15:45:59 DEBUG regionserver.Store: Completed major compaction of historian; store size is 987.0
09/07/15 15:45:59 DEBUG regionserver.Store: Started compaction of 1 file(s) into /tmp/hbase-stack/hbase/.META./compaction.dir/1028785192/6342464511479146760
09/07/15 15:45:59 DEBUG regionserver.Store: Completed major compaction of info; store size is 918.0
09/07/15 15:45:59 INFO regionserver.HRegion: compaction completed on region .META.,,1 in 0sec
09/07/15 15:45:59 DEBUG regionserver.HRegion: Closing .META.,,1: compactions & flushes disabled
09/07/15 15:45:59 DEBUG regionserver.HRegion: Updates disabled for region, no outstanding scanners on .META.,,1
09/07/15 15:45:59 DEBUG regionserver.HRegion: No more row locks outstanding on region .META.,,1
09/07/15 15:45:59 DEBUG regionserver.Store: closed historian
09/07/15 15:45:59 DEBUG regionserver.Store: closed info
09/07/15 15:45:59 INFO regionserver.HRegion: Closed .META.,,1
09/07/15 15:45:59 DEBUG regionserver.HLog: closing hlog writer in /tmp/hbase-stack/hlog.META.1247697958961
{code}
> Tool to run major compaction on catalog regions when hbase is shutdown
> ----------------------------------------------------------------------
>
> Key: HBASE-1662
> URL: https://issues.apache.org/jira/browse/HBASE-1662
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Fix For: 0.20.0
>
> Attachments: 1662.patch
>
>
> We need this tool in 0.19. Mildly useful in 0.20.
> To migrate (HBASE-1215), all tables need to have had a major compaction done. This should work fine for user-space tables as long as no writes coming in. With catalog tables its a little different. They will always have some edits and its tricky getting cluster to go down with it flushing -- adding more files to region.
> Am adding an HRegion#main that will list catalog region content by default -- may help debugging -- and then if you add the major_compact option, will major_compact passed region.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1662) Tool to run major compaction on
catalog regions when hbase is shutdown
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-1662.
--------------------------
Resolution: Fixed
Committed to branch and trunk.
> Tool to run major compaction on catalog regions when hbase is shutdown
> ----------------------------------------------------------------------
>
> Key: HBASE-1662
> URL: https://issues.apache.org/jira/browse/HBASE-1662
> Project: Hadoop HBase
> Issue Type: New Feature
> Reporter: stack
> Fix For: 0.20.0
>
> Attachments: 1662.patch
>
>
> We need this tool in 0.19. Mildly useful in 0.20.
> To migrate (HBASE-1215), all tables need to have had a major compaction done. This should work fine for user-space tables as long as no writes coming in. With catalog tables its a little different. They will always have some edits and its tricky getting cluster to go down with it flushing -- adding more files to region.
> Am adding an HRegion#main that will list catalog region content by default -- may help debugging -- and then if you add the major_compact option, will major_compact passed region.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.