You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org> on 2009/10/30 19:47:59 UTC
[jira] Created: (HBASE-1947) If HBase starts/stops often in less
than 24 hours, you end up with lots of store files
If HBase starts/stops often in less than 24 hours, you end up with lots of store files
--------------------------------------------------------------------------------------
Key: HBASE-1947
URL: https://issues.apache.org/jira/browse/HBASE-1947
Project: Hadoop HBase
Issue Type: Improvement
Affects Versions: 0.20.1
Reporter: Jean-Daniel Cryans
Fix For: 0.20.2, 0.21.0
Since we don't compact on open and close of regions, all regions that have edits will do a flush when the cluster is stopped so that if you do it a couple of times in a row it's easy to get more than 10 store files and it won't be compacted until the next day. On open, we should check that number and compact if needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1947) If HBase starts/stops often in less
than 24 hours, you end up with lots of store files
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans resolved HBASE-1947.
---------------------------------------
Resolution: Fixed
Assignee: Jean-Daniel Cryans
Hadoop Flags: [Reviewed]
Committed to branch and trunk.
> If HBase starts/stops often in less than 24 hours, you end up with lots of store files
> --------------------------------------------------------------------------------------
>
> Key: HBASE-1947
> URL: https://issues.apache.org/jira/browse/HBASE-1947
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.20.1
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1947.patch
>
>
> Since we don't compact on open and close of regions, all regions that have edits will do a flush when the cluster is stopped so that if you do it a couple of times in a row it's easy to get more than 10 store files and it won't be compacted until the next day. On open, we should check that number and compact if needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1947) If HBase starts/stops often in less
than 24 hours, you end up with lots of store files
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-1947:
--------------------------------------
Attachment: HBASE-1947.patch
Simple patch that adds new methods to do the checks. I tried it on my machine and it compacted the 20 store files I had in ROOT right away.
> If HBase starts/stops often in less than 24 hours, you end up with lots of store files
> --------------------------------------------------------------------------------------
>
> Key: HBASE-1947
> URL: https://issues.apache.org/jira/browse/HBASE-1947
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.20.1
> Reporter: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1947.patch
>
>
> Since we don't compact on open and close of regions, all regions that have edits will do a flush when the cluster is stopped so that if you do it a couple of times in a row it's easy to get more than 10 store files and it won't be compacted until the next day. On open, we should check that number and compact if needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1947) If HBase starts/stops often in less
than 24 hours, you end up with lots of store files
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772747#action_12772747 ]
Jean-Daniel Cryans commented on HBASE-1947:
-------------------------------------------
Ryan reviewed this patch and wants me to clear the compaction message. I will commit the same patch but with:
{code}
- // Startup a compaction early if one is needed, if region has references.
- if (region.hasReferences()) {
+ // Startup a compaction early if one is needed, if region has references
+ // or it a store has too many store files
+ if (region.hasReferences() || region.hasTooManyStoreFiles()) {
this.compactSplitThread.compactionRequested(region,
- "Region has references on open");
+ region.hasReferences() ? "Region has references on open" :
+ "Region has too many store files");
{code}
> If HBase starts/stops often in less than 24 hours, you end up with lots of store files
> --------------------------------------------------------------------------------------
>
> Key: HBASE-1947
> URL: https://issues.apache.org/jira/browse/HBASE-1947
> Project: Hadoop HBase
> Issue Type: Improvement
> Affects Versions: 0.20.1
> Reporter: Jean-Daniel Cryans
> Fix For: 0.20.2, 0.21.0
>
> Attachments: HBASE-1947.patch
>
>
> Since we don't compact on open and close of regions, all regions that have edits will do a flush when the cluster is stopped so that if you do it a couple of times in a row it's easy to get more than 10 store files and it won't be compacted until the next day. On open, we should check that number and compact if needed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.