You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Phabricator (Updated) (JIRA)" <ji...@apache.org> on 2011/11/10 06:39:53 UTC
[jira] [Updated] (HBASE-3690) Option to Exclude Bulk Import Files
from Minor Compaction
[ https://issues.apache.org/jira/browse/HBASE-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HBASE-3690:
-------------------------------
Attachment: D357.1.patch
nspiegelberg requested code review of "[jira] [HBASE-3690] Option to Exclude Bulk Import Files from Minor Compaction".
Reviewers: stack, Kannan, JIRA
We ran an incremental scrape with HFileOutputFormat and
encountered major compaction storms. This is caused by the bug in
HBASE-3404. The permanent fix is a little tricky without HBASE-2856. We
realized that a quicker solution for avoiding these compaction storms is
to simply exclude bulk import files from minor compactions and let them
only be handled by time-based major compactions. Add with functionality
along with a config option to enable it.
Rewrote this feature to be done on a per-bulkload basis.
TEST PLAN
- mvn test -Dtest=TestHFileOutputFormat
DiffCamp Revision:
REVISION DETAIL
https://reviews.facebook.net/D357
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
> Option to Exclude Bulk Import Files from Minor Compaction
> ---------------------------------------------------------
>
> Key: HBASE-3690
> URL: https://issues.apache.org/jira/browse/HBASE-3690
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Nicolas Spiegelberg
> Assignee: Nicolas Spiegelberg
> Priority: Minor
> Attachments: D357.1.patch, HBASE-3690.patch
>
>
> We ran an incremental scrape with HFileOutputFormat and encountered major compaction storms. This is caused by the bug in HBASE-3404. The permanent fix is a little tricky without HBASE-2856. We realized that a quicker solution for avoiding these compaction storms is to simply exclude bulk import files from minor compactions and let them only be handled by time-based major compactions. Add with functionality along with a config option to enable it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira