You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hbase.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2011/03/23 03:08:05 UTC

[jira] [Commented] (HBASE-3690) Option to Exclude Bulk Import Files from Minor Compaction

    [ https://issues.apache.org/jira/browse/HBASE-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009966#comment-13009966 ] 

Todd Lipcon commented on HBASE-3690:
------------------------------------

One thought - could this be done on a per-bulkload case? ie when you write out the files from HFOF, you specify some kind of HFile meta tag about whether to include or not?

It seems like this isn't good to be cluster-wide for mixed-use clusters, when you might have some tables that have big bulk loads and others that are small increments?

> Option to Exclude Bulk Import Files from Minor Compaction
> ---------------------------------------------------------
>
>                 Key: HBASE-3690
>                 URL: https://issues.apache.org/jira/browse/HBASE-3690
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>            Priority: Minor
>             Fix For: 0.92.0
>
>         Attachments: HBASE-3690.patch
>
>
> We ran an incremental scrape with HFileOutputFormat and encountered major compaction storms.  This is caused by the bug in HBASE-3404.  The permanent fix is a little tricky without HBASE-2856.  We realized that a quicker solution for avoiding these compaction storms is to simply exclude bulk import files from minor compactions and let them only be handled by time-based major compactions.  Add with functionality along with a config option to enable it.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira