You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Lars Hofhansl (JIRA)" <ji...@apache.org> on 2015/09/09 01:56:46 UTC
[jira] [Comment Edited] (HBASE-14272) Enforce major compaction on
stores with TTL or time.to.purge.deletes enabled
[ https://issues.apache.org/jira/browse/HBASE-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14735863#comment-14735863 ]
Lars Hofhansl edited comment on HBASE-14272 at 9/8/15 11:56 PM:
----------------------------------------------------------------
OK... Assigned to me.
Let's simply start with always doing the time based compaction when TTL is set. Period. Simple.
Then we do the following (sub jira or here):
# track a flag during compactions that is set for any HFile that has any delete markers in it.
# track the maximum number of version we have seen for any Column. Note: need to make sure that we can do that cheaply (need a compare of row-key, family, CQ).
With this in place we should be able to decide whether we need to compact by the following:
{code}
...
} else if (TTL set) {
if (MIN_VERSIONS > 0) {
if (store.maxNumVersions() > MIN_VERSIONS) {
compact();
}
} else {
compact()
}
} else if (time.to.purge.deletes > 0) {
if (store.hasDeletes() && !KEEP_DELETED_CELLS) {
compact()
}
}
{code}
I'm sure this logic can be expressed more nicely, but you get the point.
was (Author: lhofhansl):
OK... Assigned to me.
Let's simply start with always doing the time based compaction when TTL is set. Period. Simple.
Then we do the following (sub jira or here):
# track a flag during compactions that is set for any HFile that has any delete markers in it.
# track the maximum number of version we have seen for any Column. Note: need to make sure that we can do that cheaply (need a compare of row-key, family, CQ).
With this in place we should be able to decide whether a compaction by the following:
{code}
...
} else if (TTL set) {
if (MIN_VERSIONS > 0) {
if (store.maxNumVersions() > MIN_VERSIONS) {
compact();
}
} else {
compact()
}
} else if (time.to.purge.deletes > 0) {
if (store.hasDeletes() && !KEEP_DELETED_CELLS) {
compact()
}
}
{code}
I'm sure this logic can be expressed more nicely, but you get the point.
> Enforce major compaction on stores with TTL or time.to.purge.deletes enabled
> ----------------------------------------------------------------------------
>
> Key: HBASE-14272
> URL: https://issues.apache.org/jira/browse/HBASE-14272
> Project: HBase
> Issue Type: Bug
> Reporter: Vladimir Rodionov
> Assignee: Lars Hofhansl
> Fix For: 2.0.0
>
> Attachments: HBASE-14272-v2.patch, HBASE-14272.patch
>
>
> Currently, if store has one (major compacted) file, the only case when major compaction will be triggered for this file again - when locality is below threshold, defined by *hbase.hstore.min.locality.to.skip.major.compact* or TTL expired some cells. If file has locality greater than this threshold it will never be major compacted until Store's TTL kicks in. For CF with KEEP_DELETED_CELLS on, compaction must be enabled always (even for single file), regardless of locality, when deleted cells are expired (*hbase.hstore.time.to.purge.deletes*)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)