You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Viraj Jasani (Jira)" <ji...@apache.org> on 2020/01/11 04:31:00 UTC
[jira] [Comment Edited] (HBASE-23349) Low refCount preventing
archival of compacted away files
[ https://issues.apache.org/jira/browse/HBASE-23349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17013344#comment-17013344 ]
Viraj Jasani edited comment on HBASE-23349 at 1/11/20 4:30 AM:
---------------------------------------------------------------
I agree on considering removal of refCount system based on your suggestions [~apurtell] [~larsh]
However, I am just trying to give one chance to consider both points:
# Perf improvement as part of HBASE-13082
# Scanner reset during compaction if required(config based)
Tried to use volatile enum(NONE, FLUSH, COMPACTION) instead of 2 volatile booleans for Scanner.next(), seek() calls to not let perf degrade for normal scans. Hence, if archival is not happening correctly, we can notify open scanners and reset KV Heap in the next(), seek() runs. However, whether next() has to reset KVHeap is something that can be determined based on volatile enum value which would be set while notifying scanners.
[https://github.com/apache/hbase/pull/939] with some tests for Scanner reset during compaction and successful archival thereafter. You might prefer this over direct removal of refCount system.
Considering refCount presence in HBase for some time, someone might have started building some system(alert, recovery etc) based on refCount usecase. In fact, we have also done auto region reopen etc but other users might have built some other usecases too.
was (Author: vjasani):
I agree on considering removal of refCount system based on your suggestions [~apurtell] [~larsh]
However, I am just trying to give one chance to consider both points:
# Perf improvement as part of HBASE-13082
# Scanner reset during compaction if required(config based)
Tried to use volatile enum(NONE, FLUSH, COMPACTION) instead of 2 volatile booleans for Scanner.next(), seek() calls to not let perf degrade for normal scans. Hence, if archival is not happening correctly, we can notify open scanners and reset KV Heap in the next(), seek() runs. However, whether next() has to reset KVHeap is something that can be determined based on volatile enum value which would be set while notifying scanners.
[https://github.com/apache/hbase/pull/939] with some tests for Scanner reset during compaction and successful archival thereafter.
Considering refCount presence in HBase for some time, someone might have started building some system(alert, recovery etc) based on refCount usecase. In fact, we have also done auto region reopen etc but other users might have built some other usecases too.
> Low refCount preventing archival of compacted away files
> --------------------------------------------------------
>
> Key: HBASE-23349
> URL: https://issues.apache.org/jira/browse/HBASE-23349
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 3.0.0, 2.3.0, 1.6.0
> Reporter: Viraj Jasani
> Assignee: Viraj Jasani
> Priority: Major
> Fix For: 3.0.0, 2.3.0, 1.6.0
>
>
> We have observed that refCount on compacted away store files as low as 1 is prevent archival.
> {code:java}
> regionserver.HStore - Can't archive compacted file hdfs://{{root-dir}}/hbase/data/default/t1/12a9e1112e0371955b3db8d3ebb2d298/cf1/73b72f5ddfce4a34a9e01afe7b83c1f9 because of either isCompactedAway=true or file has reference, isReferencedInReads=true, refCount=1, skipping for now.
> {code}
> We should come up with core code (run as part of discharger thread) gracefully resolve reader lock issue by resetting ongoing scanners to start pointing to new store files instead of compacted away store files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)