You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "zhoushuaifeng (JIRA)" <ji...@apache.org> on 2011/06/09 09:37:58 UTC
[jira] [Updated] (HBASE-3969) Outdated data can not be cleaned in
time
[ https://issues.apache.org/jira/browse/HBASE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhoushuaifeng updated HBASE-3969:
---------------------------------
Attachment: HBASE-3969-solution1.patch
There may be several solution to this problem:
1, Setting a proper and configure able priority to regions need major compact when sending them to the compact queue; (attached patch solution1)
2, Setting the priority of regions need compact according how long it's keyvalues outdated.
3, Run a thread to check the compact queue, if some regions hanging in the quene for a long time, increase it's priority level.
> Outdated data can not be cleaned in time
> ----------------------------------------
>
> Key: HBASE-3969
> URL: https://issues.apache.org/jira/browse/HBASE-3969
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.90.1, 0.90.2, 0.90.3
> Reporter: zhoushuaifeng
> Fix For: 0.90.4
>
> Attachments: HBASE-3969-solution1.patch
>
>
> Compaction checker will send regions to the compact queue to do compact. But the priority of these regions is too low if these regions have only a few storefiles. When there is large through output, and the compact queue will aways have some regions with higher priority. This may causing the major compact be delayed for a long time(even a few days), and outdated data cleaning will also be delayed.
> In our test case, we found some regions sent to the queue by major compact checker hunging in the queue for more than 2 days! Some scanners on these regions cannot get availably data for a long time and lease expired.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira