[jira] [Updated] (HBASE-17854) Use StealJobQueue in HFileCleaner after HBASE-17215

["Yu Li (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-17854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yu Li updated HBASE-17854:
--------------------------
    Attachment: HBASE-17854.patch

Uploading the initial patch, which is based on HBASE-17215 changes.

Notice that after changing to {{StealJobQueue}}, the queue size is unbounded, and if there're already too many archived hfiles left for cleaning, there's potential risk of HMaster OOM.

Searched for some implementation of bounded PriorityBlockingQueue but seems not that easy, see this [link|http://stackoverflow.com/questions/2341615/bounded-priorityblockingqueue] on stackoverflow.

> Use StealJobQueue in HFileCleaner after HBASE-17215
> ---------------------------------------------------
>
>                 Key: HBASE-17854
>                 URL: https://issues.apache.org/jira/browse/HBASE-17854
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 2.0
>            Reporter: Yu Li
>            Assignee: Yu Li
>         Attachments: HBASE-17854.patch
>
>
> In HBASE-17215 we use specific threads for deleting large/small (archived) hfiles, and will improve it from below aspects in this JIRA:
> 1. Using {{StealJobQueue}} to allow large file deletion thread to steal jobs from small queue, based on the experience that in real world there'll be much more small hfiles
> 2. {{StealJobQueue}} is a kind of {{PriorityQueue}}, so we could also delete from the larger file in the queues.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)