[jira] [Resolved] (HBASE-20800) Master-orchestrated compactions

["Andrew Kyle Purtell (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-20800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Kyle Purtell resolved HBASE-20800.
-----------------------------------------
      Assignee:     (was: Mohit Goel)
    Resolution: Won't Fix

> Master-orchestrated compactions
> -------------------------------
>
>                 Key: HBASE-20800
>                 URL: https://issues.apache.org/jira/browse/HBASE-20800
>             Project: HBase
>          Issue Type: Umbrella
>          Components: Compaction
>            Reporter: Michael Stack
>            Priority: Major
>
> An umbrella issue for having compactions go via the Master so we can have a centralized arbitrator of cluster i/o. If we put Master in the way, we can do stuff like:
>  * Ask the Master for current cluster compaction state; what is running, what is blocked
>  * Master can manage cluster-wide compaction policy and/or throttling/or blocking of compaction i/os.
>  * Master can schedule when and where compactions run so we can guard against the pathological where all RegionServers decide now is the time to major compact bringing on a compaction storm.
> Other side-benefits might include being able to farm out the compaction work to another process -- e.g. the splice machine model of having spark run the compactions -- or just to a separate compactor that we might i/o nice.
>  * We'll need to figure how to externalize the CompactionRequest so it can be passed over RPC.
>  * We'll need to have something like a CompactionManager in the Master process that keeps up current cluster state.
> MOB needs a compaction fabric it can use. Its compactions are currently Master-based only and so don't scale. It could make use of this mechanism to ask the Master to farm out its compaction requests.
> This is an umbrella issue. I thought I'd filed one already on this topic but can't find it. Will shut it down if I trip over it.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)