[jira] [Commented] (HBASE-9528) Adaptive compaction

["stack (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-9528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13766387#comment-13766387 ] 

stack commented on HBASE-9528:
------------------------------

Sounds like good idea to me.

How you think it would work?  When a region wants to compact, it would send its desire along w/ stats on the compaction it wants to run (bytes, files)  to a central location.  Then the coordinator, or 'central planner' would give out who could compact when?  When a compaction starts/stops, it would let the coordinator know.

The Master should be the coordinator.  Would the desire to compact come in on the back of the heartbeats?  (We should try and get 'load' off the heartbeats and have 'load' instead come into the server via metrics).  Master would need to keep these stats somewhere?  In a system table?

Should we call it "central planning compaction" rather than adaptive?

Good on you [~xieliang007]
                
> Adaptive compaction
> -------------------
>
>                 Key: HBASE-9528
>                 URL: https://issues.apache.org/jira/browse/HBASE-9528
>             Project: HBase
>          Issue Type: Brainstorming
>    Affects Versions: 0.98.0
>            Reporter: Liang Xie
>
> Currently, the compaction policy granularity is based on single machine. we had a thought that introduce a new cluster granularity decision, such that we could improve those case per cluster running status:
> 1) many nodes are compacting aggressive, we call it cluster compaction storm, we should throttle it.
> 2) do more compaction if low traffic in current cluster(similar with off-peak feature), not limit by config timerange(like off-peak timerange), just trigger by load or qps or other stuff.
> comments? thanks

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira