[jira] [Comment Edited] (HBASE-12986) Compaction pressure based client pushback

["Andrew Purtell (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-12986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310920#comment-14310920 ] 

Andrew Purtell edited comment on HBASE-12986 at 2/7/15 8:44 PM:
----------------------------------------------------------------

We might also want to leave the current ExponentialClientBackoffPolicy as is and introduce a new version of it that decouples the response to load factors from contributions of load factor terms to the equation. In other words, make the collection of load factors pluggable, define how the load factor plugins should contribute terms to the backoff calculation equation, and have the policy implement only the backoff calculation equation. We can break out load factor plugins for:
- Memstore load
- Heap occupancy (HBASE-12731)
- System load (HBASE-12217)
- Compaction pressure (this JIRA)

Or we could leave it up to the policy implementations to remain up to date with all of the various server load feedbacks. The above suggestion is flexible but could make it complicated to manage. Easier to specify a single policy class then a policy class plus multiple load factor plugin classes. Then again, as we add more factors we'd probably want to introduce boolean switches into the policy classes so users can disable factors they don't care about. 

Thoughts? 

[~jesse_yates] [~lhofhansl] 


was (Author: apurtell):
We might also want to leave the current ExponentialClientBackoffPolicy as is and introduce a new version of it that decouples the response to load factors from contributions of load factor terms to the equation. In other words, make the collection of load factors pluggable, define how the load factor plugins should contribute terms to the backoff calculation equation, and have the policy implement only the backoff calculation equation. We can break out load factor plugins for:
- Memstore load
- Heap occupancy (HBASE-12731)
- System load (HBASE-12217)
- Compaction pressure (this JIRA)

Or we could leave it up to the policy implementations to remain up to date with all of the various server load feedbacks. The above suggestion is flexible but could make it complicated to manage. Easier to specify a single policy class then a policy class plus multiple load factor plugin classes.

Thoughts? 

[~jesse_yates] [~lhofhansl] 

> Compaction pressure based client pushback
> -----------------------------------------
>
>                 Key: HBASE-12986
>                 URL: https://issues.apache.org/jira/browse/HBASE-12986
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>
> HBASE-8329 recently introduced on all branches {{double RegionServerServices#getCompactionPressure()}}, which returns a value greater than or equal to 0.0, and any value greater than 1.0 means we have exceeded the store file limit on some stores. It could be reasonable to send this value along in server load statistics (clamping max at 1.0), and consider it as an additional term in the ExponentialClientBackoffPolicy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)