You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/12/14 19:42:00 UTC
[jira] [Commented] (BOOKKEEPER-950) Ledger placement policy to
accomodate different storage capacity of bookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749274#comment-15749274 ]
ASF GitHub Bot commented on BOOKKEEPER-950:
-------------------------------------------
GitHub user rithin-shetty opened a pull request:
https://github.com/apache/bookkeeper/pull/93
BOOKKEEPER-950: Ledger placement policy to accomodate different stora…
…ge capacity of bookies
This change introduces Disk weight based ledger placement. Currently free disk space is the only supported
weight for a bookie. This change also introduces a new protocol message between bk client and server
called GET_BOOKIE_INFO. This message is used by the client to retrieve the free disk space info from
all the bookies. The existing placement policies: DefaultPlacementPolicy and RackAwarePlacementPolicy
have been enhanced to make use of the weight while selecting bookies. New test cases have been added to
test RackawarePlacement with weights. A new test class has been added to test the weight based selection
algorithm in a stand alone fashion.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rithin-shetty/bookkeeper weightBasedPlacementDec13
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/bookkeeper/pull/93.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #93
----
commit e78b11d46ae4c3bdd58082d2b4ffa6e9ce06c68d
Author: Rithin <ri...@salesforce.com>
Date: 2016-12-14T04:56:15Z
BOOKKEEPER-950: Ledger placement policy to accomodate different storage capacity of bookies
This change introduces Disk weight based ledger placement. Currently free disk space is the only supported
weight for a bookie. This change also introduces a new protocol message between bk client and server
called GET_BOOKIE_INFO. This message is used by the client to retrieve the free disk space info from
all the bookies. The existing placement policies: DefaultPlacementPolicy and RackAwarePlacementPolicy
have been enhanced to make use of the weight while selecting bookies. New test cases have been added to
test RackawarePlacement with weights. A new test class has been added to test the weight based selection
algorithm in a stand alone fashion.
----
> Ledger placement policy to accomodate different storage capacity of bookies
> ---------------------------------------------------------------------------
>
> Key: BOOKKEEPER-950
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-950
> Project: Bookkeeper
> Issue Type: New Feature
> Reporter: Rithin Shetty
> Assignee: Rithin Shetty
> Fix For: 4.5.0
>
> Original Estimate: 672h
> Remaining Estimate: 672h
>
> In our environment, in Salesforce, we are likely to have bookie nodes with different storage capacity: some will have 1TB others might have 3TB. Also, our ledgers are likely going to be long lived. The current ledger placement policy selects the bookies randomly leading to uniform distribution. This would cause some of bookies to reach high utilization while the rest would be underutilized. We need a new ledger placement policy that has higher probability of selecting bookies with higher free disk space than the ones with lower disk free space.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)