[jira] [Commented] (SOLR-16806) Add a BalanceReplicas API

["Houston Putman (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-16806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17732597#comment-17732597 ] 

Houston Putman commented on SOLR-16806:
---------------------------------------

Including from the PR, since it is very important information:

 
{quote}This PR now shifts the existing PlacementPlugins to be subclasses of OrderedNodePlacementPlugin, which gives default implementations of {{computePlacements()}} and {{{}computeBalancing(){}}}. These subclasses merely need to implement a {{getWeightedNodes()}} method, which returns a mapping from {{Node}} to {{{}WeightedNode{}}}. Each plugin makes its own implementation of {{{}WeightedNode{}}}, because the "weighting" is determined by what that plugin wants to prioritize when making selections.

The {{WeightedNode}} abstract class has become a bit bloated, but basically it allows a Node to keep track of a state, and make an overall "weight" for that node with the given state. (It also has a method to project relevant weight with a given replica added, that's what used for {{{}computePlacements(){}}}).

I'm sure I need to do a lot more explanation, but in general, the given plugins should work almost exactly as they do today, with the added benefit of having {{computeBalancing()}} come for "free". (though I'm sure the implementation of that shared method can be improved with time)

This OrderedNodePlacementPlugin is just a class that the existing PlacementPlugins extend, so it is back-compat with custom plugins. We should mark the class experimental for 9.x, so that we can make improvements to OrderedNodePlacementPlugin without having to worry about back-compat guarantees while we are improving these APIs. (e.g., the WeightedNode class may need additional methods if we want to make improvements to the sorting).
{quote}

> Add a BalanceReplicas API
> -------------------------
>
>                 Key: SOLR-16806
>                 URL: https://issues.apache.org/jira/browse/SOLR-16806
>             Project: Solr
>          Issue Type: Sub-task
>            Reporter: Houston Putman
>            Priority: Major
>          Time Spent: 6h 40m
>  Remaining Estimate: 0h
>
> Currently there is no way to move replicas to a new, empty, Solr node, without manually choosing the replicas yourself, and moving/recreating the replicas. The logic we would need for this is to choose the best replicas to move to an empty Solr node, and stop when that Solr node reaches an "equilibrium" with the rest of the cluster.
> This logic is very close to merely balancing the existing replicas across a given set of nodes. So instead of creating a similar command that solves a subset of use-cases, it is probably worth it to take a bit more effort and create the general balance-replicas API to solve all balancing use-cases.
> The API would be quite simple. It would be something like {*}PUT "/api/cluster/balanceReplicas"{*}, and it would take in a list of nodes:
> {"nodes": [ "node1", "node2" ]}
> If no nodes were provided in the request body, then Solr would default to balancing across all data nodes.
> I don't see a need to write a V1 API for this.
> This requires adding method signatures to the PlacementPlugin, but we can make them default to a NO-OP, preserving compatibility for existing custom-plugins. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org