[jira] [Commented] (SOLR-11239) Deprecate maxSHardsPerNode when autoscaling policies are used

["Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127179#comment-16127179 ] 

Shalin Shekhar Mangar commented on SOLR-11239:
----------------------------------------------

To add some context behind this issue:

Firstly, {{maxShardsPerNode}} defaults to 1. So if anyone needs to host more than 1 replica per node then they must specify {{maxShardsPerNode}} while creating the collection via the collection API.

Secondly, when someone uses the bin/solr scripts to create a collection, the script automatically calculates numShards*repFactor and passes that value as the maxShardsPerNode. This is a workaround for the first point above. This effectively means that the script needs no maxShardsPerNode limit at all.

Thirdly, the {{maxShardsPerNode}} is not stored in collection state and therefore it is only applicable during the time of collection creation. Any future collection API invocations such as add replicas or split shards do not respect {{maxShardsPerNode}}.

Thirdly, the policy framework has its own way of providing constraints similar to maxShardsPerNode both globally (i.e. across all collections) and per-collection. These constraints are respected by all collection APIs such as create, addreplica, splitshard, restore etc.

So maxShardsPerNode is really a half-broken solution to the problem of placing replicas on nodes and we should deprecate it in favor of the capabilities exposed by the policy framework. So how do we do this without surprising the user?

Noble is working on providing a patch that:
# Introduces {{maxShardsPerNode=-1}} as a way of saying that there is no limit and Solr should just go ahead and create the collection as requested
# Change bin/solr to pass maxShardsPerNode=-1 by default.
# If a cluster policy exists or if a collection policy is specified then specifying maxShardsPerNode will throw an exception during collection creation because otherwise it can conflict with the limits specified in the policy. If users need a constraint like this then they must modify the policy.

This way back compat is also preserved. The default maxShardsPerNode remains 1 as before and users who don't need the policy framework can continue to rely on the existing behavior of the maxShardsPerNode parameter.

> Deprecate maxSHardsPerNode when autoscaling policies are used
> -------------------------------------------------------------
>
>                 Key: SOLR-11239
>                 URL: https://issues.apache.org/jira/browse/SOLR-11239
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>    Affects Versions: 7.0
>            Reporter: Noble Paul
>            Assignee: Noble Paul
>            Priority: Blocker
>
> We have found out that {{maxShardPerNode}} is not compatible with the new auto scaling policies. So we need to deprecate that parameter when the autoscaling policies are used.
> the {{bin/solr}} script passes that parameter all the time irrespective of whether the user needs it or not. 
> We need to fix it for 7.0 itself



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org