You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Erick Erickson (JIRA)" <ji...@apache.org> on 2017/12/11 06:22:00 UTC
[jira] [Commented] (SOLR-11634) Create collection doesn't respect
`maxShardsPerNode`
[ https://issues.apache.org/jira/browse/SOLR-11634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285553#comment-16285553 ]
Erick Erickson commented on SOLR-11634:
---------------------------------------
Here's the calculation from CreateCollectionCmd. nodeList.size() is the number of live nodes or the number of nodes in createnodeset if specified:
{noformat}
int maxShardsAllowedToCreate = maxShardsPerNode * nodeList.size();
int requestedShardsToCreate = numSlices * repFactor;
if (maxShardsAllowedToCreate < requestedShardsToCreate) {
report error back
}
{noformat}
So maxShardsPerNode is more like maxReplicasPerNode. If I have a 4 node cluster and specify numShards=4&replicationFactor=10 my maxShardsPerNode must be set to 10. What? I only have 4 shards in the first place. maxShardsPerNode only really makes sense when replicationFactor=1.
That aside, I can't reconcile getting more than 2 replicas (no matter what shard they are part of) on a single node when specifying maxShardsPerNode=2, even if I do think of it as maxReplicasPerNode. How many nodes were you trying the above with? And I'm assuming you didn't have the rule bit originally.
Solr doesn't really understand a machine as opposed to a JVM though. So if I have two JVMs running on the same physical machine, something like:
192.168.0.1:8981
192.168.0.1:8982
those are seen as separate live nodes and Solr would happily put one replica on each. Could that be what you were seeing?
Would it be possible to attach the CLUSTERSTATUS along with the command you used to create it? something like
solr_node:port/solr/admin/collections?action=CLUSTERSTATUS&wt=json
Note it's important that we see the live_nodes Znode too.
> Create collection doesn't respect `maxShardsPerNode`
> ----------------------------------------------------
>
> Key: SOLR-11634
> URL: https://issues.apache.org/jira/browse/SOLR-11634
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Affects Versions: 6.6.1
> Reporter: Nikolay Martynov
> Assignee: Erick Erickson
>
> Command
> {noformat}
> curl 'http://host:8983/solr/admin/collections?action=CREATE&name=xxx&numShards=16&replicationFactor=3&collection.configName=config&maxShardsPerNode=2&rule=shard:*,replica:<2,node:*&rule=shard:*,replica:<2,sysprop.aws.az:*'
> {noformat}
> creates collection with 1,2 and 3 shard per nodes - looks like {{maxShardsPerNode}} is being ignored.
> Adding {{rule=replica:<{},node:*}} seems to help, but I'm not sure if this is correct and it doesn't seem to match documented behaviour.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org