Incorrect shard placement during Collection creation in 7.6

[Bram Van Dam <br...@intix.eu>]

Hey folks,

TL;DR; createNodeSet & shards combination is not being respected.

I'm attempting to create a collection with multiple shards, but
apparently the value of createNodeSet is not being respected and shards
are being assigned to nodes seemingly at random.

createNodeSet.shuffle is set to false, so that's not the cause.
Furthermore, sometimes not all nodes in the request are used.

Here's my request, cleaned up for legibility. Note that the node names
are IP addresses but I've removed the first 3 octets for legibility.


admin/collections
?action=CREATE
&name=collectionName
&router.name=implicit
&shards=collectionName1,collectionName2,collectionName3,collectionName4,collectionName5,collectionName6
&maxShardsPerNode=1024
&collection.configName=some_config
&createNodeSet=171:8180_solr,172:8180_solr,173:8180_solr,177:8180_solr,179:8180_solr,179:8180_solr
&createNodeSet.shuffle=false
&waitForFinalState=true

Note that I'm creating a collection with 6 shards across 5 nodes.

Requested:
collectionName1: 171:8180_solr
collectionName2: 172:8180_solr
collectionName3: 173:8180_solr
collectionName4: 177:8180_solr
collectionName5: 179:8180_solr
collectionName6: 179:8180_solr

Actual:
collectionName1: 177:8180_solr
collectionName2: 172:8180_solr
collectionName3: 179:8180_solr
collectionName4: 173:8180_solr
collectionName5: 171:8180_solr
collectionName6: 171:8180_solr

Not a single shard ends up on the requested node.

Additionally, when the response comes back, it only contained
information about 5 of the 6 created cores (even though 6 were created).
Possibly because there are only 5 nodes?

Am I misunderstanding the way this is supposed to work? Or did I stumble
upon a bug? Should I attempt to create a collection without shards and
add them one at a time for better control?

Sidenote: having control over which shard lives where is a business
requirement, so leaving Solr to its own devices is, sadly, not an option
in this case :-(

Thanks a bunch,

 - Bram

Re: Incorrect shard placement during Collection creation in 7.6

[Bram Van Dam <br...@intix.eu>]

Thanks Erick, I just created SOLR-13247 and linked it to SOLR-12944.

 - Bram

On 13/02/2019 18:31, Erick Erickson wrote:
> I haven't verified, but this looks like a JIRA to me. Looks like some
> of the create logic may have issues, see: SOLR-12944 and maybe link to
> that JIRA?

Re: Incorrect shard placement during Collection creation in 7.6

[Erick Erickson <er...@gmail.com>]

I haven't verified, but this looks like a JIRA to me. Looks like some
of the create logic may have issues, see: SOLR-12944 and maybe link to
that JIRA?

Best,
Erick

On Wed, Feb 13, 2019 at 4:15 AM Bram Van Dam <br...@intix.eu> wrote:
>
> > TL;DR; createNodeSet & shards combination is not being respected.
>
> Update: Upgraded to 7.7, no improvement sadly.

Re: Incorrect shard placement during Collection creation in 7.6

[Bram Van Dam <br...@intix.eu>]

> TL;DR; createNodeSet & shards combination is not being respected.

Update: Upgraded to 7.7, no improvement sadly.