You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Karl Stoney <Ka...@autotrader.co.uk.INVALID> on 2020/01/30 15:58:59 UTC
Replica type affinity
Hey everyone,
Does anyone know of a way to have solr replicas assigned to specific nodes by some sort of identifying value (in solrcloud).
In summary I’m trying to have some Read only replicas only every be assigned to nodes named “solr-ephemeral-x” and my nrt and masters assigned to “solr-index”.
Kind of like rack affinity in elasticsearch!
Get Outlook for iOS<https://aka.ms/o0ukef>
This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office: 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 9439967). This email and any files transmitted with it are confidential and may be legally privileged, and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This email message has been swept for the presence of computer viruses.
Re: Replica type affinity
Posted by Jason Gerlowski <ge...@gmail.com>.
This is a bit of a guess - I haven't used this functionality before.
But to a novice the "tag" Rule Condition for "Rule Based Replica
Placement" sounds similar to the requirements you mentioned above.
https://lucene.apache.org/solr/guide/8_3/rule-based-replica-placement.html#rule-conditions
Good luck,
Jason
On Thu, Jan 30, 2020 at 1:00 PM Karl Stoney
<Ka...@autotrader.co.uk.invalid> wrote:
>
> Hey,
> Thanks for the reply but I'm trying to have something fully automated and dynamic. For context I run solr on kubernetes, and at the moment it works beautifully with autoscaling (i can scale up the kubernetes deployment and solr adds replicas and removes them).
>
> I'm trying to add a new type of node though, backed by very fast but ephemeral disks and the idea was to have only PULL replicas running on those nodes automatically and NRT on the persistent disk instances.
>
> Might be a pipe dream but I'm striving for no manual configuration.
> ________________________________
> From: Edward Ribeiro <ed...@gmail.com>
> Sent: 30 January 2020 16:56
> To: solr-user@lucene.apache.org <so...@lucene.apache.org>
> Subject: Re: Replica type affinity
>
> Hi Karl,
>
> During collection creation you can specify the `createNodeSet` parameter as
> specified by the Solr Reference Guide snippet below:
>
> "createNodeSet
> Allows defining the nodes to spread the new collection across. The format
> is a comma-separated list of node_names, such as
> localhost:8983_solr,localhost:8984_solr,localhost:8985_solr.
> If not provided, the CREATE operation will create shard-replicas spread
> across all live Solr nodes.
> Alternatively, use the special value of EMPTY to initially create no
> shard-replica within the new collection and then later use the ADDREPLICA
> operation to add shard-replicas when and where required."
>
>
> There's also Collections API that you can use the node parameter of
> ADDREPLICA to specify the node that replica shard should be created on.
> See:
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flucene.apache.org%2Fsolr%2Fguide%2F6_6%2Fcollections-api.html%23CollectionsAPI-Input.9&data=02%7C01%7Ckarl.stoney%40autotrader.co.uk%7Ce6f81aab85274cd0081408d7a5a56464%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637160002076345528&sdata=3pFUtr6o7vK0srGR60lIUc%2Fo9QSftmAcnQDkcx5z%2Bl8%3D&reserved=0
> Other
> commands that can be useful are REPLACENODE, MOVEREPLICA.
>
> Edward
>
>
> On Thu, Jan 30, 2020 at 1:00 PM Karl Stoney
> <Ka...@autotrader.co.uk.invalid> wrote:
>
> > Hey everyone,
> > Does anyone know of a way to have solr replicas assigned to specific nodes
> > by some sort of identifying value (in solrcloud).
> >
> > In summary I’m trying to have some Read only replicas only every be
> > assigned to nodes named “solr-ephemeral-x” and my nrt and masters assigned
> > to “solr-index”.
> >
> > Kind of like rack affinity in elasticsearch!
> >
> > Get Outlook for iOS<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=02%7C01%7Ckarl.stoney%40autotrader.co.uk%7Ce6f81aab85274cd0081408d7a5a56464%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637160002076345528&sdata=a%2BRpt9TyPy4oksfWZzl79rs7pLIwPnPE4AX%2B2SZr03w%3D&reserved=0>
> > This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office:
> > 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England
> > No. 9439967). This email and any files transmitted with it are confidential
> > and may be legally privileged, and intended solely for the use of the
> > individual or entity to whom they are addressed. If you have received this
> > email in error please notify the sender. This email message has been swept
> > for the presence of computer viruses.
> >
> This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office: 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 9439967). This email and any files transmitted with it are confidential and may be legally privileged, and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This email message has been swept for the presence of computer viruses.
Re: Replica type affinity
Posted by Karl Stoney <Ka...@autotrader.co.uk.INVALID>.
Hey,
Thanks for the reply but I'm trying to have something fully automated and dynamic. For context I run solr on kubernetes, and at the moment it works beautifully with autoscaling (i can scale up the kubernetes deployment and solr adds replicas and removes them).
I'm trying to add a new type of node though, backed by very fast but ephemeral disks and the idea was to have only PULL replicas running on those nodes automatically and NRT on the persistent disk instances.
Might be a pipe dream but I'm striving for no manual configuration.
________________________________
From: Edward Ribeiro <ed...@gmail.com>
Sent: 30 January 2020 16:56
To: solr-user@lucene.apache.org <so...@lucene.apache.org>
Subject: Re: Replica type affinity
Hi Karl,
During collection creation you can specify the `createNodeSet` parameter as
specified by the Solr Reference Guide snippet below:
"createNodeSet
Allows defining the nodes to spread the new collection across. The format
is a comma-separated list of node_names, such as
localhost:8983_solr,localhost:8984_solr,localhost:8985_solr.
If not provided, the CREATE operation will create shard-replicas spread
across all live Solr nodes.
Alternatively, use the special value of EMPTY to initially create no
shard-replica within the new collection and then later use the ADDREPLICA
operation to add shard-replicas when and where required."
There's also Collections API that you can use the node parameter of
ADDREPLICA to specify the node that replica shard should be created on.
See:
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flucene.apache.org%2Fsolr%2Fguide%2F6_6%2Fcollections-api.html%23CollectionsAPI-Input.9&data=02%7C01%7Ckarl.stoney%40autotrader.co.uk%7Ce6f81aab85274cd0081408d7a5a56464%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637160002076345528&sdata=3pFUtr6o7vK0srGR60lIUc%2Fo9QSftmAcnQDkcx5z%2Bl8%3D&reserved=0
Other
commands that can be useful are REPLACENODE, MOVEREPLICA.
Edward
On Thu, Jan 30, 2020 at 1:00 PM Karl Stoney
<Ka...@autotrader.co.uk.invalid> wrote:
> Hey everyone,
> Does anyone know of a way to have solr replicas assigned to specific nodes
> by some sort of identifying value (in solrcloud).
>
> In summary I’m trying to have some Read only replicas only every be
> assigned to nodes named “solr-ephemeral-x” and my nrt and masters assigned
> to “solr-index”.
>
> Kind of like rack affinity in elasticsearch!
>
> Get Outlook for iOS<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=02%7C01%7Ckarl.stoney%40autotrader.co.uk%7Ce6f81aab85274cd0081408d7a5a56464%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637160002076345528&sdata=a%2BRpt9TyPy4oksfWZzl79rs7pLIwPnPE4AX%2B2SZr03w%3D&reserved=0>
> This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office:
> 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England
> No. 9439967). This email and any files transmitted with it are confidential
> and may be legally privileged, and intended solely for the use of the
> individual or entity to whom they are addressed. If you have received this
> email in error please notify the sender. This email message has been swept
> for the presence of computer viruses.
>
This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office: 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 9439967). This email and any files transmitted with it are confidential and may be legally privileged, and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This email message has been swept for the presence of computer viruses.
Re: Replica type affinity
Posted by Edward Ribeiro <ed...@gmail.com>.
Hi Karl,
During collection creation you can specify the `createNodeSet` parameter as
specified by the Solr Reference Guide snippet below:
"createNodeSet
Allows defining the nodes to spread the new collection across. The format
is a comma-separated list of node_names, such as
localhost:8983_solr,localhost:8984_solr,localhost:8985_solr.
If not provided, the CREATE operation will create shard-replicas spread
across all live Solr nodes.
Alternatively, use the special value of EMPTY to initially create no
shard-replica within the new collection and then later use the ADDREPLICA
operation to add shard-replicas when and where required."
There's also Collections API that you can use the node parameter of
ADDREPLICA to specify the node that replica shard should be created on.
See:
https://lucene.apache.org/solr/guide/6_6/collections-api.html#CollectionsAPI-Input.9
Other
commands that can be useful are REPLACENODE, MOVEREPLICA.
Edward
On Thu, Jan 30, 2020 at 1:00 PM Karl Stoney
<Ka...@autotrader.co.uk.invalid> wrote:
> Hey everyone,
> Does anyone know of a way to have solr replicas assigned to specific nodes
> by some sort of identifying value (in solrcloud).
>
> In summary I’m trying to have some Read only replicas only every be
> assigned to nodes named “solr-ephemeral-x” and my nrt and masters assigned
> to “solr-index”.
>
> Kind of like rack affinity in elasticsearch!
>
> Get Outlook for iOS<https://aka.ms/o0ukef>
> This e-mail is sent on behalf of Auto Trader Group Plc, Registered Office:
> 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England
> No. 9439967). This email and any files transmitted with it are confidential
> and may be legally privileged, and intended solely for the use of the
> individual or entity to whom they are addressed. If you have received this
> email in error please notify the sender. This email message has been swept
> for the presence of computer viruses.
>