You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "McCallick, Paul" <Pa...@nordstrom.com> on 2016/02/14 02:01:25 UTC
Adding nodes
I’d like to verify the following:
- When creating a new collection, SOLRCloud will use all available nodes for the collection, adding cores to each. This assumes that you do not specify a replicationFactor.
- When adding new nodes to the cluster AFTER the collection is created, one must use the core admin api to add the node to the collection.
I would really like to see the second case behave more like the first. If I add a node to the cluster, it is automatically used as a replica for existing clusters without my having to do so. This would really simplify things.
Paul McCallick
Sr Manager Information Technology
eCommerce Foundation
Re: Adding nodes
Posted by Susheel Kumar <su...@gmail.com>.
Hi Paul, Thanks for the detail but I am still not able to understand how
the CoreAPI would make it easier for you to create replica's. I understand
that using Core API, you can add more cores but would that also populate
the data so that it can serve queries / act like a replica.
Second, As Shawn mentioned in the link above that adding Replica for
auto-scaling or in a near real time etc. is not a good idea since it put
more load on the system and causing delay.
Unless you have copy of indexes (assuming index is static) and you can
create more cores dynamically in which case Core API may work for your case.
Thanks,
Susheel
On Sun, Feb 14, 2016 at 2:17 PM, McCallick, Paul <
Paul.E.McCallick@nordstrom.com> wrote:
> These are excellent questions and give me a good sense of why you suggest
> using the collections api.
>
> In our case we have 8 shards of product data with a even distribution of
> data per shard, no hot spots. We have very different load at different
> points in the year (cyber monday), and we tend to have very little traffic
> at night. I'm thinking of two use cases:
>
> 1) we are seeing increased latency due to load and want to add 8 more
> replicas to handle the query volume. Once the volume subsides, we'd remove
> the nodes.
>
> 2) we lose a node due to some unexpected failure (ec2 tends to do this).
> We want auto scaling to detect the failure and add a node to replace the
> failed one.
>
> In both cases the core api makes it easy. It adds nodes to the shards
> evenly. Otherwise we have to write a fairly involved script that is subject
> to race conditions to determine which shard to add nodes to.
>
> Let me know if I'm making dangerous or uninformed assumptions, as I'm new
> to solr.
>
> Thanks,
> Paul
>
> > On Feb 14, 2016, at 10:35 AM, Susheel Kumar <su...@gmail.com>
> wrote:
> >
> > Hi Pual,
> >
> >
> > For Auto-scaling, it depends on how you are thinking to design and
> what/how
> > do you want to scale. Which scenario you think makes coreadmin API easy
> to
> > use for a sharded SolrCloud environment?
> >
> > Isn't if in a sharded environment (assume 3 shards A,B & C) and shard B
> has
> > having higher or more load, then you want to add Replica for shard B to
> > distribute the load or if a particular shard replica goes down then you
> > want to add another Replica back for the shard in which case ADDREPLICA
> > requires a shard name?
> >
> > Can you describe your scenario / provide more detail?
> >
> > Thanks,
> > Susheel
> >
> >
> >
> > On Sun, Feb 14, 2016 at 11:51 AM, McCallick, Paul <
> > Paul.E.McCallick@nordstrom.com> wrote:
> >
> >> Hi all,
> >>
> >>
> >> This doesn’t really answer the following question:
> >>
> >> What is the suggested way to add a new node to a collection via the
> >> apis? I am specifically thinking of autoscale scenarios where a node
> has
> >> gone down or more nodes are needed to handle load.
> >>
> >>
> >> The coreadmin api makes this easy. The collections api (ADDREPLICA),
> >> makes this very difficult.
> >>
> >>
> >>> On 2/14/16, 8:19 AM, "Susheel Kumar" <su...@gmail.com> wrote:
> >>>
> >>> Hi Paul,
> >>>
> >>> Shawn is referring to use Collections API
> >>> https://cwiki.apache.org/confluence/display/solr/Collections+API than
> >> Core
> >>> Admin API
> https://cwiki.apache.org/confluence/display/solr/CoreAdmin+API
> >>> for SolrCloud.
> >>>
> >>> Hope that clarifies and you mentioned about ADDREPLICA which is the
> >>> collections API, so you are on right track.
> >>>
> >>> Thanks,
> >>> Susheel
> >>>
> >>>
> >>>
> >>> On Sun, Feb 14, 2016 at 10:51 AM, McCallick, Paul <
> >>> Paul.E.McCallick@nordstrom.com> wrote:
> >>>
> >>>> Then what is the suggested way to add a new node to a collection via
> the
> >>>> apis? I am specifically thinking of autoscale scenarios where a node
> >> has
> >>>> gone down or more nodes are needed to handle load.
> >>>>
> >>>> Note that the ADDREPLICA endpoint requires a shard name, which puts
> the
> >>>> onus of how to scale out on the user. This can be challenging in an
> >>>> autoscale scenario.
> >>>>
> >>>> Thanks,
> >>>> Paul
> >>>>
> >>>>> On Feb 14, 2016, at 12:25 AM, Shawn Heisey <ap...@elyograg.org>
> >> wrote:
> >>>>>
> >>>>>> On 2/13/2016 6:01 PM, McCallick, Paul wrote:
> >>>>>> - When creating a new collection, SOLRCloud will use all available
> >>>> nodes for the collection, adding cores to each. This assumes that you
> >> do
> >>>> not specify a replicationFactor.
> >>>>>
> >>>>> The number of nodes that will be used is numShards multipled by
> >>>>> replicationFactor. The default value for replicationFactor is 1. If
> >>>>> you do not specify numShards, there is no default -- the CREATE call
> >>>>> will fail. The value of maxShardsPerNode can also affect the overall
> >>>>> result.
> >>>>>
> >>>>>> - When adding new nodes to the cluster AFTER the collection is
> >> created,
> >>>> one must use the core admin api to add the node to the collection.
> >>>>>
> >>>>> Using the CoreAdmin API is strongly discouraged when running
> >> SolrCloud.
> >>>>> It works, but it is an expert API when in cloud mode, and can cause
> >>>>> serious problems if not used correctly. Instead, use the Collections
> >>>>> API. It can handle all normal maintenance needs.
> >>>>>
> >>>>>> I would really like to see the second case behave more like the
> >> first.
> >>>> If I add a node to the cluster, it is automatically used as a replica
> >> for
> >>>> existing clusters without my having to do so. This would really
> >> simplify
> >>>> things.
> >>>>>
> >>>>> I've added a FAQ entry to address why this is a bad idea.
> >>
> https://wiki.apache.org/solr/FAQ#Why_doesn.27t_SolrCloud_automatically_create_replicas_when_I_add_nodes.3F
> >>>>>
> >>>>> Thanks,
> >>>>> Shawn
> >>
>
Re: Adding nodes
Posted by Jeff Wartes <jw...@whitepages.com>.
Solrcloud does not come with any autoscaling functionality. If you want such a thing, you’ll need to write it yourself.
https://github.com/whitepages/solrcloud_manager might be a useful head start though, particularly the “fill” and “cleancollection” commands. I don’t do *auto* scaling, but I do use this for all my cluster management, which certantly involves moving collections/shards around among nodes, adding capacity, and removing capacity.
On 2/14/16, 11:17 AM, "McCallick, Paul" <Pa...@nordstrom.com> wrote:
>These are excellent questions and give me a good sense of why you suggest using the collections api.
>
>In our case we have 8 shards of product data with a even distribution of data per shard, no hot spots. We have very different load at different points in the year (cyber monday), and we tend to have very little traffic at night. I'm thinking of two use cases:
>
>1) we are seeing increased latency due to load and want to add 8 more replicas to handle the query volume. Once the volume subsides, we'd remove the nodes.
>
>2) we lose a node due to some unexpected failure (ec2 tends to do this). We want auto scaling to detect the failure and add a node to replace the failed one.
>
>In both cases the core api makes it easy. It adds nodes to the shards evenly. Otherwise we have to write a fairly involved script that is subject to race conditions to determine which shard to add nodes to.
>
>Let me know if I'm making dangerous or uninformed assumptions, as I'm new to solr.
>
>Thanks,
>Paul
>
>> On Feb 14, 2016, at 10:35 AM, Susheel Kumar <su...@gmail.com> wrote:
>>
>> Hi Pual,
>>
>>
>> For Auto-scaling, it depends on how you are thinking to design and what/how
>> do you want to scale. Which scenario you think makes coreadmin API easy to
>> use for a sharded SolrCloud environment?
>>
>> Isn't if in a sharded environment (assume 3 shards A,B & C) and shard B has
>> having higher or more load, then you want to add Replica for shard B to
>> distribute the load or if a particular shard replica goes down then you
>> want to add another Replica back for the shard in which case ADDREPLICA
>> requires a shard name?
>>
>> Can you describe your scenario / provide more detail?
>>
>> Thanks,
>> Susheel
>>
>>
>>
>> On Sun, Feb 14, 2016 at 11:51 AM, McCallick, Paul <
>> Paul.E.McCallick@nordstrom.com> wrote:
>>
>>> Hi all,
>>>
>>>
>>> This doesn’t really answer the following question:
>>>
>>> What is the suggested way to add a new node to a collection via the
>>> apis? I am specifically thinking of autoscale scenarios where a node has
>>> gone down or more nodes are needed to handle load.
>>>
>>>
>>> The coreadmin api makes this easy. The collections api (ADDREPLICA),
>>> makes this very difficult.
>>>
>>>
>>>> On 2/14/16, 8:19 AM, "Susheel Kumar" <su...@gmail.com> wrote:
>>>>
>>>> Hi Paul,
>>>>
>>>> Shawn is referring to use Collections API
>>>> https://cwiki.apache.org/confluence/display/solr/Collections+API than
>>> Core
>>>> Admin API https://cwiki.apache.org/confluence/display/solr/CoreAdmin+API
>>>> for SolrCloud.
>>>>
>>>> Hope that clarifies and you mentioned about ADDREPLICA which is the
>>>> collections API, so you are on right track.
>>>>
>>>> Thanks,
>>>> Susheel
>>>>
>>>>
>>>>
>>>> On Sun, Feb 14, 2016 at 10:51 AM, McCallick, Paul <
>>>> Paul.E.McCallick@nordstrom.com> wrote:
>>>>
>>>>> Then what is the suggested way to add a new node to a collection via the
>>>>> apis? I am specifically thinking of autoscale scenarios where a node
>>> has
>>>>> gone down or more nodes are needed to handle load.
>>>>>
>>>>> Note that the ADDREPLICA endpoint requires a shard name, which puts the
>>>>> onus of how to scale out on the user. This can be challenging in an
>>>>> autoscale scenario.
>>>>>
>>>>> Thanks,
>>>>> Paul
>>>>>
>>>>>> On Feb 14, 2016, at 12:25 AM, Shawn Heisey <ap...@elyograg.org>
>>> wrote:
>>>>>>
>>>>>>> On 2/13/2016 6:01 PM, McCallick, Paul wrote:
>>>>>>> - When creating a new collection, SOLRCloud will use all available
>>>>> nodes for the collection, adding cores to each. This assumes that you
>>> do
>>>>> not specify a replicationFactor.
>>>>>>
>>>>>> The number of nodes that will be used is numShards multipled by
>>>>>> replicationFactor. The default value for replicationFactor is 1. If
>>>>>> you do not specify numShards, there is no default -- the CREATE call
>>>>>> will fail. The value of maxShardsPerNode can also affect the overall
>>>>>> result.
>>>>>>
>>>>>>> - When adding new nodes to the cluster AFTER the collection is
>>> created,
>>>>> one must use the core admin api to add the node to the collection.
>>>>>>
>>>>>> Using the CoreAdmin API is strongly discouraged when running
>>> SolrCloud.
>>>>>> It works, but it is an expert API when in cloud mode, and can cause
>>>>>> serious problems if not used correctly. Instead, use the Collections
>>>>>> API. It can handle all normal maintenance needs.
>>>>>>
>>>>>>> I would really like to see the second case behave more like the
>>> first.
>>>>> If I add a node to the cluster, it is automatically used as a replica
>>> for
>>>>> existing clusters without my having to do so. This would really
>>> simplify
>>>>> things.
>>>>>>
>>>>>> I've added a FAQ entry to address why this is a bad idea.
>>> https://wiki.apache.org/solr/FAQ#Why_doesn.27t_SolrCloud_automatically_create_replicas_when_I_add_nodes.3F
>>>>>>
>>>>>> Thanks,
>>>>>> Shawn
>>>
Re: Adding nodes
Posted by "McCallick, Paul" <Pa...@nordstrom.com>.
These are excellent questions and give me a good sense of why you suggest using the collections api.
In our case we have 8 shards of product data with a even distribution of data per shard, no hot spots. We have very different load at different points in the year (cyber monday), and we tend to have very little traffic at night. I'm thinking of two use cases:
1) we are seeing increased latency due to load and want to add 8 more replicas to handle the query volume. Once the volume subsides, we'd remove the nodes.
2) we lose a node due to some unexpected failure (ec2 tends to do this). We want auto scaling to detect the failure and add a node to replace the failed one.
In both cases the core api makes it easy. It adds nodes to the shards evenly. Otherwise we have to write a fairly involved script that is subject to race conditions to determine which shard to add nodes to.
Let me know if I'm making dangerous or uninformed assumptions, as I'm new to solr.
Thanks,
Paul
> On Feb 14, 2016, at 10:35 AM, Susheel Kumar <su...@gmail.com> wrote:
>
> Hi Pual,
>
>
> For Auto-scaling, it depends on how you are thinking to design and what/how
> do you want to scale. Which scenario you think makes coreadmin API easy to
> use for a sharded SolrCloud environment?
>
> Isn't if in a sharded environment (assume 3 shards A,B & C) and shard B has
> having higher or more load, then you want to add Replica for shard B to
> distribute the load or if a particular shard replica goes down then you
> want to add another Replica back for the shard in which case ADDREPLICA
> requires a shard name?
>
> Can you describe your scenario / provide more detail?
>
> Thanks,
> Susheel
>
>
>
> On Sun, Feb 14, 2016 at 11:51 AM, McCallick, Paul <
> Paul.E.McCallick@nordstrom.com> wrote:
>
>> Hi all,
>>
>>
>> This doesn’t really answer the following question:
>>
>> What is the suggested way to add a new node to a collection via the
>> apis? I am specifically thinking of autoscale scenarios where a node has
>> gone down or more nodes are needed to handle load.
>>
>>
>> The coreadmin api makes this easy. The collections api (ADDREPLICA),
>> makes this very difficult.
>>
>>
>>> On 2/14/16, 8:19 AM, "Susheel Kumar" <su...@gmail.com> wrote:
>>>
>>> Hi Paul,
>>>
>>> Shawn is referring to use Collections API
>>> https://cwiki.apache.org/confluence/display/solr/Collections+API than
>> Core
>>> Admin API https://cwiki.apache.org/confluence/display/solr/CoreAdmin+API
>>> for SolrCloud.
>>>
>>> Hope that clarifies and you mentioned about ADDREPLICA which is the
>>> collections API, so you are on right track.
>>>
>>> Thanks,
>>> Susheel
>>>
>>>
>>>
>>> On Sun, Feb 14, 2016 at 10:51 AM, McCallick, Paul <
>>> Paul.E.McCallick@nordstrom.com> wrote:
>>>
>>>> Then what is the suggested way to add a new node to a collection via the
>>>> apis? I am specifically thinking of autoscale scenarios where a node
>> has
>>>> gone down or more nodes are needed to handle load.
>>>>
>>>> Note that the ADDREPLICA endpoint requires a shard name, which puts the
>>>> onus of how to scale out on the user. This can be challenging in an
>>>> autoscale scenario.
>>>>
>>>> Thanks,
>>>> Paul
>>>>
>>>>> On Feb 14, 2016, at 12:25 AM, Shawn Heisey <ap...@elyograg.org>
>> wrote:
>>>>>
>>>>>> On 2/13/2016 6:01 PM, McCallick, Paul wrote:
>>>>>> - When creating a new collection, SOLRCloud will use all available
>>>> nodes for the collection, adding cores to each. This assumes that you
>> do
>>>> not specify a replicationFactor.
>>>>>
>>>>> The number of nodes that will be used is numShards multipled by
>>>>> replicationFactor. The default value for replicationFactor is 1. If
>>>>> you do not specify numShards, there is no default -- the CREATE call
>>>>> will fail. The value of maxShardsPerNode can also affect the overall
>>>>> result.
>>>>>
>>>>>> - When adding new nodes to the cluster AFTER the collection is
>> created,
>>>> one must use the core admin api to add the node to the collection.
>>>>>
>>>>> Using the CoreAdmin API is strongly discouraged when running
>> SolrCloud.
>>>>> It works, but it is an expert API when in cloud mode, and can cause
>>>>> serious problems if not used correctly. Instead, use the Collections
>>>>> API. It can handle all normal maintenance needs.
>>>>>
>>>>>> I would really like to see the second case behave more like the
>> first.
>>>> If I add a node to the cluster, it is automatically used as a replica
>> for
>>>> existing clusters without my having to do so. This would really
>> simplify
>>>> things.
>>>>>
>>>>> I've added a FAQ entry to address why this is a bad idea.
>> https://wiki.apache.org/solr/FAQ#Why_doesn.27t_SolrCloud_automatically_create_replicas_when_I_add_nodes.3F
>>>>>
>>>>> Thanks,
>>>>> Shawn
>>
Re: Adding nodes
Posted by Susheel Kumar <su...@gmail.com>.
Hi Pual,
For Auto-scaling, it depends on how you are thinking to design and what/how
do you want to scale. Which scenario you think makes coreadmin API easy to
use for a sharded SolrCloud environment?
Isn't if in a sharded environment (assume 3 shards A,B & C) and shard B has
having higher or more load, then you want to add Replica for shard B to
distribute the load or if a particular shard replica goes down then you
want to add another Replica back for the shard in which case ADDREPLICA
requires a shard name?
Can you describe your scenario / provide more detail?
Thanks,
Susheel
On Sun, Feb 14, 2016 at 11:51 AM, McCallick, Paul <
Paul.E.McCallick@nordstrom.com> wrote:
> Hi all,
>
>
> This doesn’t really answer the following question:
>
> What is the suggested way to add a new node to a collection via the
> apis? I am specifically thinking of autoscale scenarios where a node has
> gone down or more nodes are needed to handle load.
>
>
> The coreadmin api makes this easy. The collections api (ADDREPLICA),
> makes this very difficult.
>
>
> On 2/14/16, 8:19 AM, "Susheel Kumar" <su...@gmail.com> wrote:
>
> >Hi Paul,
> >
> >Shawn is referring to use Collections API
> >https://cwiki.apache.org/confluence/display/solr/Collections+API than
> Core
> >Admin API https://cwiki.apache.org/confluence/display/solr/CoreAdmin+API
> >for SolrCloud.
> >
> >Hope that clarifies and you mentioned about ADDREPLICA which is the
> >collections API, so you are on right track.
> >
> >Thanks,
> >Susheel
> >
> >
> >
> >On Sun, Feb 14, 2016 at 10:51 AM, McCallick, Paul <
> >Paul.E.McCallick@nordstrom.com> wrote:
> >
> >> Then what is the suggested way to add a new node to a collection via the
> >> apis? I am specifically thinking of autoscale scenarios where a node
> has
> >> gone down or more nodes are needed to handle load.
> >>
> >> Note that the ADDREPLICA endpoint requires a shard name, which puts the
> >> onus of how to scale out on the user. This can be challenging in an
> >> autoscale scenario.
> >>
> >> Thanks,
> >> Paul
> >>
> >> > On Feb 14, 2016, at 12:25 AM, Shawn Heisey <ap...@elyograg.org>
> wrote:
> >> >
> >> >> On 2/13/2016 6:01 PM, McCallick, Paul wrote:
> >> >> - When creating a new collection, SOLRCloud will use all available
> >> nodes for the collection, adding cores to each. This assumes that you
> do
> >> not specify a replicationFactor.
> >> >
> >> > The number of nodes that will be used is numShards multipled by
> >> > replicationFactor. The default value for replicationFactor is 1. If
> >> > you do not specify numShards, there is no default -- the CREATE call
> >> > will fail. The value of maxShardsPerNode can also affect the overall
> >> > result.
> >> >
> >> >> - When adding new nodes to the cluster AFTER the collection is
> created,
> >> one must use the core admin api to add the node to the collection.
> >> >
> >> > Using the CoreAdmin API is strongly discouraged when running
> SolrCloud.
> >> > It works, but it is an expert API when in cloud mode, and can cause
> >> > serious problems if not used correctly. Instead, use the Collections
> >> > API. It can handle all normal maintenance needs.
> >> >
> >> >> I would really like to see the second case behave more like the
> first.
> >> If I add a node to the cluster, it is automatically used as a replica
> for
> >> existing clusters without my having to do so. This would really
> simplify
> >> things.
> >> >
> >> > I've added a FAQ entry to address why this is a bad idea.
> >> >
> >> >
> >>
> https://wiki.apache.org/solr/FAQ#Why_doesn.27t_SolrCloud_automatically_create_replicas_when_I_add_nodes.3F
> >> >
> >> > Thanks,
> >> > Shawn
> >> >
> >>
>
Re: Adding nodes
Posted by "McCallick, Paul" <Pa...@nordstrom.com>.
Hi all,
This doesn’t really answer the following question:
What is the suggested way to add a new node to a collection via the
apis? I am specifically thinking of autoscale scenarios where a node has
gone down or more nodes are needed to handle load.
The coreadmin api makes this easy. The collections api (ADDREPLICA), makes this very difficult.
On 2/14/16, 8:19 AM, "Susheel Kumar" <su...@gmail.com> wrote:
>Hi Paul,
>
>Shawn is referring to use Collections API
>https://cwiki.apache.org/confluence/display/solr/Collections+API than Core
>Admin API https://cwiki.apache.org/confluence/display/solr/CoreAdmin+API
>for SolrCloud.
>
>Hope that clarifies and you mentioned about ADDREPLICA which is the
>collections API, so you are on right track.
>
>Thanks,
>Susheel
>
>
>
>On Sun, Feb 14, 2016 at 10:51 AM, McCallick, Paul <
>Paul.E.McCallick@nordstrom.com> wrote:
>
>> Then what is the suggested way to add a new node to a collection via the
>> apis? I am specifically thinking of autoscale scenarios where a node has
>> gone down or more nodes are needed to handle load.
>>
>> Note that the ADDREPLICA endpoint requires a shard name, which puts the
>> onus of how to scale out on the user. This can be challenging in an
>> autoscale scenario.
>>
>> Thanks,
>> Paul
>>
>> > On Feb 14, 2016, at 12:25 AM, Shawn Heisey <ap...@elyograg.org> wrote:
>> >
>> >> On 2/13/2016 6:01 PM, McCallick, Paul wrote:
>> >> - When creating a new collection, SOLRCloud will use all available
>> nodes for the collection, adding cores to each. This assumes that you do
>> not specify a replicationFactor.
>> >
>> > The number of nodes that will be used is numShards multipled by
>> > replicationFactor. The default value for replicationFactor is 1. If
>> > you do not specify numShards, there is no default -- the CREATE call
>> > will fail. The value of maxShardsPerNode can also affect the overall
>> > result.
>> >
>> >> - When adding new nodes to the cluster AFTER the collection is created,
>> one must use the core admin api to add the node to the collection.
>> >
>> > Using the CoreAdmin API is strongly discouraged when running SolrCloud.
>> > It works, but it is an expert API when in cloud mode, and can cause
>> > serious problems if not used correctly. Instead, use the Collections
>> > API. It can handle all normal maintenance needs.
>> >
>> >> I would really like to see the second case behave more like the first.
>> If I add a node to the cluster, it is automatically used as a replica for
>> existing clusters without my having to do so. This would really simplify
>> things.
>> >
>> > I've added a FAQ entry to address why this is a bad idea.
>> >
>> >
>> https://wiki.apache.org/solr/FAQ#Why_doesn.27t_SolrCloud_automatically_create_replicas_when_I_add_nodes.3F
>> >
>> > Thanks,
>> > Shawn
>> >
>>
Re: Adding nodes
Posted by Susheel Kumar <su...@gmail.com>.
Hi Paul,
Shawn is referring to use Collections API
https://cwiki.apache.org/confluence/display/solr/Collections+API than Core
Admin API https://cwiki.apache.org/confluence/display/solr/CoreAdmin+API
for SolrCloud.
Hope that clarifies and you mentioned about ADDREPLICA which is the
collections API, so you are on right track.
Thanks,
Susheel
On Sun, Feb 14, 2016 at 10:51 AM, McCallick, Paul <
Paul.E.McCallick@nordstrom.com> wrote:
> Then what is the suggested way to add a new node to a collection via the
> apis? I am specifically thinking of autoscale scenarios where a node has
> gone down or more nodes are needed to handle load.
>
> Note that the ADDREPLICA endpoint requires a shard name, which puts the
> onus of how to scale out on the user. This can be challenging in an
> autoscale scenario.
>
> Thanks,
> Paul
>
> > On Feb 14, 2016, at 12:25 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> >
> >> On 2/13/2016 6:01 PM, McCallick, Paul wrote:
> >> - When creating a new collection, SOLRCloud will use all available
> nodes for the collection, adding cores to each. This assumes that you do
> not specify a replicationFactor.
> >
> > The number of nodes that will be used is numShards multipled by
> > replicationFactor. The default value for replicationFactor is 1. If
> > you do not specify numShards, there is no default -- the CREATE call
> > will fail. The value of maxShardsPerNode can also affect the overall
> > result.
> >
> >> - When adding new nodes to the cluster AFTER the collection is created,
> one must use the core admin api to add the node to the collection.
> >
> > Using the CoreAdmin API is strongly discouraged when running SolrCloud.
> > It works, but it is an expert API when in cloud mode, and can cause
> > serious problems if not used correctly. Instead, use the Collections
> > API. It can handle all normal maintenance needs.
> >
> >> I would really like to see the second case behave more like the first.
> If I add a node to the cluster, it is automatically used as a replica for
> existing clusters without my having to do so. This would really simplify
> things.
> >
> > I've added a FAQ entry to address why this is a bad idea.
> >
> >
> https://wiki.apache.org/solr/FAQ#Why_doesn.27t_SolrCloud_automatically_create_replicas_when_I_add_nodes.3F
> >
> > Thanks,
> > Shawn
> >
>
Re: Adding nodes
Posted by "McCallick, Paul" <Pa...@nordstrom.com>.
Then what is the suggested way to add a new node to a collection via the apis? I am specifically thinking of autoscale scenarios where a node has gone down or more nodes are needed to handle load.
Note that the ADDREPLICA endpoint requires a shard name, which puts the onus of how to scale out on the user. This can be challenging in an autoscale scenario.
Thanks,
Paul
> On Feb 14, 2016, at 12:25 AM, Shawn Heisey <ap...@elyograg.org> wrote:
>
>> On 2/13/2016 6:01 PM, McCallick, Paul wrote:
>> - When creating a new collection, SOLRCloud will use all available nodes for the collection, adding cores to each. This assumes that you do not specify a replicationFactor.
>
> The number of nodes that will be used is numShards multipled by
> replicationFactor. The default value for replicationFactor is 1. If
> you do not specify numShards, there is no default -- the CREATE call
> will fail. The value of maxShardsPerNode can also affect the overall
> result.
>
>> - When adding new nodes to the cluster AFTER the collection is created, one must use the core admin api to add the node to the collection.
>
> Using the CoreAdmin API is strongly discouraged when running SolrCloud.
> It works, but it is an expert API when in cloud mode, and can cause
> serious problems if not used correctly. Instead, use the Collections
> API. It can handle all normal maintenance needs.
>
>> I would really like to see the second case behave more like the first. If I add a node to the cluster, it is automatically used as a replica for existing clusters without my having to do so. This would really simplify things.
>
> I've added a FAQ entry to address why this is a bad idea.
>
> https://wiki.apache.org/solr/FAQ#Why_doesn.27t_SolrCloud_automatically_create_replicas_when_I_add_nodes.3F
>
> Thanks,
> Shawn
>
Re: Adding nodes
Posted by Shawn Heisey <ap...@elyograg.org>.
On 2/13/2016 6:01 PM, McCallick, Paul wrote:
> - When creating a new collection, SOLRCloud will use all available nodes for the collection, adding cores to each. This assumes that you do not specify a replicationFactor.
The number of nodes that will be used is numShards multipled by
replicationFactor. The default value for replicationFactor is 1. If
you do not specify numShards, there is no default -- the CREATE call
will fail. The value of maxShardsPerNode can also affect the overall
result.
> - When adding new nodes to the cluster AFTER the collection is created, one must use the core admin api to add the node to the collection.
Using the CoreAdmin API is strongly discouraged when running SolrCloud.
It works, but it is an expert API when in cloud mode, and can cause
serious problems if not used correctly. Instead, use the Collections
API. It can handle all normal maintenance needs.
> I would really like to see the second case behave more like the first. If I add a node to the cluster, it is automatically used as a replica for existing clusters without my having to do so. This would really simplify things.
I've added a FAQ entry to address why this is a bad idea.
https://wiki.apache.org/solr/FAQ#Why_doesn.27t_SolrCloud_automatically_create_replicas_when_I_add_nodes.3F
Thanks,
Shawn