You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Nawab Zada Asad Iqbal <kh...@gmail.com> on 2018/07/31 23:12:11 UTC
SolrCloud: Different replicationFactor for different shards in same collection
Hi,
I am looking at Solr 7.x and couldn't find an answer in the documentation.
Is it possibly to specify different replicationFactor for different shards
in same collection? E.g. if a certain shard is receiving more queries than
rest of the collection I would like to add more replicas for it to help
with the query load.
Thanks
Nawab
Re: SolrCloud: Different replicationFactor for different shards in
same collection
Posted by Erick Erickson <er...@gmail.com>.
This feels like more work than necessary, especially the bit:
"which will require modification in Solr code".
If your needs are to co-locate various groups of documents
on specific nodes, composite id (the default) routing has
the ability to cluster docs together, see:
https://lucene.apache.org/solr/guide/6_6/shards-and-indexing-data-in-solrcloud.html
the "document routing" section. You can also route queries to
those shards only, see:
https://lucidworks.com/2013/06/13/solr-cloud-document-routing/
If that isn't sufficient, using "implicit" routing allows you to
send document to specific shards.
True, in both cases the _client_ has to assign the doc to a particular
shard based on whatever criteria you need, but that seems like less
work than changing Solr code.
Best,
Erick
On Tue, Jul 31, 2018 at 5:20 PM, Nawab Zada Asad Iqbal <kh...@gmail.com> wrote:
> Thanks Erick
>
>
> This is for future. I am exploring to use a custom sharding scheme (which
> will require modification in Solr code) together with the benefits of
> SolrCloud.
>
>
>
> Thanks
> Nawab
>
>
>
> On Tue, Jul 31, 2018 at 4:51 PM, Erick Erickson <er...@gmail.com>
> wrote:
>
>> Sure, just use the Collections API ADDREPLICA command to add as many
>> replicas for specific shards as you want. There's no way to specify
>> that at creation time though.
>>
>> Some of the new autoscaling can do this automatically I believe.
>>
>> I have to ask what it is about your collection that this is true. If
>> you're using the default composite id routing having one shard get
>> substantially more queries than the others is unexpected.
>>
>> If you're using implicit routing then it's perfectly understandable.
>>
>> Best,
>> Erick
>>
>> On Tue, Jul 31, 2018 at 4:12 PM, Nawab Zada Asad Iqbal <kh...@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > I am looking at Solr 7.x and couldn't find an answer in the
>> documentation.
>> > Is it possibly to specify different replicationFactor for different
>> shards
>> > in same collection? E.g. if a certain shard is receiving more queries
>> than
>> > rest of the collection I would like to add more replicas for it to help
>> > with the query load.
>> >
>> >
>> >
>> > Thanks
>> > Nawab
>>
Re: SolrCloud: Different replicationFactor for different shards in
same collection
Posted by Nawab Zada Asad Iqbal <kh...@gmail.com>.
Thanks Erick
This is for future. I am exploring to use a custom sharding scheme (which
will require modification in Solr code) together with the benefits of
SolrCloud.
Thanks
Nawab
On Tue, Jul 31, 2018 at 4:51 PM, Erick Erickson <er...@gmail.com>
wrote:
> Sure, just use the Collections API ADDREPLICA command to add as many
> replicas for specific shards as you want. There's no way to specify
> that at creation time though.
>
> Some of the new autoscaling can do this automatically I believe.
>
> I have to ask what it is about your collection that this is true. If
> you're using the default composite id routing having one shard get
> substantially more queries than the others is unexpected.
>
> If you're using implicit routing then it's perfectly understandable.
>
> Best,
> Erick
>
> On Tue, Jul 31, 2018 at 4:12 PM, Nawab Zada Asad Iqbal <kh...@gmail.com>
> wrote:
> > Hi,
> >
> > I am looking at Solr 7.x and couldn't find an answer in the
> documentation.
> > Is it possibly to specify different replicationFactor for different
> shards
> > in same collection? E.g. if a certain shard is receiving more queries
> than
> > rest of the collection I would like to add more replicas for it to help
> > with the query load.
> >
> >
> >
> > Thanks
> > Nawab
>
Re: SolrCloud: Different replicationFactor for different shards in
same collection
Posted by Erick Erickson <er...@gmail.com>.
Sure, just use the Collections API ADDREPLICA command to add as many
replicas for specific shards as you want. There's no way to specify
that at creation time though.
Some of the new autoscaling can do this automatically I believe.
I have to ask what it is about your collection that this is true. If
you're using the default composite id routing having one shard get
substantially more queries than the others is unexpected.
If you're using implicit routing then it's perfectly understandable.
Best,
Erick
On Tue, Jul 31, 2018 at 4:12 PM, Nawab Zada Asad Iqbal <kh...@gmail.com> wrote:
> Hi,
>
> I am looking at Solr 7.x and couldn't find an answer in the documentation.
> Is it possibly to specify different replicationFactor for different shards
> in same collection? E.g. if a certain shard is receiving more queries than
> rest of the collection I would like to add more replicas for it to help
> with the query load.
>
>
>
> Thanks
> Nawab