SolrCloud sharding. Replicas on different AZ

[Sergio García Maroto <ma...@gmail.com>]

Hi all,

I am trying to migrate from a SolrCloud Cluster which has 1 shard and
replication factor of 3. Each server is on a different Availability Zone
(AZ) on AWS.
In prod today we can run on 1 AZ even if the other 2 can be down.

My new requirement is basically been able to scale horizontally with
sharding and still keep the capacity of running in prod when only 1 AZ is
available out of 3.
For that I created 8 shards in 24 servers with replication factor of 3. My
question is?

Can i locate the shards in specific servers so I can have at least  each
shard replicated across the 3 different AZ's.


Thanks a lot for you help.

Sergio Maroto

Re: SolrCloud sharding. Replicas on different AZ

[Jan Høydahl <ja...@cominvent.com>]

Yes, have a look at https://solr.apache.org/guide/solr/latest/configuration-guide/replica-placement-plugins.html

Jan Høydahl

> 4. okt. 2023 kl. 15:48 skrev ufuk yılmaz <uy...@vivaldi.net.invalid>:
> 
> You should be able to specify exact replica locations when creating a collection. I don’t remember the syntax but you can check collections api addcollection command from Solr documentation
> 
> -ufuk
> 
> —
> 
>> On 4 Oct 2023, at 12:58, Sergio García Maroto <ma...@gmail.com> wrote:
>> 
>> Hi all,
>> 
>> I am trying to migrate from a SolrCloud Cluster which has 1 shard and
>> replication factor of 3. Each server is on a different Availability Zone
>> (AZ) on AWS.
>> In prod today we can run on 1 AZ even if the other 2 can be down.
>> 
>> My new requirement is basically been able to scale horizontally with
>> sharding and still keep the capacity of running in prod when only 1 AZ is
>> available out of 3.
>> For that I created 8 shards in 24 servers with replication factor of 3. My
>> question is?
>> 
>> Can i locate the shards in specific servers so I can have at least  each
>> shard replicated across the 3 different AZ's.
>> 
>> 
>> Thanks a lot for you help.
>> 
>> Sergio Maroto
> 

Re: SolrCloud sharding. Replicas on different AZ

[ufuk yılmaz <uy...@vivaldi.net.INVALID>]

You should be able to specify exact replica locations when creating a collection. I don’t remember the syntax but you can check collections api addcollection command from Solr documentation 

-ufuk

—

> On 4 Oct 2023, at 12:58, Sergio García Maroto <ma...@gmail.com> wrote:
> 
> Hi all,
> 
> I am trying to migrate from a SolrCloud Cluster which has 1 shard and
> replication factor of 3. Each server is on a different Availability Zone
> (AZ) on AWS.
> In prod today we can run on 1 AZ even if the other 2 can be down.
> 
> My new requirement is basically been able to scale horizontally with
> sharding and still keep the capacity of running in prod when only 1 AZ is
> available out of 3.
> For that I created 8 shards in 24 servers with replication factor of 3. My
> question is?
> 
> Can i locate the shards in specific servers so I can have at least  each
> shard replicated across the 3 different AZ's.
> 
> 
> Thanks a lot for you help.
> 
> Sergio Maroto

Re: SolrCloud sharding. Replicas on different AZ

[Shawn Heisey <el...@elyograg.org.INVALID>]

On 10/4/23 03:58, Sergio García Maroto wrote:
> I am trying to migrate from a SolrCloud Cluster which has 1 shard and
> replication factor of 3. Each server is on a different Availability Zone
> (AZ) on AWS.
> In prod today we can run on 1 AZ even if the other 2 can be down.

Something to keep in mind:

Solr itself might function with only one, but if it does, it will be 
read-only because Zookeeper cannot be set up to function properly in an 
event where two out of three data centers go down.  To survive two 
failures, you would need to have five ZK servers, each in a different AZ.

Thanks,
Shawn