SolrCloud across Amazon Regions?

[William Bell <bi...@gmail.com>]

Here is a weird architecture...

We have a SOLR Master today, next to our database in our local data center.
We do Solr replication to Amazon East and West coasts once the index is
completed.

We would like to deploy Solr Cloud while leaving the master in place.

1. Local has SQL Server, and Solr Master.
2. We would like to replicate to EAST 2 machines, and WEST 2 machines.
3. The 2 machines on the East coast - we want to increase to 3, and use Zoo
Keeper to keep the configs in sync with the Master local. And we would like
to use Sharding/Collections to split the core into a collection across 3
machines for performance.

Indexing can stay local.

I understand that this is not normal, and maybe we can still use Master
replication with Sharinding/or Solr Cloud ? ideas?

-- 
Bill Bell
billnbell@gmail.com
cell 720-256-8076

Re: SolrCloud across Amazon Regions?

[Erick Erickson <er...@gmail.com>]

Bill:

We're working on CDCR, Cross Data Center Replication, see:
https://issues.apache.org/jira/browse/SOLR-6273

Not entirely sure this pertains, but it might be something that's useful
in this situation. Essentially you'd have two separate clusters, one
East and one West. This is "active passive" in the sense that all
indexing should be done to one DC with the "remote" dc getting
updates  from the "one source of truth DC". Some lag of course.

This functionality is in production in a 4.x version. We're currently
having some trouble with making it pass all tests on trunk, not quite
sure what the heck is going on. My _guess_ is that there's something
about the test framework that's wonky, or perhaps the tests are exposing
an edge case. I'm sure we'll figure it out, any help appreciated
(hint, hint).....

Best
Erick

On Sun, Jun 7, 2015 at 12:31 PM, William Bell <bi...@gmail.com> wrote:
> Here is a weird architecture...
>
> We have a SOLR Master today, next to our database in our local data center.
> We do Solr replication to Amazon East and West coasts once the index is
> completed.
>
> We would like to deploy Solr Cloud while leaving the master in place.
>
> 1. Local has SQL Server, and Solr Master.
> 2. We would like to replicate to EAST 2 machines, and WEST 2 machines.
> 3. The 2 machines on the East coast - we want to increase to 3, and use Zoo
> Keeper to keep the configs in sync with the Master local. And we would like
> to use Sharding/Collections to split the core into a collection across 3
> machines for performance.
>
> Indexing can stay local.
>
> I understand that this is not normal, and maybe we can still use Master
> replication with Sharinding/or Solr Cloud ? ideas?
>
> --
> Bill Bell
> billnbell@gmail.com
> cell 720-256-8076

Re: SolrCloud across Amazon Regions?

[Walter Underwood <wu...@wunderwood.org>]

I would run two independent Solr Cloud clusters and send the data to them reliably through something like Amazon SQS.

Splitting a Zookeeper ensemble across two regions (or AZs) makes a single point of failure. One region will have the majority, and if that fails, you’re dead. If you split across three regions (or AZs), the ensemble can survive a single failure of any of them.

wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/  (my blog)

On Jun 7, 2015, at 12:31 PM, William Bell <bi...@gmail.com> wrote:

> Here is a weird architecture...
> 
> We have a SOLR Master today, next to our database in our local data center.
> We do Solr replication to Amazon East and West coasts once the index is
> completed.
> 
> We would like to deploy Solr Cloud while leaving the master in place.
> 
> 1. Local has SQL Server, and Solr Master.
> 2. We would like to replicate to EAST 2 machines, and WEST 2 machines.
> 3. The 2 machines on the East coast - we want to increase to 3, and use Zoo
> Keeper to keep the configs in sync with the Master local. And we would like
> to use Sharding/Collections to split the core into a collection across 3
> machines for performance.
> 
> Indexing can stay local.
> 
> I understand that this is not normal, and maybe we can still use Master
> replication with Sharinding/or Solr Cloud ? ideas?
> 
> -- 
> Bill Bell
> billnbell@gmail.com
> cell 720-256-8076