You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Hendrik Haddorp <he...@gmx.net> on 2019/08/29 05:31:01 UTC
Re: Question: Solr perform well with thousands of replicas?
Hi,
we are usually using Solr Clouds with 5 nodes and up to 2000 collections
and a replication factor of 2. So we have close to 1000 cores per node.
That is on Solr 7.6 but I believe 7.3 worked as well. We tuned a few
caches down to a minimum as otherwise the memory usage goes up a lot.
The Solr UI is having some problems with a high number of collections,
like lots of timeouts when loading the status.
Older Solr versions had problem with the overseer queue in ZooKeeper. If
you restarted too many nodes at once then the queue got too long and
Solr died and required some help and cleanup to start at all again.
regards,
Hendrik
On 29.08.19 05:27, Hongxu Ma wrote:
> Hi
> I have a solr-cloud cluster, but it's unstable when collection number is big: 1000 replica/core per solr node.
>
> To solve this issue, I have read the performance guide:
> https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems
>
> I noted there is a sentence on solr-cloud section:
> "Recent Solr versions perform well with thousands of replicas."
>
> I want to know does it mean a single solr node can handle thousands of replicas? or a solr cluster can (if so, what's the size of the cluster?)
>
> My solr version is 7.3.1 and 6.6.2 (looks they are the same in performance)
>
> Thanks for you help.
>
>
Re: Question: Solr perform well with thousands of replicas?
Posted by Erick Erickson <er...@gmail.com>.
There are two factors:
1> the raw number of replicas on a Solr node.
2> total resources Solr needs.
You say “..it’s unstalble…”. _How_ is it unstable? What symptoms are you seeing?
You might want to review: https://cwiki.apache.org/confluence/display/solr/UsingMailingLists
And not as you add more cores, you put more pressure on memory, I/O, etc. So
whether it’s the raw number of cores or you’re just exhausting memory, overloading
your CPU, etc. is hard to say without more information.
Best,
Erick
> On Aug 29, 2019, at 1:31 AM, Hendrik Haddorp <he...@gmx.net> wrote:
>
> Hi,
>
> we are usually using Solr Clouds with 5 nodes and up to 2000 collections
> and a replication factor of 2. So we have close to 1000 cores per node.
> That is on Solr 7.6 but I believe 7.3 worked as well. We tuned a few
> caches down to a minimum as otherwise the memory usage goes up a lot.
> The Solr UI is having some problems with a high number of collections,
> like lots of timeouts when loading the status.
>
> Older Solr versions had problem with the overseer queue in ZooKeeper. If
> you restarted too many nodes at once then the queue got too long and
> Solr died and required some help and cleanup to start at all again.
>
> regards,
> Hendrik
>
> On 29.08.19 05:27, Hongxu Ma wrote:
>> Hi
>> I have a solr-cloud cluster, but it's unstable when collection number is big: 1000 replica/core per solr node.
>>
>> To solve this issue, I have read the performance guide:
>> https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems
>>
>> I noted there is a sentence on solr-cloud section:
>> "Recent Solr versions perform well with thousands of replicas."
>>
>> I want to know does it mean a single solr node can handle thousands of replicas? or a solr cluster can (if so, what's the size of the cluster?)
>>
>> My solr version is 7.3.1 and 6.6.2 (looks they are the same in performance)
>>
>> Thanks for you help.
>>
>>
>