You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Shawn Heisey (JIRA)" <ji...@apache.org> on 2015/03/06 00:24:40 UTC
[jira] [Updated] (SOLR-7191) Improve stability and startup
performance of SolrCloud with thousands of collections
[ https://issues.apache.org/jira/browse/SOLR-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shawn Heisey updated SOLR-7191:
-------------------------------
Attachment: lots-of-zkstatereader-updates-branch_5x.log
Above I mentioned a full recovery that I expected to take half a day, testing with the 5.0.0 release. I killed that and started over AGAIN, this time with branch_5x.
I'm attaching a log excerpt from branch_5x (during the creation of 4000 collections) showing what happens between the receipt of two CREATE calls to the collections admin.
After the first create call, all the stuff for the create happens, and then a massive number of ZkStateReader lines are logged, indicating that an update is happening. The collection (mycoll3221) that was created would be the 3223rd collection on the system, and the log shows 3198 update lines from zkStateReader, then the CREATE call for the next collection.
This is happening on EVERY collection create. Why would every existing collection need to be touched when a new collection is created?
> Improve stability and startup performance of SolrCloud with thousands of collections
> ------------------------------------------------------------------------------------
>
> Key: SOLR-7191
> URL: https://issues.apache.org/jira/browse/SOLR-7191
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
> Affects Versions: 5.0
> Reporter: Shawn Heisey
> Labels: performance, scalability
> Attachments: lots-of-zkstatereader-updates-branch_5x.log
>
>
> A user on the mailing list with thousands of collections (5000 on 4.10.3, 4000 on 5.0) is having severe problems with getting Solr to restart.
> I tried as hard as I could to duplicate the user setup, but I ran into many problems myself even before I was able to get 4000 collections created on a 5.0 example cloud setup. Restarting Solr takes a very long time, and it is not very stable once it's up and running.
> This kind of setup is very much pushing the envelope on SolrCloud performance and scalability. It doesn't help that I'm running both Solr nodes on one machine (I started with 'bin/solr -e cloud') and that ZK is embedded.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org