You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Adrian Liew <ad...@avanade.com> on 2015/10/01 12:43:26 UTC
Create Collection in Solr Cloud using Solr 5.3.0 giving timeout
issues
Hi there,
I am encountering an issue whereby I am facing a timeout when attempting to create a collection in SolrCloud by linking it to an existing configuration called sitecore_common_configs uploaded to the Zookeeper cluster.
E:\solr-5.3.0\bin>solr.cmd create_collection -c sitecore_core_index -n sitecore_
common_configs -shards 1 -replicationFactor 3
Connecting to ZooKeeper at 10.0.0.4:2181,10.0.0.5:2182,10.0.0.6:2183 ...
Re-using existing configuration directory sitecore_common_configs
Creating new collection 'sitecore_core_index' using command:
http://localhost:8983/solr/admin/collections?action=CREATE&name=sitecore_core_in
dex&numShards=1&replicationFactor=3&maxShardsPerNode=2&collection.configName=sit
ecore_common_configs
ERROR: Failed to create collection 'sitecore_core_index' due to: create the coll
ection time out:180s
I have setup a Solr Cluster in Azure. Currently, I have setup three VMs with one Solr 5.3.0 instance and one Zookeeper instance in each VM for minimal HA. I can verify that a quorum is now setup between all (3) VMs with 1 leader and 2 followers. Addiitionally, I can also see the uploaded configurations (sitecore_common_configs/conf/solrconfig.xml, sitecore_common_configs/conf/schema.xml, sitecore_common_configs/conf/currency.xml, sitecore_common_configs/conf/synonyms.txt etc.) by running the following on each server:
zkcli.bat -z 10.0.0.4:2181 -cmd list
zkcli.bat -z 10.0.0.5:2182 -cmd list
zkcli.bat -z 10.0.0.6:2183 -cmd list
When attempting to run the create_collection command as above, I get the timeout issue. Anyone able to share?
Regards,
Adrian
Re: Create Collection in Solr Cloud using Solr 5.3.0 giving timeout
issues
Posted by Shawn Heisey <ap...@elyograg.org>.
On 10/1/2015 9:26 AM, Adrian Liew wrote:
> Thanks for that. You did mention about starting out with empty collections and already I am experiencing timeout issues. Could this have to do with the hardware or server spec sizing itself. For example, lack of memory allocated, network issues etc. that can possibly cause this? Given Azure has a 99.95 percent SLA, I don't think this could be a network issue.
>
> I am currently using a 4 core 7 GB RAM memory machine for an individual Solr Server.
>
> I don't quite understand why this is happening as I am just trying to setup a bare bones Solr Cloud setup using Solr 5.3.0 and Zookeeper 3.4.6.
>
> Any tips will be much appreciated.
Since I don't know anything about your install except for the number of
CPU cores and RAM, I can only give you general information.
One problem that can plague new installs is that the default Java heap
size for a Solr 5.x install is 512MB. This works great when you first
fire it up, but as you add data, quickly becomes very limiting, and
needs to be increased. The GC pauses that occur when the heap is too
small can be extreme.
Since you are on Windows, there is no install script. Adding something
like "-m 3g" to the startup commandline will allocate more memory (3GB
for my example) to the Java heap. Note that if your index data gets
very big, your VM might need more memory in order for OS disk caching to
be effective.
Thanks,
Shawn
RE: Create Collection in Solr Cloud using Solr 5.3.0 giving timeout
issues
Posted by Adrian Liew <ad...@avanade.com>.
Hi Shawn,
Thanks for that. You did mention about starting out with empty collections and already I am experiencing timeout issues. Could this have to do with the hardware or server spec sizing itself. For example, lack of memory allocated, network issues etc. that can possibly cause this? Given Azure has a 99.95 percent SLA, I don't think this could be a network issue.
I am currently using a 4 core 7 GB RAM memory machine for an individual Solr Server.
I don't quite understand why this is happening as I am just trying to setup a bare bones Solr Cloud setup using Solr 5.3.0 and Zookeeper 3.4.6.
Any tips will be much appreciated.
Best regards,
Adrian
-----Original Message-----
From: Shawn Heisey [mailto:apache@elyograg.org]
Sent: Thursday, October 1, 2015 11:12 PM
To: solr-user@lucene.apache.org
Subject: Re: Create Collection in Solr Cloud using Solr 5.3.0 giving timeout issues
On 10/1/2015 4:43 AM, Adrian Liew wrote:
> E:\solr-5.3.0\bin>solr.cmd create_collection -c sitecore_core_index -n
> sitecore_ common_configs -shards 1 -replicationFactor 3
>
> Connecting to ZooKeeper at 10.0.0.4:2181,10.0.0.5:2182,10.0.0.6:2183 ...
> Re-using existing configuration directory sitecore_common_configs
>
> Creating new collection 'sitecore_core_index' using command:
> http://localhost:8983/solr/admin/collections?action=CREATE&name=siteco
> re_core_in
> dex&numShards=1&replicationFactor=3&maxShardsPerNode=2&collection.conf
> igName=sit
> ecore_common_configs
>
> ERROR: Failed to create collection 'sitecore_core_index' due to:
> create the coll ection time out:180s
The timeout, as it mentions, is 180 seconds, or three minutes. This is a the default timeout for the Collections API, and it is a particularly long timeout. When it is exceeded, it is usually an indication of a serious problem. The collection create will likely succeed eventually, after an unknown amount of time ... the collections API just gave up on waiting for the response.
There are two things that I know of that can cause this: A very large number of collections, and general performance issues.
I did some testing a while back with thousands of empty collections on the Solr 5.x cloud example. It did not turn out well. Many things timed out, and a server restart would throw the whole cloud into chaos for a very long time. If those collections were not empty, then I suspect the problems would be even worse.
General performance issues (usually RAM-related) can cause big problems with SolrCloud too. The following wiki page is most of my accumulated knowledge about what causes performance problems with Solr:
https://wiki.apache.org/solr/SolrPerformanceProblems
Thanks,
Shawn
Re: Create Collection in Solr Cloud using Solr 5.3.0 giving timeout
issues
Posted by Shawn Heisey <ap...@elyograg.org>.
On 10/1/2015 4:43 AM, Adrian Liew wrote:
> E:\solr-5.3.0\bin>solr.cmd create_collection -c sitecore_core_index -n sitecore_
> common_configs -shards 1 -replicationFactor 3
>
> Connecting to ZooKeeper at 10.0.0.4:2181,10.0.0.5:2182,10.0.0.6:2183 ...
> Re-using existing configuration directory sitecore_common_configs
>
> Creating new collection 'sitecore_core_index' using command:
> http://localhost:8983/solr/admin/collections?action=CREATE&name=sitecore_core_in
> dex&numShards=1&replicationFactor=3&maxShardsPerNode=2&collection.configName=sit
> ecore_common_configs
>
> ERROR: Failed to create collection 'sitecore_core_index' due to: create the coll
> ection time out:180s
The timeout, as it mentions, is 180 seconds, or three minutes. This is
a the default timeout for the Collections API, and it is a particularly
long timeout. When it is exceeded, it is usually an indication of a
serious problem. The collection create will likely succeed eventually,
after an unknown amount of time ... the collections API just gave up on
waiting for the response.
There are two things that I know of that can cause this: A very large
number of collections, and general performance issues.
I did some testing a while back with thousands of empty collections on
the Solr 5.x cloud example. It did not turn out well. Many things
timed out, and a server restart would throw the whole cloud into chaos
for a very long time. If those collections were not empty, then I
suspect the problems would be even worse.
General performance issues (usually RAM-related) can cause big problems
with SolrCloud too. The following wiki page is most of my accumulated
knowledge about what causes performance problems with Solr:
https://wiki.apache.org/solr/SolrPerformanceProblems
Thanks,
Shawn