You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Aman Tandon <am...@gmail.com> on 2015/03/07 17:28:19 UTC
how to change configurations in solrcloud setup
Hi,
Please tell me what is best way to apply configuration changes in solr
cloud and how to do that.
Thanks in advance.
With Regards
Aman Tandon
Re: how to change configurations in solrcloud setup
Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/11/2015 10:45 PM, Aman Tandon wrote:
>> You may need to manually remove the 127.0.1.1 entries from zookeeper
>> after you fix the IP address problem.
>
>
> How to do that?
The zkcli script included with Solr should have everything you need --
getfile, putfile, and clear ... but that would be a rather frustrating
way to handle it. You won't be able to accomplish your goal by only
deleting znodes, you'll have to edit some json structures and replace
them in zookeeper. The main thing you'll need to edit is the
clusterstate.json ... this is a single "file" in Solr 4.x, in 5.0 it has
changed to a clusterstate for every collection.
There are not very many GUI clients for zookeeper. The only one that
I've really found is the one that is a plugin for eclipse. I happen to
use eclipse, so this is fairly convenient for me:
http://www.massedynamic.org/mediawiki/index.php?title=Eclipse_Plug-in_for_ZooKeeper
Thanks,
Shawn
Re: how to change configurations in solrcloud setup
Posted by Aman Tandon <am...@gmail.com>.
Hi Shawn,
As suggested i gave the -Dhost=192.168.5.236 in command line for the server
which was showing 127.0.0.1.
*./solr start -c -z 192.168.6.217:2181
<http://192.168.6.217:2181>,192.168.5.81:2181
<http://192.168.5.81:2181>,192.168.5.236:2181 <http://192.168.5.236:2181>
-p 4567 -Dhost=192.168.5.236 -V*
Now everything is good. Thank you so much.
You may need to manually remove the 127.0.1.1 entries from zookeeper
> after you fix the IP address problem.
How to do that?
With Regards
Aman Tandon
On Wed, Mar 11, 2015 at 8:02 PM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 3/11/2015 6:32 AM, Aman Tandon wrote:
> > I restart my complete cluster but the problem still present. Please help.
> >
> > *Here is the screenshot url:*
> >
> > *http://i.imgur.com/QFdg89S.png
> >
> > http://i.imgur.com/tS0yTNh.png
>
> The first screenshot actually shows the problem, but it may not be
> immediately obvious. In both cases where the shard leader is on
> 127.0.1.1, recovery is not able to complete.
>
> The reason for this is extremely simple -- 127.0.1.1 is a loopback
> address, not accessible remotely ... so when the machine at
> 192.168.6.217 tries to contact the other machine, it is only talking to
> itself, and is not able to find the leader to initiate recovery.
>
> I don't know why that machine chose to register itself in zookeeper
> using a loopback address, unless perhaps the hosts file on that machine
> is set up incorrectly so that the local hostname is associated with that
> IP address. If you can't fix the IP lookup problem, you can override
> the value that Solr uses with the "host" property, which you can set on
> the java commandline (-Dhost=) or in the solr.xml file.
>
> http://wiki.apache.org/solr/SolrCloud#SolrCloud_Instance_Params
>
> You may need to manually remove the 127.0.1.1 entries from zookeeper
> after you fix the IP address problem.
>
> Thanks,
> Shawn
>
>
Re: how to change configurations in solrcloud setup
Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/11/2015 6:32 AM, Aman Tandon wrote:
> I restart my complete cluster but the problem still present. Please help.
>
> *Here is the screenshot url:*
>
> *http://i.imgur.com/QFdg89S.png
>
> http://i.imgur.com/tS0yTNh.png
The first screenshot actually shows the problem, but it may not be
immediately obvious. In both cases where the shard leader is on
127.0.1.1, recovery is not able to complete.
The reason for this is extremely simple -- 127.0.1.1 is a loopback
address, not accessible remotely ... so when the machine at
192.168.6.217 tries to contact the other machine, it is only talking to
itself, and is not able to find the leader to initiate recovery.
I don't know why that machine chose to register itself in zookeeper
using a loopback address, unless perhaps the hosts file on that machine
is set up incorrectly so that the local hostname is associated with that
IP address. If you can't fix the IP lookup problem, you can override
the value that Solr uses with the "host" property, which you can set on
the java commandline (-Dhost=) or in the solr.xml file.
http://wiki.apache.org/solr/SolrCloud#SolrCloud_Instance_Params
You may need to manually remove the 127.0.1.1 entries from zookeeper
after you fix the IP address problem.
Thanks,
Shawn
Re: how to change configurations in solrcloud setup
Posted by Aman Tandon <am...@gmail.com>.
Hi Shawn,
I make the changes in my schema.xml & uploaded the configuration from one
of my server, which is now visible in all other servers (I confirmed it by
checking from admin interface).
*My Solr Cloud arch is :*
I have two collections, mcat & intent in my external zookeeper ensemble of
3.
*On Machine1*
I created the mcat_shard1 of mcat in core 1 & intent_shard1 of intent in
core 2.
*On Machine2*
I created the mcat_shard2 of mcat in core 1 & intent_shard2 of intent in
core 2.
*On Machine 3*
I created the replica of mcat_shard1 in core 1
I created the replica of mcat_shard2 in core 2
I created the replica of intent_shard1 in core 3
I created the replica of intent_shard2 in core 4
So to apply the changes i reloaded my collection using the url:
http://localhost:4567/solr/admin/collections?action=RELOAD&name=intent
Due to which the two shard present on one of my machine (Machine 2) goes in
recovery mode and i am not able to understand what's wrong here.
*Please see the solr logs of that machine:*
INFO - 2015-03-11 12:12:41.630;
> org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null
> path=/admin/cores
> params={coreNodeName=core_node2&onlyIfLeaderActive=true&state=recovering&nodeName=192.168.5.81:4567_solr&action=PREPRECOVERY&checkLive=true&core=mcat_shard2_replica_core&wt=javabin&onlyIfLeader=true&version=2}
> status=400 QTime=1
> ERROR - 2015-03-11 12:12:41.643; org.apache.solr.common.SolrException;
> Error while trying to recover.
> core=mcat_core:java.util.concurrent.ExecutionException:
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
> from server at http://127.0.1.1:4567/solr: I was asked to wait on state
> recovering for null in null on 192.168.5.81:4567_solr but I still do not
> see the requested state. I see state: null live:false leader from ZK: Not
> available due to: java.lang.NullPointerException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:597)
> at
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:369)
> at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:235)
> Caused by:
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
> from server at http://127.0.1.1:4567/solr: I was asked to wait on state
> recovering for null in null on 192.168.5.81:4567_solr but I still do not
> see the requested state. I see state: null live:false leader from ZK: Not
> available due to: java.lang.NullPointerException
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:558)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:249)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient$1.call(HttpSolrClient.java:245)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> ERROR - 2015-03-11 12:12:41.644; org.apache.solr.cloud.RecoveryStrategy;
> Recovery failed - trying again... (6) core=mcat_core
> INFO - 2015-03-11 12:12:41.644; org.apache.solr.cloud.RecoveryStrategy;
> Wait 60.0 seconds before trying to recover again (7)
> INFO - 2015-03-11 12:12:42.665;
> org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change:
> WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> INFO - 2015-03-11 12:12:43.355;
> org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change:
> WatchedEvent state:SyncConnected type:NodeDataChanged
> path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
I restart my complete cluster but the problem still present. Please help.
*Here is the screenshot url:*
*http://i.imgur.com/QFdg89S.png <http://i.imgur.com/QFdg89S.png>*
http://i.imgur.com/tS0yTNh.png
With Regards
Aman Tandon
On Wed, Mar 11, 2015 at 2:02 PM, Aman Tandon <am...@gmail.com>
wrote:
> Thanks Shawn.
>>
>>
>> except that you should reload the collection, which
>> will reload all cores for that collection
>
>
> So i could reload a collection via Collection API's
>
>
> http://localhost:8983/solr/admin/collections?action=RELOAD&name=newCollection
>
> right?
>
> With Regards
> Aman Tandon
>
> On Wed, Mar 11, 2015 at 1:48 PM, Shawn Heisey <ap...@elyograg.org> wrote:
>
>> On 3/11/2015 12:43 AM, Aman Tandon wrote:
>> > Thanks Nitin for replying, isn't it will be costly operation to restart
>> all
>> > nodes.
>> >
>> > What i am doing in this is uploading the configurations again to
>> zookeeper
>> > and then reloading my core. And it is working well. So am i missing
>> > something?
>>
>> Yes, that is enough, except that you should reload the collection, which
>> will reload all cores for that collection. Fully restarting all Solr
>> instances is not required for most changes to a collection config.
>>
>> There is an exception - when you are adding config that uses new jars
>> added to ${solr.solr.home}/lib after Solr startup. In that situation, a
>> restart would be required so that the new jars get loaded. It is
>> strongly recommended that you use an external zookeeper and that you do
>> a rolling restart, where you restart one node, wait for the cloud graph
>> in the admin UI to show 100% green, then restart the next node.
>>
>> Nitin's suggestion shows the zkcli.sh script starting with "sudo" which
>> runs the command as root. This is not necessary. As long as the
>> contents of the example directory (or server directory in 5.0) is
>> accessible to a normal user and the script is marked executable for that
>> user, no special permissions are required.
>>
>> Thanks,
>> Shawn
>>
>>
>
Re: how to change configurations in solrcloud setup
Posted by Aman Tandon <am...@gmail.com>.
Thanks Shawn.
>
>
> except that you should reload the collection, which
> will reload all cores for that collection
So i could reload a collection via Collection API's
http://localhost:8983/solr/admin/collections?action=RELOAD&name=newCollection
right?
With Regards
Aman Tandon
On Wed, Mar 11, 2015 at 1:48 PM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 3/11/2015 12:43 AM, Aman Tandon wrote:
> > Thanks Nitin for replying, isn't it will be costly operation to restart
> all
> > nodes.
> >
> > What i am doing in this is uploading the configurations again to
> zookeeper
> > and then reloading my core. And it is working well. So am i missing
> > something?
>
> Yes, that is enough, except that you should reload the collection, which
> will reload all cores for that collection. Fully restarting all Solr
> instances is not required for most changes to a collection config.
>
> There is an exception - when you are adding config that uses new jars
> added to ${solr.solr.home}/lib after Solr startup. In that situation, a
> restart would be required so that the new jars get loaded. It is
> strongly recommended that you use an external zookeeper and that you do
> a rolling restart, where you restart one node, wait for the cloud graph
> in the admin UI to show 100% green, then restart the next node.
>
> Nitin's suggestion shows the zkcli.sh script starting with "sudo" which
> runs the command as root. This is not necessary. As long as the
> contents of the example directory (or server directory in 5.0) is
> accessible to a normal user and the script is marked executable for that
> user, no special permissions are required.
>
> Thanks,
> Shawn
>
>
Re: how to change configurations in solrcloud setup
Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/11/2015 12:43 AM, Aman Tandon wrote:
> Thanks Nitin for replying, isn't it will be costly operation to restart all
> nodes.
>
> What i am doing in this is uploading the configurations again to zookeeper
> and then reloading my core. And it is working well. So am i missing
> something?
Yes, that is enough, except that you should reload the collection, which
will reload all cores for that collection. Fully restarting all Solr
instances is not required for most changes to a collection config.
There is an exception - when you are adding config that uses new jars
added to ${solr.solr.home}/lib after Solr startup. In that situation, a
restart would be required so that the new jars get loaded. It is
strongly recommended that you use an external zookeeper and that you do
a rolling restart, where you restart one node, wait for the cloud graph
in the admin UI to show 100% green, then restart the next node.
Nitin's suggestion shows the zkcli.sh script starting with "sudo" which
runs the command as root. This is not necessary. As long as the
contents of the example directory (or server directory in 5.0) is
accessible to a normal user and the script is marked executable for that
user, no special permissions are required.
Thanks,
Shawn
Re: how to change configurations in solrcloud setup
Posted by Aman Tandon <am...@gmail.com>.
Hi,
Thanks Nitin for replying, isn't it will be costly operation to restart all
nodes.
What i am doing in this is uploading the configurations again to zookeeper
and then reloading my core. And it is working well. So am i missing
something?
With Regards
Aman Tandon
On Wed, Mar 11, 2015 at 11:21 AM, Nitin Solanki <ni...@gmail.com>
wrote:
> Hi Aman,
> You can apply configuration on solr cloud by using this
> command -
>
> sudo
> <path_of_solr>/<solr_folder_name>/example/scripts/cloud-scripts/zkcli.sh
> -zkhost localhost:9983 -cmd upconfig -confdir
> <path_of_solr>/<solr_folder_name>/example/solr/collection1/conf -confname
> default
>
> and then restart all nodes of solrcloud.
>
> On Mon, Mar 9, 2015 at 11:43 AM, Aman Tandon <am...@gmail.com>
> wrote:
>
> > Please help.
> >
> > With Regards
> > Aman Tandon
> >
> > On Sat, Mar 7, 2015 at 9:58 PM, Aman Tandon <am...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Please tell me what is best way to apply configuration changes in solr
> > > cloud and how to do that.
> > >
> > > Thanks in advance.
> > >
> > > With Regards
> > > Aman Tandon
> > >
> >
>
Re: how to change configurations in solrcloud setup
Posted by Nitin Solanki <ni...@gmail.com>.
Hi Aman,
You can apply configuration on solr cloud by using this
command -
sudo
<path_of_solr>/<solr_folder_name>/example/scripts/cloud-scripts/zkcli.sh
-zkhost localhost:9983 -cmd upconfig -confdir
<path_of_solr>/<solr_folder_name>/example/solr/collection1/conf -confname
default
and then restart all nodes of solrcloud.
On Mon, Mar 9, 2015 at 11:43 AM, Aman Tandon <am...@gmail.com>
wrote:
> Please help.
>
> With Regards
> Aman Tandon
>
> On Sat, Mar 7, 2015 at 9:58 PM, Aman Tandon <am...@gmail.com>
> wrote:
>
> > Hi,
> >
> > Please tell me what is best way to apply configuration changes in solr
> > cloud and how to do that.
> >
> > Thanks in advance.
> >
> > With Regards
> > Aman Tandon
> >
>
Re: how to change configurations in solrcloud setup
Posted by Aman Tandon <am...@gmail.com>.
Please help.
With Regards
Aman Tandon
On Sat, Mar 7, 2015 at 9:58 PM, Aman Tandon <am...@gmail.com> wrote:
> Hi,
>
> Please tell me what is best way to apply configuration changes in solr
> cloud and how to do that.
>
> Thanks in advance.
>
> With Regards
> Aman Tandon
>