You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Salman Ansari <sa...@gmail.com> on 2016/04/07 13:40:35 UTC
Re: Problem in Issuing a Command to Upload Configuration
Any comments regarding the issue I mentioned above "the proper procedure of
bringing old collections up after a restart of zookeeper ensemble and Solr
instances"?
Your feedback is appreciated.
Regards,
Salman
On Tue, Mar 29, 2016 at 12:04 PM, Salman Ansari <sa...@gmail.com>
wrote:
> Moreover, I have created those new collections as a work around as my past
> collections were not coming up after a complete restart for machines
> hosting zookeepers and Solr. I would be interested to know what is the
> proper procedure of bringing old collections up after a restart of
> zookeeper ensemble and Solr instances.
>
> Appreciate any feedback and comments.
>
> Regards,
> Salman
>
>
> On Tue, Mar 29, 2016 at 11:53 AM, Salman Ansari <sa...@gmail.com>
> wrote:
>
>> Thanks Reth for your response. It did work.
>>
>> Regards,
>> Salman
>>
>> On Mon, Mar 28, 2016 at 8:01 PM, Reth RM <re...@gmail.com> wrote:
>>
>>> I think it should be "zkcli.bat" (all in lower case) that is shipped with
>>> solr not zkCli.cmd(that is shipped with zookeeper)
>>>
>>> solr_home/server/scripts/cloud-scripts/zkcli.bat -zkhost 127.0.0.1:9983
>>> \
>>> -cmd upconfig -confname my_new_config -confdir
>>> server/solr/configsets/basic_configs/conf
>>>
>>> On Mon, Mar 28, 2016 at 8:18 PM, Salman Ansari <sa...@gmail.com>
>>> wrote:
>>>
>>> > Hi,
>>> >
>>> > I am facing issue uploading configuration to Zookeeper ensemble. I am
>>> > running this on Windows as
>>> >
>>> > *Command*
>>> > *========*
>>> > zkCli.cmd -cmd upconfig -zkhost
>>> > "[localserver]:2181,[second_server]:2181,[third_server]:2181" -confname
>>> > [config_name] -confdir "[config_dir]"
>>> >
>>> > and I got the following result
>>> >
>>> > *Result*
>>> > =====
>>> > Connecting to localhost:2181
>>> > 2016-03-28 14:40:12,849 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> > ent:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
>>> > 2016-03-28 14:40:12,849 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> > ent:host.name=SabrSolrServer1.SabrSolrServer1.a2.internal.cloudapp.net
>>> > 2016-03-28 14:40:12,849 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> > ent:java.version=1.8.0_77
>>> > 2016-03-28 14:40:12,849 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> > ent:java.vendor=Oracle Corporation
>>> > 2016-03-28 14:40:12,849 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> > ent:java.home=C:\Program Files\Java\jre1.8.0_77
>>> > 2016-03-28 14:40:12,849 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> >
>>> >
>>> ent:java.class.path=C:\Solr\Zookeeper\zookeeper-3.4.6\bin\..\build\classes;C:\So
>>> >
>>> >
>>> lr\Zookeeper\zookeeper-3.4.6\bin\..\build\lib\*;C:\Solr\Zookeeper\zookeeper-3.4.
>>> >
>>> >
>>> 6\bin\..\zookeeper-3.4.6.jar;C:\Solr\Zookeeper\zookeeper-3.4.6\bin\..\lib\jline-
>>> >
>>> >
>>> 0.9.94.jar;C:\Solr\Zookeeper\zookeeper-3.4.6\bin\..\lib\log4j-1.2.16.jar;C:\Solr
>>> >
>>> >
>>> \Zookeeper\zookeeper-3.4.6\bin\..\lib\netty-3.7.0.Final.jar;C:\Solr\Zookeeper\zo
>>> >
>>> >
>>> okeeper-3.4.6\bin\..\lib\slf4j-api-1.6.1.jar;C:\Solr\Zookeeper\zookeeper-3.4.6\b
>>> >
>>> >
>>> in\..\lib\slf4j-log4j12-1.6.1.jar;C:\Solr\Zookeeper\zookeeper-3.4.6\bin\..\conf
>>> > 2016-03-28 14:40:12,865 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> >
>>> >
>>> ent:java.library.path=C:\ProgramData\Oracle\Java\javapath;C:\Windows\Sun\Java\bi
>>> >
>>> >
>>> n;C:\Windows\system32;C:\Windows;C:\ProgramData\Oracle\Java\javapath;C:\Windows\
>>> >
>>> >
>>> system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShe
>>> > ll\v1.0\;C:\Program Files\Java\JDK\bin;.
>>> > 2016-03-28 14:40:12,865 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> > ent:java.io.tmpdir=C:\Users\ADMIN_~1\AppData\Local\Temp\2\
>>> > 2016-03-28 14:40:12,865 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> > ent:java.compiler=<NA>
>>> > 2016-03-28 14:40:12,865 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> > ent:os.name=Windows Server 2012 R2
>>> > 2016-03-28 14:40:12,865 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> > ent:os.arch=amd64
>>> > 2016-03-28 14:40:12,865 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> > ent:os.version=6.3
>>> > 2016-03-28 14:40:12,865 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> > ent:user.name=admin_user
>>> > 2016-03-28 14:40:12,865 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> > ent:user.home=C:\Users\admin_user
>>> > 2016-03-28 14:40:12,865 [myid:] - INFO [main:Environment@100] -
>>> Client
>>> > environm
>>> > ent:user.dir=C:\Solr\Zookeeper\zookeeper-3.4.6\bin
>>> > 2016-03-28 14:40:12,865 [myid:] - INFO [main:ZooKeeper@438] -
>>> Initiating
>>> > client
>>> > connection, connectString=localhost:2181 sessionTimeout=30000
>>> > watcher=org.apach
>>> > e.zookeeper.ZooKeeperMain$MyWatcher@506c589e
>>> >
>>> > It looks like that it is not even calling the command. Any idea why is
>>> that
>>> > happening?
>>> >
>>> > Regards,
>>> > Salman
>>> >
>>>
>>
>>
>
Re: Problem in Issuing a Command to Upload Configuration
Posted by Erick Erickson <er...@gmail.com>.
If Solr is killed un-gracefully, " it can leave the lock files
in the index directory and when Solr comes back up
it thinks some other Solr process is writing the files
and refuses to allow _this_ Solr process to write to
those files
Probably this:
bq: machines faced a forced
restart to install Windows Updates
shut the Solr instances down un-gracefully. So next time
Windows "helpfully" wants to kill all your processes,
don't let it until you've stopped Solr gracefully via the
bin\solr script.
BTW, I've seen some situations where the bin/solr script
will forcefully kill the Solr instance.
And to cure the situation, remove the write.lock file from the
index directory (when Solr is stopped) and then start
Solr.
Best,
Erick
On Mon, May 2, 2016 at 2:36 AM, Salman Ansari <sa...@gmail.com> wrote:
> Well, that just happened! Solr and Zookeeper machines faced a forced
> restart to install Windows Updates. This caused Zookeeper ensemble and Solr
> instances to go down. When the machines came back up again. I tried the
> following
>
> 1) Started Zookeeper on all machines using the following command
> zkServer.cmd (on all three machines)
>
> 2) Started Solr on two of those machines using
>
> solr.cmd start -c -p 8983 -h [server1_name] -z
> "[server1_ip]:2181,[server2_name]:2181,[server3_name]:2181"
> solr.cmd start -c -p 8983 -h [server2_name] -z
> "[server2_ip]:2181,[server1_name]:2181,[server3_name]:2181"
> solr.cmd start -c -p 7574 -h [server1_name] -z
> "[server1_ip]:2181,[server2_name]:2181,[server3_name]:2181"
> solr.cmd start -c -p 7574 -h [server2_name] -z
> "[server2_ip]:2181,[server1_name]:2181,[server3_name]:2181"
>
> After several trials, it did start the solr on both machines but *non of
> the previous collections came back normally.* When I look at the admin
> page, it shows errors as follows
>
> *[Collection_name]_shard2_replica2:*
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Index locked for write for core '[Collection_name]_shard2_replica2'. Solr
> now longer supports forceful unlocking via 'unlockOnStartup'. Please verify
> locks manually!
>
> So probably I am doing something wrong or the simple scenario is not
> straight forward to recover from.
>
> Your comment/feedback is appreciated.
>
> Regards,
> Salman
>
>
>
> On Thu, Apr 7, 2016 at 3:56 PM, Shawn Heisey <ap...@elyograg.org> wrote:
>
>> On 4/7/2016 5:40 AM, Salman Ansari wrote:
>> > Any comments regarding the issue I mentioned above "the proper procedure
>> of
>> > bringing old collections up after a restart of zookeeper ensemble and
>> Solr
>> > instances"?
>>
>> What precisely do you mean by "old collections"? The simplest
>> interpretation of that is that you are trying to restart your servers
>> and have everything you already had in the cloud work properly. An
>> alternate interpretation, which might be just as valid, is that you have
>> some collections on some old servers that you want to incorporate into a
>> new cloud.
>>
>> If it's the simple scenario: shut down solr, shut down zookeeper, start
>> zookeeper, start solr. If it's the other scenario, that is not quite so
>> simple.
>>
>> Thanks,
>> Shawn
>>
>>
Re: Problem in Issuing a Command to Upload Configuration
Posted by Salman Ansari <sa...@gmail.com>.
Well, that just happened! Solr and Zookeeper machines faced a forced
restart to install Windows Updates. This caused Zookeeper ensemble and Solr
instances to go down. When the machines came back up again. I tried the
following
1) Started Zookeeper on all machines using the following command
zkServer.cmd (on all three machines)
2) Started Solr on two of those machines using
solr.cmd start -c -p 8983 -h [server1_name] -z
"[server1_ip]:2181,[server2_name]:2181,[server3_name]:2181"
solr.cmd start -c -p 8983 -h [server2_name] -z
"[server2_ip]:2181,[server1_name]:2181,[server3_name]:2181"
solr.cmd start -c -p 7574 -h [server1_name] -z
"[server1_ip]:2181,[server2_name]:2181,[server3_name]:2181"
solr.cmd start -c -p 7574 -h [server2_name] -z
"[server2_ip]:2181,[server1_name]:2181,[server3_name]:2181"
After several trials, it did start the solr on both machines but *non of
the previous collections came back normally.* When I look at the admin
page, it shows errors as follows
*[Collection_name]_shard2_replica2:*
org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
Index locked for write for core '[Collection_name]_shard2_replica2'. Solr
now longer supports forceful unlocking via 'unlockOnStartup'. Please verify
locks manually!
So probably I am doing something wrong or the simple scenario is not
straight forward to recover from.
Your comment/feedback is appreciated.
Regards,
Salman
On Thu, Apr 7, 2016 at 3:56 PM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 4/7/2016 5:40 AM, Salman Ansari wrote:
> > Any comments regarding the issue I mentioned above "the proper procedure
> of
> > bringing old collections up after a restart of zookeeper ensemble and
> Solr
> > instances"?
>
> What precisely do you mean by "old collections"? The simplest
> interpretation of that is that you are trying to restart your servers
> and have everything you already had in the cloud work properly. An
> alternate interpretation, which might be just as valid, is that you have
> some collections on some old servers that you want to incorporate into a
> new cloud.
>
> If it's the simple scenario: shut down solr, shut down zookeeper, start
> zookeeper, start solr. If it's the other scenario, that is not quite so
> simple.
>
> Thanks,
> Shawn
>
>
Re: Problem in Issuing a Command to Upload Configuration
Posted by Shawn Heisey <ap...@elyograg.org>.
On 4/7/2016 5:40 AM, Salman Ansari wrote:
> Any comments regarding the issue I mentioned above "the proper procedure of
> bringing old collections up after a restart of zookeeper ensemble and Solr
> instances"?
What precisely do you mean by "old collections"? The simplest
interpretation of that is that you are trying to restart your servers
and have everything you already had in the cloud work properly. An
alternate interpretation, which might be just as valid, is that you have
some collections on some old servers that you want to incorporate into a
new cloud.
If it's the simple scenario: shut down solr, shut down zookeeper, start
zookeeper, start solr. If it's the other scenario, that is not quite so
simple.
Thanks,
Shawn