You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Utkarsh Sengar <ut...@gmail.com> on 2013/06/25 10:29:06 UTC
Updating solrconfig and schema.xml for solrcloud in multicore setup
Hello,
I am trying to update schema.xml for a core in a multicore setup and this
is what I do to update it:
I have 3 nodes in my solr cluster.
1. Pick node1 and manually update schema.xml
2. Restart node1 with -Dbootstrap_conf=true
java -Dsolr.solr.home=multicore -DnumShards=3 -Dbootstrap_conf=true
-DzkHost=localhost:2181 -DSTOP.PORT=8079 -DSTOP.KEY=mysecret -jar start.jar
3. Restart the other 2 nodes using this command (without
-Dbootstrap_conf=true since these should pull from zk).:
java -Dsolr.solr.home=multicore -DnumShards=3 -DzkHost=localhost:2181
-DSTOP.PORT=8079 -DSTOP.KEY=mysecret -jar start.jar
But, when I do that. node1 displays all of my cores and the other 2 nodes
displays just one core.
Then, I found this:
http://mail-archives.apache.org/mod_mbox/lucene-dev/201205.mbox/%3CBB7AD9BF-389B-4B94-8C1B-BBFC4028A115@gmail.com%3E
Which says bootstrap_conf is used for multicore setup.
But if I use bootstrap_conf for every node, then I will have to manually
update schema.xml (for any config file) everywhere? That does not sound
like an efficient way of managing configuration right?
--
Thanks,
-Utkarsh
Re: Updating solrconfig and schema.xml for solrcloud in multicore setup
Posted by Utkarsh Sengar <ut...@gmail.com>.
I believe I am hitting this bug:
https://issues.apache.org/jira/browse/SOLR-4805
I am using solr 4.3.1
-Utkarsh
On Tue, Jun 25, 2013 at 2:56 AM, Utkarsh Sengar <ut...@gmail.com>wrote:
> Yes, I have tried zkCli and it works.
> But I also need to restart solr after the schema change right?
>
> I tried to reload the core, but I think there is an open bug where a core
> reload is successful but a shard goes down for that core. I just tried it
> out, i.e tried to reload a core after config change via zkCli and a shard
> went down.
>
> Since I am not able to reload a core, I am restarting the whole solr
> process for make the change.
>
> Thanks,
> -Utkarsh
>
>
> On Tue, Jun 25, 2013 at 2:46 AM, Jan Høydahl <ja...@cominvent.com>wrote:
>
>> Hi,
>>
>> As I understand, your initial bootstrap works ok (boostrap_conf). What
>> you want help with is *changing* the config on a live system.
>> That's when you are encouraged to use zkCli and don't mess with trying to
>> let Solr bootstrap things - after all it's not a bootstrap anymore, it's a
>> change.
>>
>> Did you try updating schema.xml for a specific collection using zkCli?
>> Any issues?
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>>
>> 25. juni 2013 kl. 11:24 skrev Utkarsh Sengar <ut...@gmail.com>:
>>
>> > But as when I launch a solr instance without "-Dbootstrap_conf=true",
>> just
>> > once core is launched and I cannot see the other core.
>> >
>> > This behavior is the same as Mark's reply here:
>> >
>> http://mail-archives.apache.org/mod_mbox/lucene-dev/201205.mbox/%3CBB7AD9BF-389B-4B94-8C1B-BBFC4028A115@gmail.com%3E
>> >
>> > - bootstrap_conf: you pass it true and it reads solr.xml and uploads
>> > the conf set for each
>> > SolrCore it finds, gives the conf set the name of the collection and
>> > associates each collection
>> > with the same named config set.
>> >
>> > So the first just lets you boot strap one collection easily...but what
>> > if you start with a
>> > multi-core, multi-collection setup that you want to bootstrap into
>> > SolrCloud? And they don't
>> > share a common config set? That's what the second command is for. You
>> > can setup 30 local SolrCores
>> > in solr.xml and then just bootstrap all 30 different config sets up
>> > and have them fully linked
>> > with each collection just by passing bootstrap_conf=true.
>> >
>> >
>> >
>> > Note: I am using -Dbootstrap_conf=true and not -Dbootstrap_confdir
>> >
>> >
>> > Thanks,
>> > -Utkarsh
>> >
>> >
>> > On Tue, Jun 25, 2013 at 2:14 AM, Jan Høydahl <ja...@cominvent.com>
>> wrote:
>> >
>> >> Hi,
>> >>
>> >> The -Dbootstrap_confdir option is really only meant for a first-time
>> >> bootstrap for your development environment, not for serious use.
>> >>
>> >> Once you got your config into ZK you should modify the config directly
>> in
>> >> ZK.
>> >> There are many tools (also 3rd party) for this. But your best choice is
>> >> probably zkCli shipping with Solr.
>> >> See http://wiki.apache.org/solr/SolrCloud#Command_Line_Util
>> >> This means you will NOT need to start Solr with -Dboostrap_confdir at
>> all.
>> >>
>> >> --
>> >> Jan Høydahl, search solution architect
>> >> Cominvent AS - www.cominvent.com
>> >>
>> >> 25. juni 2013 kl. 10:29 skrev Utkarsh Sengar <ut...@gmail.com>:
>> >>
>> >>> Hello,
>> >>>
>> >>> I am trying to update schema.xml for a core in a multicore setup and
>> this
>> >>> is what I do to update it:
>> >>>
>> >>> I have 3 nodes in my solr cluster.
>> >>>
>> >>> 1. Pick node1 and manually update schema.xml
>> >>>
>> >>> 2. Restart node1 with -Dbootstrap_conf=true
>> >>> java -Dsolr.solr.home=multicore -DnumShards=3 -Dbootstrap_conf=true
>> >>> -DzkHost=localhost:2181 -DSTOP.PORT=8079 -DSTOP.KEY=mysecret -jar
>> >> start.jar
>> >>>
>> >>> 3. Restart the other 2 nodes using this command (without
>> >>> -Dbootstrap_conf=true since these should pull from zk).:
>> >>> java -Dsolr.solr.home=multicore -DnumShards=3 -DzkHost=localhost:2181
>> >>> -DSTOP.PORT=8079 -DSTOP.KEY=mysecret -jar start.jar
>> >>>
>> >>> But, when I do that. node1 displays all of my cores and the other 2
>> nodes
>> >>> displays just one core.
>> >>>
>> >>> Then, I found this:
>> >>>
>> >>
>> http://mail-archives.apache.org/mod_mbox/lucene-dev/201205.mbox/%3CBB7AD9BF-389B-4B94-8C1B-BBFC4028A115@gmail.com%3E
>> >>> Which says bootstrap_conf is used for multicore setup.
>> >>>
>> >>>
>> >>> But if I use bootstrap_conf for every node, then I will have to
>> manually
>> >>> update schema.xml (for any config file) everywhere? That does not
>> sound
>> >>> like an efficient way of managing configuration right?
>> >>>
>> >>>
>> >>> --
>> >>> Thanks,
>> >>> -Utkarsh
>> >>
>> >>
>> >
>> >
>> > --
>> > Thanks,
>> > -Utkarsh
>>
>>
>
>
> --
> Thanks,
> -Utkarsh
>
--
Thanks,
-Utkarsh
Re: Updating solrconfig and schema.xml for solrcloud in multicore setup
Posted by Utkarsh Sengar <ut...@gmail.com>.
Yes, I have tried zkCli and it works.
But I also need to restart solr after the schema change right?
I tried to reload the core, but I think there is an open bug where a core
reload is successful but a shard goes down for that core. I just tried it
out, i.e tried to reload a core after config change via zkCli and a shard
went down.
Since I am not able to reload a core, I am restarting the whole solr
process for make the change.
Thanks,
-Utkarsh
On Tue, Jun 25, 2013 at 2:46 AM, Jan Høydahl <ja...@cominvent.com> wrote:
> Hi,
>
> As I understand, your initial bootstrap works ok (boostrap_conf). What you
> want help with is *changing* the config on a live system.
> That's when you are encouraged to use zkCli and don't mess with trying to
> let Solr bootstrap things - after all it's not a bootstrap anymore, it's a
> change.
>
> Did you try updating schema.xml for a specific collection using zkCli? Any
> issues?
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> 25. juni 2013 kl. 11:24 skrev Utkarsh Sengar <ut...@gmail.com>:
>
> > But as when I launch a solr instance without "-Dbootstrap_conf=true",
> just
> > once core is launched and I cannot see the other core.
> >
> > This behavior is the same as Mark's reply here:
> >
> http://mail-archives.apache.org/mod_mbox/lucene-dev/201205.mbox/%3CBB7AD9BF-389B-4B94-8C1B-BBFC4028A115@gmail.com%3E
> >
> > - bootstrap_conf: you pass it true and it reads solr.xml and uploads
> > the conf set for each
> > SolrCore it finds, gives the conf set the name of the collection and
> > associates each collection
> > with the same named config set.
> >
> > So the first just lets you boot strap one collection easily...but what
> > if you start with a
> > multi-core, multi-collection setup that you want to bootstrap into
> > SolrCloud? And they don't
> > share a common config set? That's what the second command is for. You
> > can setup 30 local SolrCores
> > in solr.xml and then just bootstrap all 30 different config sets up
> > and have them fully linked
> > with each collection just by passing bootstrap_conf=true.
> >
> >
> >
> > Note: I am using -Dbootstrap_conf=true and not -Dbootstrap_confdir
> >
> >
> > Thanks,
> > -Utkarsh
> >
> >
> > On Tue, Jun 25, 2013 at 2:14 AM, Jan Høydahl <ja...@cominvent.com>
> wrote:
> >
> >> Hi,
> >>
> >> The -Dbootstrap_confdir option is really only meant for a first-time
> >> bootstrap for your development environment, not for serious use.
> >>
> >> Once you got your config into ZK you should modify the config directly
> in
> >> ZK.
> >> There are many tools (also 3rd party) for this. But your best choice is
> >> probably zkCli shipping with Solr.
> >> See http://wiki.apache.org/solr/SolrCloud#Command_Line_Util
> >> This means you will NOT need to start Solr with -Dboostrap_confdir at
> all.
> >>
> >> --
> >> Jan Høydahl, search solution architect
> >> Cominvent AS - www.cominvent.com
> >>
> >> 25. juni 2013 kl. 10:29 skrev Utkarsh Sengar <ut...@gmail.com>:
> >>
> >>> Hello,
> >>>
> >>> I am trying to update schema.xml for a core in a multicore setup and
> this
> >>> is what I do to update it:
> >>>
> >>> I have 3 nodes in my solr cluster.
> >>>
> >>> 1. Pick node1 and manually update schema.xml
> >>>
> >>> 2. Restart node1 with -Dbootstrap_conf=true
> >>> java -Dsolr.solr.home=multicore -DnumShards=3 -Dbootstrap_conf=true
> >>> -DzkHost=localhost:2181 -DSTOP.PORT=8079 -DSTOP.KEY=mysecret -jar
> >> start.jar
> >>>
> >>> 3. Restart the other 2 nodes using this command (without
> >>> -Dbootstrap_conf=true since these should pull from zk).:
> >>> java -Dsolr.solr.home=multicore -DnumShards=3 -DzkHost=localhost:2181
> >>> -DSTOP.PORT=8079 -DSTOP.KEY=mysecret -jar start.jar
> >>>
> >>> But, when I do that. node1 displays all of my cores and the other 2
> nodes
> >>> displays just one core.
> >>>
> >>> Then, I found this:
> >>>
> >>
> http://mail-archives.apache.org/mod_mbox/lucene-dev/201205.mbox/%3CBB7AD9BF-389B-4B94-8C1B-BBFC4028A115@gmail.com%3E
> >>> Which says bootstrap_conf is used for multicore setup.
> >>>
> >>>
> >>> But if I use bootstrap_conf for every node, then I will have to
> manually
> >>> update schema.xml (for any config file) everywhere? That does not sound
> >>> like an efficient way of managing configuration right?
> >>>
> >>>
> >>> --
> >>> Thanks,
> >>> -Utkarsh
> >>
> >>
> >
> >
> > --
> > Thanks,
> > -Utkarsh
>
>
--
Thanks,
-Utkarsh
Re: Updating solrconfig and schema.xml for solrcloud in multicore setup
Posted by Jan Høydahl <ja...@cominvent.com>.
Hi,
As I understand, your initial bootstrap works ok (boostrap_conf). What you want help with is *changing* the config on a live system.
That's when you are encouraged to use zkCli and don't mess with trying to let Solr bootstrap things - after all it's not a bootstrap anymore, it's a change.
Did you try updating schema.xml for a specific collection using zkCli? Any issues?
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
25. juni 2013 kl. 11:24 skrev Utkarsh Sengar <ut...@gmail.com>:
> But as when I launch a solr instance without "-Dbootstrap_conf=true", just
> once core is launched and I cannot see the other core.
>
> This behavior is the same as Mark's reply here:
> http://mail-archives.apache.org/mod_mbox/lucene-dev/201205.mbox/%3CBB7AD9BF-389B-4B94-8C1B-BBFC4028A115@gmail.com%3E
>
> - bootstrap_conf: you pass it true and it reads solr.xml and uploads
> the conf set for each
> SolrCore it finds, gives the conf set the name of the collection and
> associates each collection
> with the same named config set.
>
> So the first just lets you boot strap one collection easily...but what
> if you start with a
> multi-core, multi-collection setup that you want to bootstrap into
> SolrCloud? And they don't
> share a common config set? That's what the second command is for. You
> can setup 30 local SolrCores
> in solr.xml and then just bootstrap all 30 different config sets up
> and have them fully linked
> with each collection just by passing bootstrap_conf=true.
>
>
>
> Note: I am using -Dbootstrap_conf=true and not -Dbootstrap_confdir
>
>
> Thanks,
> -Utkarsh
>
>
> On Tue, Jun 25, 2013 at 2:14 AM, Jan Høydahl <ja...@cominvent.com> wrote:
>
>> Hi,
>>
>> The -Dbootstrap_confdir option is really only meant for a first-time
>> bootstrap for your development environment, not for serious use.
>>
>> Once you got your config into ZK you should modify the config directly in
>> ZK.
>> There are many tools (also 3rd party) for this. But your best choice is
>> probably zkCli shipping with Solr.
>> See http://wiki.apache.org/solr/SolrCloud#Command_Line_Util
>> This means you will NOT need to start Solr with -Dboostrap_confdir at all.
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>>
>> 25. juni 2013 kl. 10:29 skrev Utkarsh Sengar <ut...@gmail.com>:
>>
>>> Hello,
>>>
>>> I am trying to update schema.xml for a core in a multicore setup and this
>>> is what I do to update it:
>>>
>>> I have 3 nodes in my solr cluster.
>>>
>>> 1. Pick node1 and manually update schema.xml
>>>
>>> 2. Restart node1 with -Dbootstrap_conf=true
>>> java -Dsolr.solr.home=multicore -DnumShards=3 -Dbootstrap_conf=true
>>> -DzkHost=localhost:2181 -DSTOP.PORT=8079 -DSTOP.KEY=mysecret -jar
>> start.jar
>>>
>>> 3. Restart the other 2 nodes using this command (without
>>> -Dbootstrap_conf=true since these should pull from zk).:
>>> java -Dsolr.solr.home=multicore -DnumShards=3 -DzkHost=localhost:2181
>>> -DSTOP.PORT=8079 -DSTOP.KEY=mysecret -jar start.jar
>>>
>>> But, when I do that. node1 displays all of my cores and the other 2 nodes
>>> displays just one core.
>>>
>>> Then, I found this:
>>>
>> http://mail-archives.apache.org/mod_mbox/lucene-dev/201205.mbox/%3CBB7AD9BF-389B-4B94-8C1B-BBFC4028A115@gmail.com%3E
>>> Which says bootstrap_conf is used for multicore setup.
>>>
>>>
>>> But if I use bootstrap_conf for every node, then I will have to manually
>>> update schema.xml (for any config file) everywhere? That does not sound
>>> like an efficient way of managing configuration right?
>>>
>>>
>>> --
>>> Thanks,
>>> -Utkarsh
>>
>>
>
>
> --
> Thanks,
> -Utkarsh
Re: Updating solrconfig and schema.xml for solrcloud in multicore setup
Posted by Utkarsh Sengar <ut...@gmail.com>.
But as when I launch a solr instance without "-Dbootstrap_conf=true", just
once core is launched and I cannot see the other core.
This behavior is the same as Mark's reply here:
http://mail-archives.apache.org/mod_mbox/lucene-dev/201205.mbox/%3CBB7AD9BF-389B-4B94-8C1B-BBFC4028A115@gmail.com%3E
- bootstrap_conf: you pass it true and it reads solr.xml and uploads
the conf set for each
SolrCore it finds, gives the conf set the name of the collection and
associates each collection
with the same named config set.
So the first just lets you boot strap one collection easily...but what
if you start with a
multi-core, multi-collection setup that you want to bootstrap into
SolrCloud? And they don't
share a common config set? That's what the second command is for. You
can setup 30 local SolrCores
in solr.xml and then just bootstrap all 30 different config sets up
and have them fully linked
with each collection just by passing bootstrap_conf=true.
Note: I am using -Dbootstrap_conf=true and not -Dbootstrap_confdir
Thanks,
-Utkarsh
On Tue, Jun 25, 2013 at 2:14 AM, Jan Høydahl <ja...@cominvent.com> wrote:
> Hi,
>
> The -Dbootstrap_confdir option is really only meant for a first-time
> bootstrap for your development environment, not for serious use.
>
> Once you got your config into ZK you should modify the config directly in
> ZK.
> There are many tools (also 3rd party) for this. But your best choice is
> probably zkCli shipping with Solr.
> See http://wiki.apache.org/solr/SolrCloud#Command_Line_Util
> This means you will NOT need to start Solr with -Dboostrap_confdir at all.
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> 25. juni 2013 kl. 10:29 skrev Utkarsh Sengar <ut...@gmail.com>:
>
> > Hello,
> >
> > I am trying to update schema.xml for a core in a multicore setup and this
> > is what I do to update it:
> >
> > I have 3 nodes in my solr cluster.
> >
> > 1. Pick node1 and manually update schema.xml
> >
> > 2. Restart node1 with -Dbootstrap_conf=true
> > java -Dsolr.solr.home=multicore -DnumShards=3 -Dbootstrap_conf=true
> > -DzkHost=localhost:2181 -DSTOP.PORT=8079 -DSTOP.KEY=mysecret -jar
> start.jar
> >
> > 3. Restart the other 2 nodes using this command (without
> > -Dbootstrap_conf=true since these should pull from zk).:
> > java -Dsolr.solr.home=multicore -DnumShards=3 -DzkHost=localhost:2181
> > -DSTOP.PORT=8079 -DSTOP.KEY=mysecret -jar start.jar
> >
> > But, when I do that. node1 displays all of my cores and the other 2 nodes
> > displays just one core.
> >
> > Then, I found this:
> >
> http://mail-archives.apache.org/mod_mbox/lucene-dev/201205.mbox/%3CBB7AD9BF-389B-4B94-8C1B-BBFC4028A115@gmail.com%3E
> > Which says bootstrap_conf is used for multicore setup.
> >
> >
> > But if I use bootstrap_conf for every node, then I will have to manually
> > update schema.xml (for any config file) everywhere? That does not sound
> > like an efficient way of managing configuration right?
> >
> >
> > --
> > Thanks,
> > -Utkarsh
>
>
--
Thanks,
-Utkarsh
Re: Updating solrconfig and schema.xml for solrcloud in multicore setup
Posted by Jan Høydahl <ja...@cominvent.com>.
Hi,
The -Dbootstrap_confdir option is really only meant for a first-time bootstrap for your development environment, not for serious use.
Once you got your config into ZK you should modify the config directly in ZK.
There are many tools (also 3rd party) for this. But your best choice is probably zkCli shipping with Solr.
See http://wiki.apache.org/solr/SolrCloud#Command_Line_Util
This means you will NOT need to start Solr with -Dboostrap_confdir at all.
--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
25. juni 2013 kl. 10:29 skrev Utkarsh Sengar <ut...@gmail.com>:
> Hello,
>
> I am trying to update schema.xml for a core in a multicore setup and this
> is what I do to update it:
>
> I have 3 nodes in my solr cluster.
>
> 1. Pick node1 and manually update schema.xml
>
> 2. Restart node1 with -Dbootstrap_conf=true
> java -Dsolr.solr.home=multicore -DnumShards=3 -Dbootstrap_conf=true
> -DzkHost=localhost:2181 -DSTOP.PORT=8079 -DSTOP.KEY=mysecret -jar start.jar
>
> 3. Restart the other 2 nodes using this command (without
> -Dbootstrap_conf=true since these should pull from zk).:
> java -Dsolr.solr.home=multicore -DnumShards=3 -DzkHost=localhost:2181
> -DSTOP.PORT=8079 -DSTOP.KEY=mysecret -jar start.jar
>
> But, when I do that. node1 displays all of my cores and the other 2 nodes
> displays just one core.
>
> Then, I found this:
> http://mail-archives.apache.org/mod_mbox/lucene-dev/201205.mbox/%3CBB7AD9BF-389B-4B94-8C1B-BBFC4028A115@gmail.com%3E
> Which says bootstrap_conf is used for multicore setup.
>
>
> But if I use bootstrap_conf for every node, then I will have to manually
> update schema.xml (for any config file) everywhere? That does not sound
> like an efficient way of managing configuration right?
>
>
> --
> Thanks,
> -Utkarsh