You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Vincenzo D'Amore <v....@gmail.com> on 2015/12/11 17:22:54 UTC
Solrcloud 4.8.1 - Solr cores reload
Hi all,
in day by day work, often I need to change the solr configurations files.
Often adding new synonyms, changing the schema or the solrconfig.xml.
Everything is stored in zookeeper.
But I have inherited a piece of code that, after every change, reload all
the cores using CoreAdmin API.
Now I have 15 replicas in the collection, and after every core reload the
code waits for 60 seconds (I suppose it's because who wrote the code was
worried about the cache invalidation).
Given that, it takes about 25 minutes to update all the cores. Obviously
during this time we cannot modify the collection.
The question is, to reduce this wait, if I use the collection API RELOAD,
what are the counter indication?
Thanks in advance for your time,
Vincenzo
--
Vincenzo D'Amore
email: v.damore@gmail.com
skype: free.dev
mobile: +39 349 8513251
Re: Solrcloud 4.8.1 - Solr cores reload
Posted by Erick Erickson <er...@gmail.com>.
Right. What's happening is, essentially what used to be
happening in your custom code where individual core
reload commands were being sent. Except it's all happening
in Solr. To whit:
1> the code looks at the collection state
2> for each replica it sends a core admin API reload command
to the appropriate node.
It's really nothing different than what you probably had before
but I'm much more confident in code that's
1> written by the same people who wrote the rest of the Cloud code
2> tested in the Solr test case
3> not something I can forget to maintain ;)
Best,
Erick
On Fri, Dec 11, 2015 at 6:10 PM, Vincenzo D'Amore <v....@gmail.com> wrote:
> Thanks for your suggestion Erick, I'm changing the code and I'll use the
> Collections API RELOAD.
> I have done few test changing synonyms dictionary or solrconfig and
> everything works fine.
>
> Well, I think you already know, but looking at solr.log file after the
> collections api reload call, I have seen a bunch of lines like this one:
>
> - Collection Admin sending CoreAdmin cmd to http://192.168.101.118:8080/solr
> params:action=RELOAD&core=collection1_shard1_replica1&qt=%2Fadmin%2Fcores
> ...
>
> Best regards and thanks again,
> Vincenzo
>
>
> On Fri, Dec 11, 2015 at 7:38 PM, Erick Erickson <er...@gmail.com>
> wrote:
>
>> You should absolutely always use the Collection API rather than
>> any core admin API if at all possible. If for no other reason
>> than your client will be _lots_ simpler (i.e. you don't have
>> to find all the replicas and issue the core admin RELOAD
>> command for each one).
>>
>> I'm not entirely sure whether the RELOAD command is
>> synchronous or not though.
>>
>> Best,
>> erick
>>
>> On Fri, Dec 11, 2015 at 8:22 AM, Vincenzo D'Amore <v....@gmail.com>
>> wrote:
>> > Hi all,
>> >
>> > in day by day work, often I need to change the solr configurations files.
>> > Often adding new synonyms, changing the schema or the solrconfig.xml.
>> >
>> > Everything is stored in zookeeper.
>> >
>> > But I have inherited a piece of code that, after every change, reload all
>> > the cores using CoreAdmin API.
>> >
>> > Now I have 15 replicas in the collection, and after every core reload the
>> > code waits for 60 seconds (I suppose it's because who wrote the code was
>> > worried about the cache invalidation).
>> >
>> > Given that, it takes about 25 minutes to update all the cores. Obviously
>> > during this time we cannot modify the collection.
>> >
>> > The question is, to reduce this wait, if I use the collection API RELOAD,
>> > what are the counter indication?
>> >
>> > Thanks in advance for your time,
>> > Vincenzo
>> >
>> >
>> > --
>> > Vincenzo D'Amore
>> > email: v.damore@gmail.com
>> > skype: free.dev
>> > mobile: +39 349 8513251
>>
>
>
>
> --
> Vincenzo D'Amore
> email: v.damore@gmail.com
> skype: free.dev
> mobile: +39 349 8513251
Re: Solrcloud 4.8.1 - Solr cores reload
Posted by Vincenzo D'Amore <v....@gmail.com>.
Thanks for your suggestion Erick, I'm changing the code and I'll use the
Collections API RELOAD.
I have done few test changing synonyms dictionary or solrconfig and
everything works fine.
Well, I think you already know, but looking at solr.log file after the
collections api reload call, I have seen a bunch of lines like this one:
- Collection Admin sending CoreAdmin cmd to http://192.168.101.118:8080/solr
params:action=RELOAD&core=collection1_shard1_replica1&qt=%2Fadmin%2Fcores
...
Best regards and thanks again,
Vincenzo
On Fri, Dec 11, 2015 at 7:38 PM, Erick Erickson <er...@gmail.com>
wrote:
> You should absolutely always use the Collection API rather than
> any core admin API if at all possible. If for no other reason
> than your client will be _lots_ simpler (i.e. you don't have
> to find all the replicas and issue the core admin RELOAD
> command for each one).
>
> I'm not entirely sure whether the RELOAD command is
> synchronous or not though.
>
> Best,
> erick
>
> On Fri, Dec 11, 2015 at 8:22 AM, Vincenzo D'Amore <v....@gmail.com>
> wrote:
> > Hi all,
> >
> > in day by day work, often I need to change the solr configurations files.
> > Often adding new synonyms, changing the schema or the solrconfig.xml.
> >
> > Everything is stored in zookeeper.
> >
> > But I have inherited a piece of code that, after every change, reload all
> > the cores using CoreAdmin API.
> >
> > Now I have 15 replicas in the collection, and after every core reload the
> > code waits for 60 seconds (I suppose it's because who wrote the code was
> > worried about the cache invalidation).
> >
> > Given that, it takes about 25 minutes to update all the cores. Obviously
> > during this time we cannot modify the collection.
> >
> > The question is, to reduce this wait, if I use the collection API RELOAD,
> > what are the counter indication?
> >
> > Thanks in advance for your time,
> > Vincenzo
> >
> >
> > --
> > Vincenzo D'Amore
> > email: v.damore@gmail.com
> > skype: free.dev
> > mobile: +39 349 8513251
>
--
Vincenzo D'Amore
email: v.damore@gmail.com
skype: free.dev
mobile: +39 349 8513251
Re: Solrcloud 4.8.1 - Solr cores reload
Posted by Erick Erickson <er...@gmail.com>.
You should absolutely always use the Collection API rather than
any core admin API if at all possible. If for no other reason
than your client will be _lots_ simpler (i.e. you don't have
to find all the replicas and issue the core admin RELOAD
command for each one).
I'm not entirely sure whether the RELOAD command is
synchronous or not though.
Best,
erick
On Fri, Dec 11, 2015 at 8:22 AM, Vincenzo D'Amore <v....@gmail.com> wrote:
> Hi all,
>
> in day by day work, often I need to change the solr configurations files.
> Often adding new synonyms, changing the schema or the solrconfig.xml.
>
> Everything is stored in zookeeper.
>
> But I have inherited a piece of code that, after every change, reload all
> the cores using CoreAdmin API.
>
> Now I have 15 replicas in the collection, and after every core reload the
> code waits for 60 seconds (I suppose it's because who wrote the code was
> worried about the cache invalidation).
>
> Given that, it takes about 25 minutes to update all the cores. Obviously
> during this time we cannot modify the collection.
>
> The question is, to reduce this wait, if I use the collection API RELOAD,
> what are the counter indication?
>
> Thanks in advance for your time,
> Vincenzo
>
>
> --
> Vincenzo D'Amore
> email: v.damore@gmail.com
> skype: free.dev
> mobile: +39 349 8513251