You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Shashank Pedamallu <sp...@vmware.com> on 2017/05/02 16:53:11 UTC
Reload an unloaded core
Hi all,
I want to unload a core from Solr without deleting data-dir or instance-dir. I’m performing some operations on the data-dir after this and then I would like to reload the core from the same data-dir. These are the things I tried:
1. Reload api – throws an exception saying no such core exists.
2. Create api – throws an exception saying a core with given name already exists.
Can someone point me what api I could use to achieve this. Please note that, I’m working with Solr in Non-Cloud mode without Zookeeper, Collections, etc.
Thanks in advance!
Thanks,
Shashank Pedamallu
Re: Reload an unloaded core
Posted by David Lee <ni...@comcast.net>.
I have similar needs but for a slightly different use-case.
In my case, I am breaking up cores / indexes based on the month and year
so that I can add an alias that always points to the last few months,
but beyond that I want to simply unload the other indexes once they get
past a few months old. The indexes will remain on disk but I simply
don't want my queries to have to go through the older "archived" documents.
However, users will occasionally need to have those indexes reloaded for
research reasons so what I was doing in ES was simply re-loading all of
the indexes that fit within the range being searched for and added those
to an alias (let's call it "archived", for example). Once they are
finished querying on that older data, I again unload those indexes and
remove the alias.
From what I'm reading in this thread, this isn't quite as
straight-forward in Solr so I'm looking for other options.
Thanks,
David
On 5/2/2017 5:04 PM, Shashank Pedamallu wrote:
> Thank you Simon, Erick and Shawn for your replies. Unfortunately, restarting Solr is not a option for me. So, I’ll try to follow the steps given by Shawn to see where I’m standing. Btw, I’m using Solr 6.4.2.
>
> Shawn, once again thank you very much for the detailed reply.
>
> Thanks,
> Shashank Pedamallu
>
>
>
>
>
>
>
> On 5/2/17, 2:51 PM, "Shawn Heisey" <ap...@elyograg.org> wrote:
>
>> On 5/2/2017 10:53 AM, Shashank Pedamallu wrote:
>>> I want to unload a core from Solr without deleting data-dir or instance-dir. I’m performing some operations on the data-dir after this and then I would like to reload the core from the same data-dir. These are the things I tried:
>>>
>>> 1. Reload api – throws an exception saying no such core exists.
>>> 2. Create api – throws an exception saying a core with given name already exists.
>>>
>>> Can someone point me what api I could use to achieve this. Please note that, I’m working with Solr in Non-Cloud mode without Zookeeper, Collections, etc.
>> The RELOAD command isn't going to work at all because the core has been
>> unloaded -- Solr doesn't know about the core, so it can't reload it.
>> This is a case where the language used is somewhat confusing, even
>> though it's completely correct.
>>
>> I am about 90 percent certain that the reason the CREATE command gave
>> you an error message is because you tried to make a new core.properties
>> file before you did the CREATE. When things are working correctly, the
>> CREATE command itself is what will create core.properties. If it
>> already exists, CoreAdmin will give you an error. This is the exact
>> text of the error I encountered when trying to use CREATE after building
>> a core.properties file manually:
>>
>> Error CREATEing SolrCore 'foo': Could not create a new core in
>> C:\Users\sheisey\Downloads\solr-6.5.1\server\solr\fooas another core is
>> already defined there
>>
>> That error message is confusing, so I will be fixing it:
>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_SOLR-2D10599&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=NQtabafDgjFF7o57qRBjFMUG9UDK92RE8pAtrnHmuTs&s=YimxLbwsjFGBwV4_LR5LK4yXu_uafFvvVujg-7MDJFE&e=
>>
>> To verify what you need to do, I fired up Solr 6.5.1 from an extracted
>> download directory. I created two cores, "foo" and "bar", using the
>> commandline "bin\solr create" command. Then I went to the admin UI and
>> unloaded foo. The foo directory was still there, but the core was gone
> >from Solr's list.
>> By clicking on the "Add Core" button in the Core Admin tab, typing "foo"
>> into name and instanceDir, and clearing the other text boxes, the core
>> was recreated exactly as it was before it was unloaded.
>>
>> This is the log from the CREATE command that the admin UI sent:
>>
>> 2017-05-02 18:02:49.232 INFO (qtp1543727556-18) [ x:foo]
>> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
>> params={schema=&dataDir=&name=foo&action=CREATE&config=&instanceDir=foo&wt=json&_=1493747904891}
>> status=0 QTime=396
>>
>> To double-check this and show how it can be done without the admin UI, I
>> accessed these two URLs (in a browser), and accomplished the exact same
>> thing again. The first URL unloads the core, the second asks Solr to
>> find the core and re-add it with default settings.
>>
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8983_solr_admin_cores-3Faction-3DUNLOAD-26core-3Dfoo&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=NQtabafDgjFF7o57qRBjFMUG9UDK92RE8pAtrnHmuTs&s=CkhSi_Ik3vbgx1UYDGYcifbIuN8GUpc64dpm_hxYy8U&e=
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8983_solr_admin_cores-3Faction-3DCREATE-26name-3Dfoo-26instanceDir-3Dfoo&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=NQtabafDgjFF7o57qRBjFMUG9UDK92RE8pAtrnHmuTs&s=vORlMj_KMQCCLbYsbQM3t2y5Fy8i6IKI0zw3O8zsbcI&e=
>>
>> If you are using additional options with your cores, such as the
>> configset parameter, you would need to include those options on your
>> CREATE call, similar to what you might have done when you initially
>> created the core. With some of the options you can use, re-adding a
>> core might be better done by re-creating the correct core.properties
>> file and restarting Solr so it discovers the core.
>>
>> Erick, I seem to remember the "core.properties.unloaded" rename
>> happening in the past as well, but when I unloaded on 6.5.1, the
>> core.properties file was simply deleted. I don't think that's a good
>> idea because it may contain information that's not available anywhere else.
>>
>> Thanks,
>> Shawn
>>
---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
Re: Reload an unloaded core
Posted by Shashank Pedamallu <sp...@vmware.com>.
Thank you Simon, Erick and Shawn for your replies. Unfortunately, restarting Solr is not a option for me. So, I’ll try to follow the steps given by Shawn to see where I’m standing. Btw, I’m using Solr 6.4.2.
Shawn, once again thank you very much for the detailed reply.
Thanks,
Shashank Pedamallu
On 5/2/17, 2:51 PM, "Shawn Heisey" <ap...@elyograg.org> wrote:
>On 5/2/2017 10:53 AM, Shashank Pedamallu wrote:
>> I want to unload a core from Solr without deleting data-dir or instance-dir. I’m performing some operations on the data-dir after this and then I would like to reload the core from the same data-dir. These are the things I tried:
>>
>> 1. Reload api – throws an exception saying no such core exists.
>> 2. Create api – throws an exception saying a core with given name already exists.
>>
>> Can someone point me what api I could use to achieve this. Please note that, I’m working with Solr in Non-Cloud mode without Zookeeper, Collections, etc.
>
>The RELOAD command isn't going to work at all because the core has been
>unloaded -- Solr doesn't know about the core, so it can't reload it.
>This is a case where the language used is somewhat confusing, even
>though it's completely correct.
>
>I am about 90 percent certain that the reason the CREATE command gave
>you an error message is because you tried to make a new core.properties
>file before you did the CREATE. When things are working correctly, the
>CREATE command itself is what will create core.properties. If it
>already exists, CoreAdmin will give you an error. This is the exact
>text of the error I encountered when trying to use CREATE after building
>a core.properties file manually:
>
>Error CREATEing SolrCore 'foo': Could not create a new core in
>C:\Users\sheisey\Downloads\solr-6.5.1\server\solr\fooas another core is
>already defined there
>
>That error message is confusing, so I will be fixing it:
>
>https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_SOLR-2D10599&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=NQtabafDgjFF7o57qRBjFMUG9UDK92RE8pAtrnHmuTs&s=YimxLbwsjFGBwV4_LR5LK4yXu_uafFvvVujg-7MDJFE&e=
>
>To verify what you need to do, I fired up Solr 6.5.1 from an extracted
>download directory. I created two cores, "foo" and "bar", using the
>commandline "bin\solr create" command. Then I went to the admin UI and
>unloaded foo. The foo directory was still there, but the core was gone
>from Solr's list.
>
>By clicking on the "Add Core" button in the Core Admin tab, typing "foo"
>into name and instanceDir, and clearing the other text boxes, the core
>was recreated exactly as it was before it was unloaded.
>
>This is the log from the CREATE command that the admin UI sent:
>
>2017-05-02 18:02:49.232 INFO (qtp1543727556-18) [ x:foo]
>o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
>params={schema=&dataDir=&name=foo&action=CREATE&config=&instanceDir=foo&wt=json&_=1493747904891}
>status=0 QTime=396
>
>To double-check this and show how it can be done without the admin UI, I
>accessed these two URLs (in a browser), and accomplished the exact same
>thing again. The first URL unloads the core, the second asks Solr to
>find the core and re-add it with default settings.
>
>https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8983_solr_admin_cores-3Faction-3DUNLOAD-26core-3Dfoo&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=NQtabafDgjFF7o57qRBjFMUG9UDK92RE8pAtrnHmuTs&s=CkhSi_Ik3vbgx1UYDGYcifbIuN8GUpc64dpm_hxYy8U&e=
>https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8983_solr_admin_cores-3Faction-3DCREATE-26name-3Dfoo-26instanceDir-3Dfoo&d=DwIDaQ&c=uilaK90D4TOVoH58JNXRgQ&r=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI&m=NQtabafDgjFF7o57qRBjFMUG9UDK92RE8pAtrnHmuTs&s=vORlMj_KMQCCLbYsbQM3t2y5Fy8i6IKI0zw3O8zsbcI&e=
>
>If you are using additional options with your cores, such as the
>configset parameter, you would need to include those options on your
>CREATE call, similar to what you might have done when you initially
>created the core. With some of the options you can use, re-adding a
>core might be better done by re-creating the correct core.properties
>file and restarting Solr so it discovers the core.
>
>Erick, I seem to remember the "core.properties.unloaded" rename
>happening in the past as well, but when I unloaded on 6.5.1, the
>core.properties file was simply deleted. I don't think that's a good
>idea because it may contain information that's not available anywhere else.
>
>Thanks,
>Shawn
>
Re: Reload an unloaded core
Posted by Shawn Heisey <ap...@elyograg.org>.
On 5/2/2017 10:53 AM, Shashank Pedamallu wrote:
> I want to unload a core from Solr without deleting data-dir or instance-dir. I’m performing some operations on the data-dir after this and then I would like to reload the core from the same data-dir. These are the things I tried:
>
> 1. Reload api – throws an exception saying no such core exists.
> 2. Create api – throws an exception saying a core with given name already exists.
>
> Can someone point me what api I could use to achieve this. Please note that, I’m working with Solr in Non-Cloud mode without Zookeeper, Collections, etc.
The RELOAD command isn't going to work at all because the core has been
unloaded -- Solr doesn't know about the core, so it can't reload it.
This is a case where the language used is somewhat confusing, even
though it's completely correct.
I am about 90 percent certain that the reason the CREATE command gave
you an error message is because you tried to make a new core.properties
file before you did the CREATE. When things are working correctly, the
CREATE command itself is what will create core.properties. If it
already exists, CoreAdmin will give you an error. This is the exact
text of the error I encountered when trying to use CREATE after building
a core.properties file manually:
Error CREATEing SolrCore 'foo': Could not create a new core in
C:\Users\sheisey\Downloads\solr-6.5.1\server\solr\fooas another core is
already defined there
That error message is confusing, so I will be fixing it:
https://issues.apache.org/jira/browse/SOLR-10599
To verify what you need to do, I fired up Solr 6.5.1 from an extracted
download directory. I created two cores, "foo" and "bar", using the
commandline "bin\solr create" command. Then I went to the admin UI and
unloaded foo. The foo directory was still there, but the core was gone
from Solr's list.
By clicking on the "Add Core" button in the Core Admin tab, typing "foo"
into name and instanceDir, and clearing the other text boxes, the core
was recreated exactly as it was before it was unloaded.
This is the log from the CREATE command that the admin UI sent:
2017-05-02 18:02:49.232 INFO (qtp1543727556-18) [ x:foo]
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
params={schema=&dataDir=&name=foo&action=CREATE&config=&instanceDir=foo&wt=json&_=1493747904891}
status=0 QTime=396
To double-check this and show how it can be done without the admin UI, I
accessed these two URLs (in a browser), and accomplished the exact same
thing again. The first URL unloads the core, the second asks Solr to
find the core and re-add it with default settings.
http://localhost:8983/solr/admin/cores?action=UNLOAD&core=foo
http://localhost:8983/solr/admin/cores?action=CREATE&name=foo&instanceDir=foo
If you are using additional options with your cores, such as the
configset parameter, you would need to include those options on your
CREATE call, similar to what you might have done when you initially
created the core. With some of the options you can use, re-adding a
core might be better done by re-creating the correct core.properties
file and restarting Solr so it discovers the core.
Erick, I seem to remember the "core.properties.unloaded" rename
happening in the past as well, but when I unloaded on 6.5.1, the
core.properties file was simply deleted. I don't think that's a good
idea because it may contain information that's not available anywhere else.
Thanks,
Shawn
Re: Reload an unloaded core
Posted by simon <mt...@gmail.com>.
the core properties definitely disappears if you use a configset, as in
#####
#Written by CorePropertiesLocator
#Tue May 02 20:19:40 UTC 2017
name=testcore
dataDir=/indexes/solrindexes/testcore
configSet=myconf
Using a conf directory, as in
#Written by CorePropertiesLocator
#Tue May 02 20:30:44 UTC 2017
name=testcorewithconf
schema=conf/schema.xml
dataDir=/indexes/solrindexes/testcorewithconf
has the same behavior.
This is Solr 6.3.0 standalone, and I share your memory that at one point in
the distant past core.properties was renamed on an unload.
Probably worth submitting a JIRA
-Simon
On Tue, May 2, 2017 at 4:04 PM, Erick Erickson <er...@gmail.com>
wrote:
> IIRC, the core.properties file _is_ renamed to
> core.properties.unloaded or something like that.
>
> Yeah, this is something of a pain. The inverse of "unload" is "create"
> but you have to know exactly how to create a core, and in SolrCloud
> mode that's...interesting. It's much safer to bring the Solr node
> down, do what you want then start it up, although not always possible.
>
> Best,
> Erick
>
> On Tue, May 2, 2017 at 10:55 AM, simon <mt...@gmail.com> wrote:
> > I ran into the exact same situation recently. I unloaded from the
> browser
> > GUI which does not delete the data or instance dirs, but does delete
> > core.properties. I couldn't find any API either so I eventually
> manually
> > recreated core.properties and restarted Solr.
> >
> > Would be nice if the core.properties file were to be renamed rather than
> > deleted and if there were a RESCAN action to scan for unloaded cores and
> > reload them.
> >
> > On Tue, May 2, 2017 at 12:53 PM, Shashank Pedamallu <
> spedamallu@vmware.com>
> > wrote:
> >
> >> Hi all,
> >>
> >> I want to unload a core from Solr without deleting data-dir or
> >> instance-dir. I’m performing some operations on the data-dir after this
> and
> >> then I would like to reload the core from the same data-dir. These are
> the
> >> things I tried:
> >>
> >> 1. Reload api – throws an exception saying no such core exists.
> >> 2. Create api – throws an exception saying a core with given name
> >> already exists.
> >>
> >> Can someone point me what api I could use to achieve this. Please note
> >> that, I’m working with Solr in Non-Cloud mode without Zookeeper,
> >> Collections, etc.
> >>
> >> Thanks in advance!
> >>
> >> Thanks,
> >> Shashank Pedamallu
> >>
>
Re: Reload an unloaded core
Posted by Erick Erickson <er...@gmail.com>.
IIRC, the core.properties file _is_ renamed to
core.properties.unloaded or something like that.
Yeah, this is something of a pain. The inverse of "unload" is "create"
but you have to know exactly how to create a core, and in SolrCloud
mode that's...interesting. It's much safer to bring the Solr node
down, do what you want then start it up, although not always possible.
Best,
Erick
On Tue, May 2, 2017 at 10:55 AM, simon <mt...@gmail.com> wrote:
> I ran into the exact same situation recently. I unloaded from the browser
> GUI which does not delete the data or instance dirs, but does delete
> core.properties. I couldn't find any API either so I eventually manually
> recreated core.properties and restarted Solr.
>
> Would be nice if the core.properties file were to be renamed rather than
> deleted and if there were a RESCAN action to scan for unloaded cores and
> reload them.
>
> On Tue, May 2, 2017 at 12:53 PM, Shashank Pedamallu <sp...@vmware.com>
> wrote:
>
>> Hi all,
>>
>> I want to unload a core from Solr without deleting data-dir or
>> instance-dir. I’m performing some operations on the data-dir after this and
>> then I would like to reload the core from the same data-dir. These are the
>> things I tried:
>>
>> 1. Reload api – throws an exception saying no such core exists.
>> 2. Create api – throws an exception saying a core with given name
>> already exists.
>>
>> Can someone point me what api I could use to achieve this. Please note
>> that, I’m working with Solr in Non-Cloud mode without Zookeeper,
>> Collections, etc.
>>
>> Thanks in advance!
>>
>> Thanks,
>> Shashank Pedamallu
>>
Re: Reload an unloaded core
Posted by simon <mt...@gmail.com>.
I ran into the exact same situation recently. I unloaded from the browser
GUI which does not delete the data or instance dirs, but does delete
core.properties. I couldn't find any API either so I eventually manually
recreated core.properties and restarted Solr.
Would be nice if the core.properties file were to be renamed rather than
deleted and if there were a RESCAN action to scan for unloaded cores and
reload them.
On Tue, May 2, 2017 at 12:53 PM, Shashank Pedamallu <sp...@vmware.com>
wrote:
> Hi all,
>
> I want to unload a core from Solr without deleting data-dir or
> instance-dir. I’m performing some operations on the data-dir after this and
> then I would like to reload the core from the same data-dir. These are the
> things I tried:
>
> 1. Reload api – throws an exception saying no such core exists.
> 2. Create api – throws an exception saying a core with given name
> already exists.
>
> Can someone point me what api I could use to achieve this. Please note
> that, I’m working with Solr in Non-Cloud mode without Zookeeper,
> Collections, etc.
>
> Thanks in advance!
>
> Thanks,
> Shashank Pedamallu
>