You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jie Sun <js...@yahoo.com> on 2013/04/16 22:02:40 UTC
solr 3.5 core rename issue
We just tried to use
.../solr/admin/cores?action=RENAME&core=core0&other=core5
to rename a core 'old' to 'new'.
After the request is done, the solr.xml has new core name, and the solr
admin shows the new core name in the list. But the index dir still has the
old name as the directory name. I looked into solr 3.5 code, this is what
the code does.
However, if I bounce tomcat/solr, when solr is started up, it creates new
index dir with 'new', and now of course there is no longer any document
returned if you search the core.
is this a bug? or did I miss anything?
thanks
Jie
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-3-5-core-rename-issue-tp4056425.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr 3.5 core rename issue
Posted by Jie Sun <js...@yahoo.com>.
yeah I realize using ${solr.core.name} for dataDir must be the cause for the
issue we see... it is fair to say the SWAP and RENAME just create an alias
that still points to the old datadir.
if they can not fix it then it is not a bug :-) at least we understand
exactly what is going on there.
thanks so much for your help!
Jie
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-3-5-core-rename-issue-tp4056425p4057037.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr 3.5 core rename issue
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/17/2013 7:07 PM, Jie Sun wrote:
> thanks Shawn for filing the issue.
>
> by the way my solrconfig.xml has:
>
> <dataDir>${MYSOLRROOT:/mysolrroot}/messages/solr/data/${solr.core.name}</dataDir>
>
> For now I will have to shutdown solr and write a script to modify the
> solr.xml manually and rename the core data directory to new one.
>
> by the way when I try to remove a core using unload (I am using solr 3.5):
>
> .../solr/admin/cores?action=UNLOAD&core=4130&deleteIndex=true
>
> it removes the core from solr.xml, but it leaves the data directory '413',
> but the index subfolder under 413 is removed, however there are
> spellchecker1 and spellchecker2 still remain.
The dataDir option you have in solrconfig.xml completely explains why
this is happening. One detail regarding RENAME and SWAP is that the
'solr.core.name' property is never updated unless you completely restart
Solr. I have closed the SOLR-4372 as invalid. This is not a bug, it's
a side-effect of how the CoreAdmin API works.
For RENAME and SWAP to work correctly through a Solr restart, you will
need to include the dataDir option in all your core definitions in
solr.xml and remove it from solrconfig.xml. If dataDir is specified in
solr.xml, then the correct dataDir will be associated with the correct
core after restart.
You won't be able to use solr.core.name in your dataDir tags, because
that doesn't exist at the solr.xml level, and even if it did exist, it
would be wrong after a restart, and you would have the same problem
you're having now.
When you rename a core, it will always retain the old dataDir. I am
pretty sure there is no way to fix this, but if I'm wrong, I'm sure that
someone will let me know.
I ran into the problem with solr.core.name in 3.5.0, because my
PingRequestHandler enable/disable functionality included this property
in the enable filename, and it would always use the old one until Solr
was restarted. Solr 4.x solved this problem for me by moving the enable
file from the servlet container's current working directory to dataDir.
When you use the deleteIndex parameter on a core UNLOAD, the index is
all it will delete. Solr 4.0 added deleteDataDir and deleteInstanceDir
parameters.
http://wiki.apache.org/solr/CoreAdmin#UNLOAD
Thanks,
Shawn
Re: solr 3.5 core rename issue
Posted by Jie Sun <js...@yahoo.com>.
thanks Shawn for filing the issue.
by the way my solrconfig.xml has:
<dataDir>${MYSOLRROOT:/mysolrroot}/messages/solr/data/${solr.core.name}</dataDir>
For now I will have to shutdown solr and write a script to modify the
solr.xml manually and rename the core data directory to new one.
by the way when I try to remove a core using unload (I am using solr 3.5):
.../solr/admin/cores?action=UNLOAD&core=4130&deleteIndex=true
it removes the core from solr.xml, but it leaves the data directory '413',
but the index subfolder under 413 is removed, however there are
spellchecker1 and spellchecker2 still remain.
Do you know why?
thanks
Jie
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-3-5-core-rename-issue-tp4056425p4056865.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr 3.5 core rename issue
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/16/2013 2:39 PM, Jie Sun wrote:
> <?xml version="1.0" encoding="UTF-8" ?>
> <solr persistent="true">
> <cores adminPath="/admin/cores">
> <core name="default" instanceDir="./"/>
> <core name="413a" instanceDir="./"/>
> <core name="blah" instanceDir="./"/>
> ...
> </cores>
> </solr>
>
> the command I ran was to rename from '413' to '413a'.
>
> when i debug through solr CoreAdminHandler, I notice the persistent flag
> only controls if the new data will be persisted to solr.xml or not, thus as
> you can see, it did changed my solr.xml, there is no problem here.
>
> But the index dir ends up with no change at all (still '413').
I filed an issue for this.
https://issues.apache.org/jira/browse/SOLR-4732
Thanks,
Shawn
Re: solr 3.5 core rename issue
Posted by Shawn Heisey <el...@elyograg.org>.
On 4/16/2013 4:48 PM, Shawn Heisey wrote:
Here's an excerpt from my solr.xml:
>
> <core name="s0live" instanceDir="/index/solr/cores/s0_1/"
> dataDir="/index/solr/data/s0_1"/>
> <core name="s0build" instanceDir="cores/s0_0/"
> dataDir="../../data/s0_0"/>
Sorry about the incomplete edit there. My solr.xml from 4.x uses
relative paths, but my 3.5 setup still has absolute paths. I was trying
to make the 3.5 excerpt have relative paths before I sent it, but I only
edited one of the lines. Here's the 4.2.1 excerpt, unedited:
<core loadOnStartup="true" instanceDir="cores/s0_1/"
transient="false" name="s0live" dataDir="../../data/s0_1"/>
<core loadOnStartup="true" instanceDir="cores/s0_0/"
transient="false" name="s0build" dataDir="../../data/s0_0"/>
The loadOnStartup and transient parameters are automatically added when
solr.xml gets updated by later Solr versions.
Thanks,
Shawn
Re: solr 3.5 core rename issue
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/16/2013 2:39 PM, Jie Sun wrote:
> Hi Shawn,
> I do have persistent="true" in my solr.xml:
>
> <?xml version="1.0" encoding="UTF-8" ?>
> <solr persistent="true">
> <cores adminPath="/admin/cores">
> <core name="default" instanceDir="./"/>
> <core name="413a" instanceDir="./"/>
> <core name="blah" instanceDir="./"/>
> ...
> </cores>
> </solr>
>
> the command I ran was to rename from '413' to '413a'.
I think I see the problem. You have three cores that all point to the
same instanceDir, and no dataDir parameter. Normally the dataDir
parameter defaults to "data" in the instanceDir, but perhaps if you have
multiple cores sharing the instanceDir, it will use the core name
instead. With this solr.xml, I can see why you're having a problem.
The solr.xml file doesn't tell Solr where the dataDir is.
If you set up an explicit dataDir option for each core, then it should
work out the way you expect it to. Here's an excerpt from my solr.xml:
<core name="s0live" instanceDir="/index/solr/cores/s0_1/"
dataDir="/index/solr/data/s0_1"/>
<core name="s0build" instanceDir="cores/s0_0/"
dataDir="../../data/s0_0"/>
You are correct about what happens with my directories on a swap, but
because solr.xml gets updated and has an explicit dataDir for each core,
everything works.
Thanks,
Shawn
Re: solr 3.5 core rename issue
Posted by Jie Sun <js...@yahoo.com>.
Hi Shawn,
I do have persistent="true" in my solr.xml:
<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true">
<cores adminPath="/admin/cores">
<core name="default" instanceDir="./"/>
<core name="413a" instanceDir="./"/>
<core name="blah" instanceDir="./"/>
...
</cores>
</solr>
the command I ran was to rename from '413' to '413a'.
when i debug through solr CoreAdminHandler, I notice the persistent flag
only controls if the new data will be persisted to solr.xml or not, thus as
you can see, it did changed my solr.xml, there is no problem here.
But the index dir ends up with no change at all (still '413'). I guess swap
will have similar issue, I bet your 's0_0' directory actually hold data for
core s0build, and s0_1 holds data for s0live after you swap them. Because I
dont see anywhere in CoreAdminHandler and CoreContainer code actually rename
the index directory. I might be wrong, but you can test and find out.
Jie
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-3-5-core-rename-issue-tp4056425p4056435.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr 3.5 core rename issue
Posted by Shawn Heisey <so...@elyograg.org>.
On 4/16/2013 2:02 PM, Jie Sun wrote:
> We just tried to use
> .../solr/admin/cores?action=RENAME&core=core0&other=core5
>
> to rename a core 'old' to 'new'.
>
> After the request is done, the solr.xml has new core name, and the solr
> admin shows the new core name in the list. But the index dir still has the
> old name as the directory name. I looked into solr 3.5 code, this is what
> the code does.
>
> However, if I bounce tomcat/solr, when solr is started up, it creates new
> index dir with 'new', and now of course there is no longer any document
> returned if you search the core.
>
> is this a bug? or did I miss anything?
If your solr.xml is missing the 'persistent' attribute on the <solr>
tag, or it is set to false, then I can imagine it behaving this way.
This must be set to true, or changes that you make with the core admin
API will not be written to disk, so they will not survive a restart.
<solr sharedLib="lib" persistent="true">
<cores adminPath="/admin/cores">
I haven't used the RENAME functionality, but I use the core SWAP feature
extensively. I have cores with names like s0live and s0build, but they
actually refer to directories with names like s0_0 and s0_1. When they
swap, the directory location of the index doesn't change, but it's like
I have renamed both of them with each other's name.
Thanks,
Shawn