You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Andre Bois-Crettez <an...@kelkoo.com> on 2012/12/04 13:57:32 UTC
How to SWAP cores (or collections) with SolrCloud (SOLR-3866)
Hello,
With solr-4.0.0, the useful SWAP command
<http://wiki.apache.org/solr/CoreAdmin#SWAP> that allows to have a main
core serving searches, while a temp core can be re-indexed from scratch,
no longer works on SolrCloud, as was discussed here :Solr Swap Function
doesn't work when using Solr Cloud Beta
<https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201209.mbox/%3CCALB4QrMi_fjpES8oNyoEUsbk1E51SqScxg6aaEULmvDL0F21ig@mail.gmail.com%3E>
As we really need the ability to rebuild a complete index behind the
scenes without interrupting the search service, we tried some
workarounds, without success.
Typical setup is :
solr.xml:
<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true">
<cores defaultCoreName="core_main" adminPath="/admin/cores"
zkClientTimeout="${zkClientTimeout:15000}" hostPort="8983"
hostContext="solr">
<core shard="shard1" instanceDir="instance_main" name="core_main"
collection="collection_main" dataDir="data"/>
<core shard="shard1" instanceDir="instance_temp" name="core_temp"
collection="collection_temp" dataDir="data"/>
</cores>
</solr>
solrconfig.xml etc are all the same for all cores/collections.
We start 1 leader and 1 replica.
Excerpt of tests/workarounds that were tried (please ask if details are
needed) :
1.a switch instanceDir path in solr.xml, set /clusterstate.json {} (to
reset it), restart leader
1.bswitch instanceDir path and core names in solr.xml, set
/clusterstate.json {}, restart leader
-> solr leader and replica don't respond anymore when clusterstate.json
is empty
-> leader: KeeperErrorCode = NoNode for
/collections/collection_temp/leaders/shard1
-> replica: Could not find collection in zk: collection_main
2.a switch instanceDir path in solr.xml, RELOAD cores / collections
2.b switch instanceDir path and core names in solr.xml, RELOAD cores /
collections
-> No visible effect
3. RENAME core_temp to core_main
-> query on collection_main returns data initially indexed to core_temp
(what we want),but collection_main is no more in solr.xml and a 404
error is returned for any further document updates to collection_main
4. switch instanceDir directories on the filesystem, using a sequence of
dir moves, RELOAD cores
-> 500 error is returned for the replica cores RELOAD, with SEVERE error
in log, seems like solr did not like our messing with lucene files.
(Surprisingly, if there is no replica, this seems to work)
We also tried to use SYNCSHARD collection command and try to manually
set clusterstate.json with a new configuration but these workarounds
don't work.
Now we are running short of ideas to progress further...
It seems that we really need to patch solr to properly implement SWAP
for cores (or collections by the way) in the context of SolrCloud.
So,
* what can we do to help progress on SOLR-3866 ? Maybe use case
scenarios, detailing desired behavior ? Constrains on what cores or
collections are allowed to SWAP, ie. same config, same doc->shard
assignments ?
* our naive idea would be to update the /clusterstate.json config on Zk,
and propagate this configuration change to all solr instances. Is that
even remotely realistic ?
--
André Bois-Crettez
Search technology, Kelkoo
http://www.kelkoo.com/
Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 8, rue du Sentier 75002 Paris
425 093 069 RCS Paris
Ce message et les pièces jointes sont confidentiels et établis à l'attention exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce message, merci de le détruire et d'en avertir l'expéditeur.
Re: How to SWAP cores (or collections) with SolrCloud (SOLR-3866)
Posted by Andre Bois-Crettez <an...@kelkoo.com>.
On 12/05/2012 02:09 AM, Mark Miller wrote:
> On Dec 4, 2012, at 4:57 AM, Andre Bois-Crettez<an...@kelkoo.com> wrote:
>
>> * what can we do to help progress on SOLR-3866 ? Maybe use case
>> scenarios, detailing desired behavior ? Constrains on what cores or
>> collections are allowed to SWAP, ie. same config, same doc->shard
>> assignments ?
>
> Yes please - if you could elaborate on that issue, I can help you try and get something in.
>
> - Mark
Thanks.
I detailed our use case on the jira issue.
Hoping it is clear enough, let me know if anything more is needed.
--
André Bois-Crettez
Search technology, Kelkoo
http://www.kelkoo.com/
Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 8, rue du Sentier 75002 Paris
425 093 069 RCS Paris
Ce message et les pièces jointes sont confidentiels et établis à l'attention exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce message, merci de le détruire et d'en avertir l'expéditeur.
Re: How to SWAP cores (or collections) with SolrCloud (SOLR-3866)
Posted by Mark Miller <ma...@gmail.com>.
On Dec 4, 2012, at 4:57 AM, Andre Bois-Crettez <an...@kelkoo.com> wrote:
> * what can we do to help progress on SOLR-3866 ? Maybe use case
> scenarios, detailing desired behavior ? Constrains on what cores or
> collections are allowed to SWAP, ie. same config, same doc->shard
> assignments ?
Yes please - if you could elaborate on that issue, I can help you try and get something in.
- Mark