You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by rosbaldeston <ro...@hotmail.com> on 2016/08/01 15:45:36 UTC
Replication with managed resources?
I've a single core index with a managed schema, synonyms and stopwords that
I'm thinking of making a master/slave pair via replication. How does does
the replication confFiles option work with managed resources?
Should I use their internal '_managed_xxx' filenames?
--
View this message in context: http://lucene.472066.n3.nabble.com/Replication-with-managed-resources-tp4289880.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Replication with managed resources?
Posted by rosbaldeston <ro...@hotmail.com>.
I was just running my own test and it seems it doesn't replicate or reload
the managed schema synonyms file. Not on a manual replication request after
a synonym change and not on an index change triggering an automatic
replication at least.
Used this as the slaves confFiles, not sure if this allows globs for the
language variants?
<str
name="confFiles">solrconfig.xml,managed-schema,_schema_analysis_stopwords_english.json,_schema_analysis_synonyms_english.json</str>
This is with a Solr 5.5, new schemas for both master & slave and all on
Centos 6.5 with Java 7.
--
View this message in context: http://lucene.472066.n3.nabble.com/Replication-with-managed-resources-tp4289880p4290248.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Replication with managed resources?
Posted by rosbaldeston <ro...@hotmail.com>.
Raised as https://issues.apache.org/jira/browse/SOLR-9382
--
View this message in context: http://lucene.472066.n3.nabble.com/Replication-with-managed-resources-tp4289880p4290386.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Replication with managed resources?
Posted by Erick Erickson <er...@gmail.com>.
bq: I'm also guessing those _schema and managed_schema files are an
implementation detail for the missing zookeeper functionality. But if I did
add those to a conffiles option it might automate the slave core reloads for
me?
You're getting closer ;). There's nothing Cloud specific about the whole
managed schema functionality, although that is where it's gotten the most
exercise so....
So if you're saying that you change the managed schema file on the
master and it is _not_ replicated automatically to the slave (you'll
have to have added docs I believe on the master, I don't think replication
happens just because of config changes) then I think that's worth a
JIRA, can you please confirm?
So if this sequence doesn't work:
1> change the managed schema
2> index some docs
3> wait for a replication
4> the managed schema file on the slave has _not_ been updated
then please raise a JIRA. Make sure you identify that this is stand-alone.
NOTE: I'm not sure what the right thing to do in this case is, but the JIRA
would allow a place to discuss what "the right thing" would be.
In the meantime, you should be able to work around that by explicitly listing
them in the conffiles section.
Best,
Erick
On Wed, Aug 3, 2016 at 8:58 AM, rosbaldeston <ro...@hotmail.com> wrote:
> Erick Erickson wrote
>> It Depends. When running in Cloud mode then "yes". If you're running
>> stand-alone
>> then there is no Zookeeper running so the answer is "no".
>
> Ah that helps, so no zookeeper in my case. I did wonder if it wasn't just
> sharing the same config files between master and slave from sharing the same
> configset. So it would appear I'm not replicating any of the managed files
> and reloading the slave core probably just reread the shared synonyms file.
>
> I'm also guessing those _schema and managed_schema files are an
> implementation detail for the missing zookeeper functionality. But if I did
> add those to a conffiles option it might automate the slave core reloads for
> me?
>
>
>> If a replication involved downloading of at least one configuration file,
>> the ReplicationHandler issues a core-reload command instead of a commit
>> command.
>
> (from https://cwiki.apache.org/confluence/display/solr/Index+Replication)
>
> Currently I've no conffiles set on the slave and I know it didn't get
> reloaded after synonym changes to the master.
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Replication-with-managed-resources-tp4289880p4290242.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Replication with managed resources?
Posted by rosbaldeston <ro...@hotmail.com>.
Erick Erickson wrote
> It Depends. When running in Cloud mode then "yes". If you're running
> stand-alone
> then there is no Zookeeper running so the answer is "no".
Ah that helps, so no zookeeper in my case. I did wonder if it wasn't just
sharing the same config files between master and slave from sharing the same
configset. So it would appear I'm not replicating any of the managed files
and reloading the slave core probably just reread the shared synonyms file.
I'm also guessing those _schema and managed_schema files are an
implementation detail for the missing zookeeper functionality. But if I did
add those to a conffiles option it might automate the slave core reloads for
me?
> If a replication involved downloading of at least one configuration file,
> the ReplicationHandler issues a core-reload command instead of a commit
> command.
(from https://cwiki.apache.org/confluence/display/solr/Index+Replication)
Currently I've no conffiles set on the slave and I know it didn't get
reloaded after synonym changes to the master.
--
View this message in context: http://lucene.472066.n3.nabble.com/Replication-with-managed-resources-tp4289880p4290242.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Replication with managed resources?
Posted by Erick Erickson <er...@gmail.com>.
bq: Am I right in saying managed resources are handled by zookeeper rather than
files on the filesystem....
It Depends. When running in Cloud mode then "yes". If you're running stand-alone
then there is no Zookeeper running so the answer is "no".
You can run Solr just like you always have in master/slave setups. In
that case you
need to manage your own configurations on every node just like you always have,
probably through replication.
In stand-alone mode, you should send all your managed schema API calls to the
master core and let the replication distribute the changes to the slaves.
Best,
Erick
On Wed, Aug 3, 2016 at 4:48 AM, rosbaldeston <ro...@hotmail.com> wrote:
> Am I right in saying managed resources are handled by zookeeper rather than
> files on the filesystem and I should ignore any files such as:
> managed-schema, _rest_managed.json,
> _schema_analysis_stopwords_english.json,
> _schema_analysis_synonyms_english.json ...
>
> I should not try to copy any of these via the slaves confFiles option?
>
> What I was planning to do was have the master as the indexing source and all
> slaves as query sources. But they need the same synonyms & stopwords.
>
> One thing I am seeing is when I create my master and slave from a custom
> configset without any copying of configs is when the masters synonyms have
> been changed the synonyms on the slave don't reflect these changes even
> sometime after after replication?
>
> It appears I need to reload the slave core(s) before they show the same
> synonyms as the master? is this because they're sharing the same file? how
> do should I keep slaves in sync with managed resources? do I just have to
> keep reloading all slave cores ever so often?
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Replication-with-managed-resources-tp4289880p4290177.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Replication with managed resources?
Posted by rosbaldeston <ro...@hotmail.com>.
Am I right in saying managed resources are handled by zookeeper rather than
files on the filesystem and I should ignore any files such as:
managed-schema, _rest_managed.json,
_schema_analysis_stopwords_english.json,
_schema_analysis_synonyms_english.json ...
I should not try to copy any of these via the slaves confFiles option?
What I was planning to do was have the master as the indexing source and all
slaves as query sources. But they need the same synonyms & stopwords.
One thing I am seeing is when I create my master and slave from a custom
configset without any copying of configs is when the masters synonyms have
been changed the synonyms on the slave don't reflect these changes even
sometime after after replication?
It appears I need to reload the slave core(s) before they show the same
synonyms as the master? is this because they're sharing the same file? how
do should I keep slaves in sync with managed resources? do I just have to
keep reloading all slave cores ever so often?
--
View this message in context: http://lucene.472066.n3.nabble.com/Replication-with-managed-resources-tp4289880p4290177.html
Sent from the Solr - User mailing list archive at Nabble.com.