You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Martin Grotzke <ma...@googlemail.com> on 2010/12/08 14:48:25 UTC
Multicore and Replication (scripts vs. java, spellchecker)
Hi,
we're just planning to move from our replicated single index setup to
a replicated setup with multiple cores.
We're going to start with 2 cores, but the number of cores may
change/increase over time.
Our replication is still based on scripts/rsync, and I'm wondering if
it's worth moving to java based replication.
AFAICS the main advantage is simplicity, as with scripts based
replication our operations team would have to maintain rsync daemons /
cron jobs for each core.
Therefore my own preference would be to drop scripts and chose the
java based replication.
I'd just wanted to ask for experiences with the one or another in a
multicore setup. What do you say?
Another question is regarding spellchecker replication. I know there's
#SOLR-433 "MultiCore and SpellChecker replication" [1]. Based on the
status of this feature request I'd asume that the normal procedure of
keeping the spellchecker index up2date would be running a cron job on
each node/slave that updates the spellchecker.
Is that right?
And a final one: are there other things we should be aware of / keep
in mind when planning the migration to multiple cores?
(Ok, I'm risking to get "ask specific questions!" as an answer, but
perhaps s.o. has interesting, related stories to tell :-))
Thanx in advance,
cheers,
Martin
[1] https://issues.apache.org/jira/browse/SOLR-433
Re: Multicore and Replication (scripts vs. java, spellchecker)
Posted by Martin Grotzke <ma...@googlemail.com>.
Hi,
that there's no feedback indicates that our plans/preferences are
fine. Otherwise it's now a good opportunity to feed back :-)
Cheers,
Martin
On Wed, Dec 8, 2010 at 2:48 PM, Martin Grotzke
<ma...@googlemail.com> wrote:
> Hi,
>
> we're just planning to move from our replicated single index setup to
> a replicated setup with multiple cores.
> We're going to start with 2 cores, but the number of cores may
> change/increase over time.
>
> Our replication is still based on scripts/rsync, and I'm wondering if
> it's worth moving to java based replication.
> AFAICS the main advantage is simplicity, as with scripts based
> replication our operations team would have to maintain rsync daemons /
> cron jobs for each core.
> Therefore my own preference would be to drop scripts and chose the
> java based replication.
>
> I'd just wanted to ask for experiences with the one or another in a
> multicore setup. What do you say?
>
> Another question is regarding spellchecker replication. I know there's
> #SOLR-433 "MultiCore and SpellChecker replication" [1]. Based on the
> status of this feature request I'd asume that the normal procedure of
> keeping the spellchecker index up2date would be running a cron job on
> each node/slave that updates the spellchecker.
> Is that right?
>
> And a final one: are there other things we should be aware of / keep
> in mind when planning the migration to multiple cores?
> (Ok, I'm risking to get "ask specific questions!" as an answer, but
> perhaps s.o. has interesting, related stories to tell :-))
>
> Thanx in advance,
> cheers,
> Martin
>
>
> [1] https://issues.apache.org/jira/browse/SOLR-433
>
--
Martin Grotzke
http://www.javakaffee.de/blog/
Re: Multicore and Replication (scripts vs. java, spellchecker)
Posted by Martin Grotzke <ma...@googlemail.com>.
On Sat, Dec 11, 2010 at 12:38 AM, Chris Hostetter
<ho...@fucit.org> wrote:
>
> : #SOLR-433 "MultiCore and SpellChecker replication" [1]. Based on the
> : status of this feature request I'd asume that the normal procedure of
> : keeping the spellchecker index up2date would be running a cron job on
> : each node/slave that updates the spellchecker.
> : Is that right?
>
> i'm not 100% certain, but i suspect a lot of people just build the
> spellcheck dictionaries on the slave machines (redundently) using
> buildOnCommit....
>
> http://wiki.apache.org/solr/SpellCheckComponent#Building_on_Commits
Ok, also a good option. Though, for us this is not that perfect
because we have 4 different spellcheckers configured so that this
would eat some cpu that we'd prefer to have left for searching.
I think what would be desirable (in our case) is s.th. like rebuilding
the spellchecker based on a cron expression, so that we could recreate
it e.g. every night at 1 am.
When thinking about creating s.th. like this, do you have some advice
where I could have a look at in solr? Is there already some
"framework" for running regular tasks, or should I pull up my own
Timer/TimerTask etc. and create it from scratch?
Cheers,
Martin
>
>
>
>
>
>
> -Hoss
>
--
Martin Grotzke
http://www.javakaffee.de/blog/
Re: Multicore and Replication (scripts vs. java, spellchecker)
Posted by Chris Hostetter <ho...@fucit.org>.
: #SOLR-433 "MultiCore and SpellChecker replication" [1]. Based on the
: status of this feature request I'd asume that the normal procedure of
: keeping the spellchecker index up2date would be running a cron job on
: each node/slave that updates the spellchecker.
: Is that right?
i'm not 100% certain, but i suspect a lot of people just build the
spellcheck dictionaries on the slave machines (redundently) using
buildOnCommit....
http://wiki.apache.org/solr/SpellCheckComponent#Building_on_Commits
-Hoss