You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Anoop Bhatti <an...@gmail.com> on 2008/10/08 17:23:45 UTC
solr + zookeeper
Hello,
I am evaluating Solr on a small cluster (~8 nodes) and will likely
deploy it to a cluster of ~128 nodes. Of course, configuration
management will be a big concern. Has anyone used Zookeeper for a
Solr cluster? I'm looking for any documentation, tutorials, etc. on
how to setup Zookeeper for a Solr cluster, and any architecture
recommendations.
Thanks,
Anoop Bhatti
--
Committed to open source technology.
Re: solr + zookeeper
Posted by Aleksandar Bradic <al...@vast.com>.
Hi,
In such setup, you would definitely want to use zookeeper in
conjunction with Solr for at least two things :
a) deploy management / (having cron-based pulls for a 128-node setup
could be a nightmare without appropriate sync guarantees,
coordination, etc)
b) configuration distribution / (having solr boxes registering on
zookeeper and receiving dynamical configuration updates, etc - you
would definitely want this in such setup)
You could trivially implement stuff like a) on sysadmin-level by
wrapping up execution of appropriate solr bin/ scripts inside other
pieces of code that schedule their execution by synchronizing on
appropriate structures in zookeeper (there are command-line tools for
basic zookeeper sync, which might help you get started)
b)-style problems could be solved much more elegant (than on sysadmin-
level), by implementing Solr module which would use zookeeper's async
api for the general procedures of conf-updates, core swaps, etc.
However, as far as I know, no such solution currently exists
("dynamical refresh of Solr conf from zookeeper in runtime") - so I
guess if you really want it... you would have to custom-code it (or
just solve the problem the bunch-of-scripts-synchronizing-with-
zookeeper-copying-files-and-restarting-solr-when-needed - way :) )
.Alek
On Oct 8, 2008, at 5:23 PM, Anoop Bhatti wrote:
> Hello,
>
> I am evaluating Solr on a small cluster (~8 nodes) and will likely
> deploy it to a cluster of ~128 nodes. Of course, configuration
> management will be a big concern. Has anyone used Zookeeper for a
> Solr cluster? I'm looking for any documentation, tutorials, etc. on
> how to setup Zookeeper for a Solr cluster, and any architecture
> recommendations.
>
> Thanks,
>
> Anoop Bhatti
> --
> Committed to open source technology.
Re: solr + zookeeper
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Hi Anoop,
If you are willing to go a little cutting edge, you might want to look at
https://issues.apache.org/jira/browse/SOLR-561
The patch is a little out of date. I shall post a new one soon. The patch
allows for index and configuration replication (except for solrconfig.xml on
which we are working).
On Wed, Oct 8, 2008 at 8:53 PM, Anoop Bhatti <an...@gmail.com> wrote:
> Hello,
>
> I am evaluating Solr on a small cluster (~8 nodes) and will likely
> deploy it to a cluster of ~128 nodes. Of course, configuration
> management will be a big concern. Has anyone used Zookeeper for a
> Solr cluster? I'm looking for any documentation, tutorials, etc. on
> how to setup Zookeeper for a Solr cluster, and any architecture
> recommendations.
>
> Thanks,
>
> Anoop Bhatti
> --
> Committed to open source technology.
>
--
Regards,
Shalin Shekhar Mangar.