You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Anoop Bhatti <an...@gmail.com> on 2008/10/08 17:23:45 UTC

solr + zookeeper

Hello,

I am evaluating Solr on a small cluster (~8 nodes) and will likely
deploy it to a cluster of ~128 nodes.  Of course, configuration
management will be a big concern.  Has anyone used Zookeeper for a
Solr cluster?  I'm looking for any documentation, tutorials, etc. on
how to setup Zookeeper for a Solr cluster, and any architecture
recommendations.

Thanks,

Anoop Bhatti
--
Committed to open source technology.

Re: solr + zookeeper

Posted by Aleksandar Bradic <al...@vast.com>.
Hi,

In such setup, you would definitely want to use zookeeper in  
conjunction with Solr for at least two things :

a) deploy management / (having cron-based pulls for a 128-node setup  
could be a nightmare without appropriate sync guarantees,  
coordination, etc)
b) configuration distribution / (having solr boxes registering on  
zookeeper and receiving dynamical configuration updates, etc - you  
would definitely want this in such setup)

You could trivially implement stuff like a) on sysadmin-level by  
wrapping up execution of appropriate solr bin/ scripts inside other  
pieces of code that schedule their execution by synchronizing on  
appropriate structures in zookeeper (there are command-line tools for  
basic zookeeper sync, which might help you get started)

b)-style problems could be solved much more elegant (than on sysadmin- 
level), by implementing Solr module which would use zookeeper's async  
api for the general procedures of conf-updates, core swaps, etc.  
However, as far as I know, no such solution currently exists  
("dynamical refresh of Solr conf from zookeeper in runtime") - so I  
guess if you really want it... you would have to custom-code it (or  
just solve the problem the bunch-of-scripts-synchronizing-with- 
zookeeper-copying-files-and-restarting-solr-when-needed - way :) )

.Alek

On Oct 8, 2008, at 5:23 PM, Anoop Bhatti wrote:

> Hello,
>
> I am evaluating Solr on a small cluster (~8 nodes) and will likely
> deploy it to a cluster of ~128 nodes.  Of course, configuration
> management will be a big concern.  Has anyone used Zookeeper for a
> Solr cluster?  I'm looking for any documentation, tutorials, etc. on
> how to setup Zookeeper for a Solr cluster, and any architecture
> recommendations.
>
> Thanks,
>
> Anoop Bhatti
> --
> Committed to open source technology.


Re: solr + zookeeper

Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Hi Anoop,

If you are willing to go a little cutting edge, you might want to look at
https://issues.apache.org/jira/browse/SOLR-561

The patch is a little out of date. I shall post a new one soon. The patch
allows for index and configuration replication (except for solrconfig.xml on
which we are working).

On Wed, Oct 8, 2008 at 8:53 PM, Anoop Bhatti <an...@gmail.com> wrote:

> Hello,
>
> I am evaluating Solr on a small cluster (~8 nodes) and will likely
> deploy it to a cluster of ~128 nodes.  Of course, configuration
> management will be a big concern.  Has anyone used Zookeeper for a
> Solr cluster?  I'm looking for any documentation, tutorials, etc. on
> how to setup Zookeeper for a Solr cluster, and any architecture
> recommendations.
>
> Thanks,
>
> Anoop Bhatti
> --
> Committed to open source technology.
>



-- 
Regards,
Shalin Shekhar Mangar.