You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Burton-West, Tom" <tb...@umich.edu> on 2008/12/02 22:34:08 UTC

Load balancing for distributed Solr

Hello all,

As I understand distributed Solr, a request for a distributed search
goes to a particular Solr instance with a list of arguments specifying
the addresses of the shards to search.  The Solr instance to which the
request is first directed is responsible for distributing the query to
the other shards and pulling together the results.  My questions are:

1 Does it make sense to 
 A.  Always have the same Solr instance responsible for distributing the
query to the other shards
                   or 
 B.   Rotate which shard does the distributing/result aggregating?  

2. For scenario A, are there different requirements (memory,cpu,
processors etc) for the machine doing the distribution versus the
machines hosting the shards responding to the distributed requests?

3. For scenario B, are people using some kind of load balancing to
distribute which Solr instance acts as the query distributor/response
aggregator? 

Tom

Tom Burton-West
Information Retrieval Programmer
Digital Library Production Services
University of Michigan


 

Re: Load balancing for distributed Solr

Posted by Jeremy Hinegardner <je...@hinegardner.org>.
On Tue, Dec 02, 2008 at 04:34:08PM -0500, Burton-West, Tom wrote:
> Hello all,
> 
> As I understand distributed Solr, a request for a distributed search
> goes to a particular Solr instance with a list of arguments specifying
> the addresses of the shards to search.  The Solr instance to which the
> request is first directed is responsible for distributing the query to
> the other shards and pulling together the results.  My questions are:
> 
> 1 Does it make sense to 
>  A.  Always have the same Solr instance responsible for distributing the
> query to the other shards
>                    or 
>  B.   Rotate which shard does the distributing/result aggregating?  
> 
> 2. For scenario A, are there different requirements (memory,cpu,
> processors etc) for the machine doing the distribution versus the
> machines hosting the shards responding to the distributed requests?
> 
> 3. For scenario B, are people using some kind of load balancing to
> distribute which Solr instance acts as the query distributor/response
> aggregator? 

We use Scenario B, we have 4 Solr instances (4 machines), each with N data
SolrCores and 1 'default' Core which does the dispatch and aggregation of
requests between the 4*N total data cores.  We then use HAproxy to load balance
the requests between the dispatch Cores.

enjoy,

-jeremy

-- 
========================================================================
 Jeremy Hinegardner                              jeremy@hinegardner.org