Rotating the primary shard in /solr/select

[Phillip Farber <pf...@umich.edu>]

Is there any value in a round-robin scheme to cycle through the Solr 
instances supporting a multi-shard index over several machines when 
sending queries or is it better to just pick one instance and stick with 
it.  I'm assuming all machines in the cluster have the same hardware specs.

So scenario A (round-robin):

query 1: /solr-shard-1/select?q=dog... shards=shard-1,shard2
query 2: /solr-shard-2/select?q=dog... shards=shard-1,shard2
query 3: /solr-shard-1/select?q=dog... shards=shard-1,shard2
etc.

or or scenario B (fixed):

query 1: /solr-shard-1/select?q=dog... shards=shard-1,shard2
query 2: /solr-shard-1/select?q=dog... shards=shard-1,shard2
query 3: /solr-shard-1/select?q=dog... shards=shard-1,shard2
etc.

Is there evidence that distributing the overhead of result merging over 
more machines (A) gives a performance boost?

Thanks,

Phil

Re: Rotating the primary shard in /solr/select

[Shalin Shekhar Mangar <sh...@gmail.com>]

On Wed, Jul 29, 2009 at 2:57 AM, Phillip Farber <pf...@umich.edu> wrote:

>
> Is there any value in a round-robin scheme to cycle through the Solr
> instances supporting a multi-shard index over several machines when sending
> queries or is it better to just pick one instance and stick with it.  I'm
> assuming all machines in the cluster have the same hardware specs.
>
> So scenario A (round-robin):
>
> query 1: /solr-shard-1/select?q=dog... shards=shard-1,shard2
> query 2: /solr-shard-2/select?q=dog... shards=shard-1,shard2
> query 3: /solr-shard-1/select?q=dog... shards=shard-1,shard2
> etc.
>
> or or scenario B (fixed):
>
> query 1: /solr-shard-1/select?q=dog... shards=shard-1,shard2
> query 2: /solr-shard-1/select?q=dog... shards=shard-1,shard2
> query 3: /solr-shard-1/select?q=dog... shards=shard-1,shard2
> etc.
>
> Is there evidence that distributing the overhead of result merging over
> more machines (A) gives a performance boost?
>

We issue distributed search queries through a load balancer. So in effect,
the merging server (or aggregator) keeps changing. I don't know if that
leads to a performance boost or not but I guess spreading the load is a good
idea.

-- 
Regards,
Shalin Shekhar Mangar.

Re: Rotating the primary shard in /solr/select

[Rahul R <ra...@gmail.com>]

Shalin, thank you for the clarification.

Philip, I just realized that I have diverted the original topic of the
thread. My apologies.

Regards
Rahul

On Tue, Aug 4, 2009 at 3:35 PM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:

> On Tue, Aug 4, 2009 at 2:37 PM, Rahul R <ra...@gmail.com> wrote:
>
> > *The SolrServer is initialized to the server to which you want to send
> the
> > request. It has nothing to do with distributed search by itself.*
> >
> > But isn't the request sent to all the shards ? We set all the shard urls
> in
> > the 'shards' parameter of our HttpRequest.Or is it something like the
> > request is first sent to the server (with which SolrServer is
> initialized)
> > and from there it is sent to all the other shards ?
> >
>
> The request is sent to the server with which SolrServer is initialized.
> That
> server makes use of the shards parameter, queries other servers, merges the
> responses and sends it back to the client.
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

Re: Rotating the primary shard in /solr/select

[Shalin Shekhar Mangar <sh...@gmail.com>]

On Tue, Aug 4, 2009 at 2:37 PM, Rahul R <ra...@gmail.com> wrote:

> *The SolrServer is initialized to the server to which you want to send the
> request. It has nothing to do with distributed search by itself.*
>
> But isn't the request sent to all the shards ? We set all the shard urls in
> the 'shards' parameter of our HttpRequest.Or is it something like the
> request is first sent to the server (with which SolrServer is initialized)
> and from there it is sent to all the other shards ?
>

The request is sent to the server with which SolrServer is initialized. That
server makes use of the shards parameter, queries other servers, merges the
responses and sends it back to the client.

-- 
Regards,
Shalin Shekhar Mangar.

Re: Rotating the primary shard in /solr/select

[Rahul R <ra...@gmail.com>]

*The SolrServer is initialized to the server to which you want to send the
request. It has nothing to do with distributed search by itself.*

But isn't the request sent to all the shards ? We set all the shard urls in
the 'shards' parameter of our HttpRequest.Or is it something like the
request is first sent to the server (with which SolrServer is initialized)
and from there it is sent to all the other shards ?

Regards
Rahul
On Tue, Aug 4, 2009 at 2:29 PM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:

> On Tue, Aug 4, 2009 at 11:26 AM, Rahul R <ra...@gmail.com> wrote:
>
> > Philip,
> > I cannot answer your question, but I do have a question for you. Does
> > aggregation happen at the primary shard ? For eg : if I have three JVMs
> > JVM 1 : My application powered by Solr
> > JVM 2 : Shard 1
> > JVM 3 : Shard 2
> >
> > I initialize my SolrServer like this
> > SolrServer _solrServer = *new* CommonsHttpSolrServer(shard1);
> >
> > Does aggregation now happen at JVM 2 ?
>
>
> Yes.
>
>
> > Is there any other reason for
> > initializing the SolrServer with one of the shard URLs ?
> >
>
> The SolrServer is initialized to the server to which you want to send the
> request. It has nothing to do with distributed search by itself.
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

Re: Rotating the primary shard in /solr/select

[Shalin Shekhar Mangar <sh...@gmail.com>]

On Tue, Aug 4, 2009 at 11:26 AM, Rahul R <ra...@gmail.com> wrote:

> Philip,
> I cannot answer your question, but I do have a question for you. Does
> aggregation happen at the primary shard ? For eg : if I have three JVMs
> JVM 1 : My application powered by Solr
> JVM 2 : Shard 1
> JVM 3 : Shard 2
>
> I initialize my SolrServer like this
> SolrServer _solrServer = *new* CommonsHttpSolrServer(shard1);
>
> Does aggregation now happen at JVM 2 ?


Yes.


> Is there any other reason for
> initializing the SolrServer with one of the shard URLs ?
>

The SolrServer is initialized to the server to which you want to send the
request. It has nothing to do with distributed search by itself.

-- 
Regards,
Shalin Shekhar Mangar.

Re: Rotating the primary shard in /solr/select

[Rahul R <ra...@gmail.com>]

Philip,
I cannot answer your question, but I do have a question for you. Does
aggregation happen at the primary shard ? For eg : if I have three JVMs
JVM 1 : My application powered by Solr
JVM 2 : Shard 1
JVM 3 : Shard 2

I initialize my SolrServer like this
SolrServer _solrServer = *new* CommonsHttpSolrServer(shard1);

Does aggregation now happen at JVM 2 ? Is there any other reason for
initializing the SolrServer with one of the shard URLs ?

On Wed, Jul 29, 2009 at 2:57 AM, Phillip Farber <pf...@umich.edu> wrote:

>
> Is there any value in a round-robin scheme to cycle through the Solr
> instances supporting a multi-shard index over several machines when sending
> queries or is it better to just pick one instance and stick with it.  I'm
> assuming all machines in the cluster have the same hardware specs.
>
> So scenario A (round-robin):
>
> query 1: /solr-shard-1/select?q=dog... shards=shard-1,shard2
> query 2: /solr-shard-2/select?q=dog... shards=shard-1,shard2
> query 3: /solr-shard-1/select?q=dog... shards=shard-1,shard2
> etc.
>
> or or scenario B (fixed):
>
> query 1: /solr-shard-1/select?q=dog... shards=shard-1,shard2
> query 2: /solr-shard-1/select?q=dog... shards=shard-1,shard2
> query 3: /solr-shard-1/select?q=dog... shards=shard-1,shard2
> etc.
>
> Is there evidence that distributing the overhead of result merging over
> more machines (A) gives a performance boost?
>
> Thanks,
>
> Phil
>
>
>