Issue with Shard configuration in solrconfig.xml (Solr 3.1)

[Rahul Warawdekar <ra...@gmail.com>]

Hi,

I am trying to evaluate distributed search for my project by splitting up
our single index on 2 shards with Solr 3.1
When I query the first solr server by passing the "shards" parameter, I get
correct search results from both shards.
(
http://server1:8080/solr/test/select/?shards=server1:8080/solr/test,server2:8080/solr/test&q=solr&start=0&rows=20
)

I want to avoid the use of this shards parameter in the http url and specify
it in solrconfig.xml as follows.

<requestHandler name="my_custom_handler" class="solr.SearchHandler"
default="true">
    <str name="shards">server1:8080/solr/test,server2:8080/solr/test</str>
..
</requestHandler>

After adding the shards parameter in solrconfig.xml, I get search results
only from the first shard and not from the from the second one.
Am I missing any configuration ?

Also, can the urls with the shard parameter be load balanced for a failover
mechanism ?



-- 
Thanks and Regards
Rahul A. Warawdekar

Re: Issue with Shard configuration in solrconfig.xml (Solr 3.1)

[Shawn Heisey <so...@elyograg.org>]

On 10/20/2011 9:33 AM, Rahul Warawdekar wrote:
> Hi,
>
> I am trying to evaluate distributed search for my project by splitting up
> our single index on 2 shards with Solr 3.1
> When I query the first solr server by passing the "shards" parameter, I get
> correct search results from both shards.
> (
> http://server1:8080/solr/test/select/?shards=server1:8080/solr/test,server2:8080/solr/test&q=solr&start=0&rows=20
> )
>
> I want to avoid the use of this shards parameter in the http url and specify
> it in solrconfig.xml as follows.
>
> <requestHandler name="my_custom_handler" class="solr.SearchHandler"
> default="true">
>      <str name="shards">server1:8080/solr/test,server2:8080/solr/test</str>
> ..
> </requestHandler>
>
> After adding the shards parameter in solrconfig.xml, I get search results
> only from the first shard and not from the from the second one.
> Am I missing any configuration ?
>
> Also, can the urls with the shard parameter be load balanced for a failover
> mechanism ?

If you want to put the shards parameter in solrconfig.xml, it's 
important to make sure that the search handler is not referring to 
itself in its own shards parameter.  That leads to a problem with 
recursion.  I don't know if that's why you're only seeing results from 
one shard, but it's a starting point.

I've found a way around this problem by creating a completely separate 
core with the shards parameter in its solrconfig.xml referring to the 
other server/core combinations.  Alternatively, you could create a 
separate request handler on the same core.

Load balancing with only these two servers might increase your query 
capacity, but it won't do anything for redundancy.  If either server 
goes down, the whole index is effectively down, because distributed 
indexing returns an error if any shard doesn't work.  For redundancy, 
you'll need a second pair of servers and load balancing hardware or 
software.  You can then use Solr replication or have your build system 
take care of both sets.

Thanks,
Shawn

Re: Issue with Shard configuration in solrconfig.xml (Solr 3.1)

[Yury Kats <yu...@yahoo.com>]

On 10/20/2011 11:33 AM, Rahul Warawdekar wrote:
> Hi,
> 
> I am trying to evaluate distributed search for my project by splitting up
> our single index on 2 shards with Solr 3.1
> When I query the first solr server by passing the "shards" parameter, I get
> correct search results from both shards.
> (
> http://server1:8080/solr/test/select/?shards=server1:8080/solr/test,server2:8080/solr/test&q=solr&start=0&rows=20
> )
> 
> I want to avoid the use of this shards parameter in the http url and specify
> it in solrconfig.xml as follows.
> 
> <requestHandler name="my_custom_handler" class="solr.SearchHandler"
> default="true">
>     <str name="shards">server1:8080/solr/test,server2:8080/solr/test</str>
> ..
> </requestHandler>

Don't you need to wrap it in <lst name="default"> or <lst name="appends">?

> After adding the shards parameter in solrconfig.xml, I get search results
> only from the first shard and not from the from the second one.
> Am I missing any configuration ?

This means your 'shards' parameter is not being used, because it's not specified properly.

> Also, can the urls with the shard parameter be load balanced for a failover
> mechanism ?

See SolrCloud http://wiki.apache.org/solr/SolrCloud