You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Otis Gospodnetic <ot...@yahoo.com> on 2009/11/11 03:44:42 UTC
Re: Request assistance with distributed search multi shard/core setup and configuration
Hm, I don't follow. You don't need to create a custom (request) handler to make use of Solr's distributed search.
Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
----- Original Message ----
> From: "Turner, Robbin J" <Ro...@boeing.com>
> To: "solr-user@lucene.apache.org" <so...@lucene.apache.org>
> Sent: Tue, November 10, 2009 6:41:32 PM
> Subject: RE: Request assistance with distributed search multi shard/core setup and configuration
>
> Thanks, I had already read through this url. I guess my request was is there a
> way to setup something that is already part of solr itself to pass the
> URL[shard...] then having create a custom handler.
>
> thanks
>
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
> Sent: Tuesday, November 10, 2009 6:09 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Request assistance with distributed search multi shard/core setup
> and configuration
>
> Right, that's http://wiki.apache.org/solr/DistributedSearch
>
> Otis
> --
> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
>
>
>
> ----- Original Message ----
> > From: "Turner, Robbin J"
> > To: "solr-user@lucene.apache.org"
> > Sent: Tue, November 10, 2009 6:05:19 PM
> > Subject: RE: Request assistance with distributed search multi
> > shard/core setup and configuration
> >
> > I've already done the single Solr, that's why my request. I read on
> > some site that there is a way to setup the configuration so I can send
> > a query to one solr instance and have it pass it on or distribute it across
> all the instances?
> >
> > Btw, thanks for the quick reply.
> > RJ
> >
> > -----Original Message-----
> > From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
> > Sent: Tuesday, November 10, 2009 6:02 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: Request assistance with distributed search multi
> > shard/core setup and configuration
> >
> > RJ,
> >
> > You may want to take a simpler step - single Solr core (no solr.xml
> > needed) per machine. Then distributed search really only requires
> > that you specify shard URLs in the URL of the search requests. In
> > practice/production you rarely benefit from distributed search against
> > multiple cores on the same server anyway.
> >
> > Otis
> > --
> > Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> > Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
> >
> >
> >
> >
> > ________________________________
> > From: "Turner, Robbin J"
> > To: "solr-user@lucene.apache.org"
> > Sent: Tue, November 10, 2009 5:58:52 PM
> > Subject: Request assistance with distributed search multi shard/core
> > setup and configuration
> >
> > I've been looking through all the documentation. I've set up a single
> > solr instance, and one multicore instance. If someone would be
> > willing to share some configuration examples and/or advise for setting
> > up solr for distributing the search, I would really appreciate it.
> > I've read that there is a way to do it, but most of the current
> > documentation doesn't provide enough example on what to do with
> > solr.xml, and the solrconfig.xml. Also, I'm using tomcat 6 for the servlet
> container. I deployed the solr 1.4.0 released yesterday.
> >
> > Thanks
> > RJ
Re: Solr 1.4 distributed search configuration
Posted by Joe Calderon <ca...@gmail.com>.
you can set a default shard parameter on the request handler doing
distributed search, you can set up two different request handlers one
with shards default and one without
On Thu, Feb 25, 2010 at 1:35 PM, Jeffrey Zhao
<Je...@metalogic-inc.com> wrote:
> Now I got it, just forgot put qt=search in query.
>
> By the way, in solr 1.3, I used shards.txt under conf directory and
> "distributed=true" in query for distributed search. In that way,in my
> java application, I can hard code solr query with "distributed=true" and
> control the using of distributed search by define shards.txt or not.
>
> In solr 1.4, it is more difficult to use distributed search dynamically.Is
> there a way I just change configuration without changing query to make DS
> work?
>
> Thanks,
>
>
>
> From: Mark Miller <ma...@gmail.com>
> To: solr-user@lucene.apache.org
> Date: 25/02/2010 04:13 PM
> Subject: Re: Solr 1.4 distributed search configuration
>
>
>
> Can you elaborate on "doesn't work" when you put it in the /search
> handler?
>
> You get an error in the logs? Nothing happens?
>
> On 02/25/2010 03:47 PM, Jeffrey Zhao wrote:
>> Hi Mark,
>>
>> Thanks for your reply. I did make a new handler as following, but it
> does
>> not work, anything wrong with my configuration?
>>
>> Thanks,
>>
>> <requestHandler name="search" class="solr.SearchHandler">
>> <!-- default values for query parameters -->
>> <lst name="defaults">
>> <str>
>> name="shards">202.161.196.189:8080/solr,localhost:8080/solr</str>
>> </lst>
>> <arr name="components">
>> <str>query</str>
>> <str>facet</str>
>> <str>spellcheck</str>
>> <str>debug</str>
>> </arr>
>> </requestHandler>
>>
>>
>>
>> From: Mark Miller<ma...@gmail.com>
>> To: solr-user@lucene.apache.org
>> Date: 25/02/2010 03:41 PM
>> Subject: Re: Solr 1.4 distributed search configuration
>>
>>
>>
>> On 02/25/2010 03:32 PM, Jeffrey Zhao wrote:
>>
>>> How do define a new search handler with a shards parameter? I defined
>>>
>> as
>>
>>> following way but it doesn't work. If I put the shards parameter in
>>> default handler, it seems I got an infinite loop.
>>>
>>>
>>> <requestHandler name="standard" class="solr.SearchHandler"
>>>
>> default="true">
>>
>>> <!-- default values for query parameters -->
>>> <lst name="defaults">
>>> <str name="echoParams">explicit</str>
>>> </lst>
>>> </requestHandler>
>>>
>>> <requestHandler name="search" class="solr.SearchHandler">
>>> <!-- default values for query parameters -->
>>> <lst name="defaults">
>>> <str
>>> name="shards">202.161.196.189:8080/solr,localhost:8080/solr</str>
>>> </lst>
>>> <arr name="components">
>>> <str>query</str>
>>> <str>facet</str>
>>> <str>spellcheck</str>
>>> <str>debug</str>
>>> </arr>
>>> </requestHandler>
>>>
>>>
>>> Thanks,
>>>
>>>
>> Not seeing this on the wiki (it should be there), but you can't put the
>> shards param on the default search handler without causing an infinite
>> loop - you have to make a new request handler and put it on that.
>>
>>
>
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
>
>
Re: Solr 1.4 distributed search configuration
Posted by Jeffrey Zhao <Je...@metaLogic-inc.com>.
Now I got it, just forgot put qt=search in query.
By the way, in solr 1.3, I used shards.txt under conf directory and
"distributed=true" in query for distributed search. In that way,in my
java application, I can hard code solr query with "distributed=true" and
control the using of distributed search by define shards.txt or not.
In solr 1.4, it is more difficult to use distributed search dynamically.Is
there a way I just change configuration without changing query to make DS
work?
Thanks,
From: Mark Miller <ma...@gmail.com>
To: solr-user@lucene.apache.org
Date: 25/02/2010 04:13 PM
Subject: Re: Solr 1.4 distributed search configuration
Can you elaborate on "doesn't work" when you put it in the /search
handler?
You get an error in the logs? Nothing happens?
On 02/25/2010 03:47 PM, Jeffrey Zhao wrote:
> Hi Mark,
>
> Thanks for your reply. I did make a new handler as following, but it
does
> not work, anything wrong with my configuration?
>
> Thanks,
>
> <requestHandler name="search" class="solr.SearchHandler">
> <!-- default values for query parameters -->
> <lst name="defaults">
> <str>
> name="shards">202.161.196.189:8080/solr,localhost:8080/solr</str>
> </lst>
> <arr name="components">
> <str>query</str>
> <str>facet</str>
> <str>spellcheck</str>
> <str>debug</str>
> </arr>
> </requestHandler>
>
>
>
> From: Mark Miller<ma...@gmail.com>
> To: solr-user@lucene.apache.org
> Date: 25/02/2010 03:41 PM
> Subject: Re: Solr 1.4 distributed search configuration
>
>
>
> On 02/25/2010 03:32 PM, Jeffrey Zhao wrote:
>
>> How do define a new search handler with a shards parameter? I defined
>>
> as
>
>> following way but it doesn't work. If I put the shards parameter in
>> default handler, it seems I got an infinite loop.
>>
>>
>> <requestHandler name="standard" class="solr.SearchHandler"
>>
> default="true">
>
>> <!-- default values for query parameters -->
>> <lst name="defaults">
>> <str name="echoParams">explicit</str>
>> </lst>
>> </requestHandler>
>>
>> <requestHandler name="search" class="solr.SearchHandler">
>> <!-- default values for query parameters -->
>> <lst name="defaults">
>> <str
>> name="shards">202.161.196.189:8080/solr,localhost:8080/solr</str>
>> </lst>
>> <arr name="components">
>> <str>query</str>
>> <str>facet</str>
>> <str>spellcheck</str>
>> <str>debug</str>
>> </arr>
>> </requestHandler>
>>
>>
>> Thanks,
>>
>>
> Not seeing this on the wiki (it should be there), but you can't put the
> shards param on the default search handler without causing an infinite
> loop - you have to make a new request handler and put it on that.
>
>
--
- Mark
http://www.lucidimagination.com
Re: Solr 1.4 distributed search configuration
Posted by Mark Miller <ma...@gmail.com>.
Can you elaborate on "doesn't work" when you put it in the /search handler?
You get an error in the logs? Nothing happens?
On 02/25/2010 03:47 PM, Jeffrey Zhao wrote:
> Hi Mark,
>
> Thanks for your reply. I did make a new handler as following, but it does
> not work, anything wrong with my configuration?
>
> Thanks,
>
> <requestHandler name="search" class="solr.SearchHandler">
> <!-- default values for query parameters -->
> <lst name="defaults">
> <str>
> name="shards">202.161.196.189:8080/solr,localhost:8080/solr</str>
> </lst>
> <arr name="components">
> <str>query</str>
> <str>facet</str>
> <str>spellcheck</str>
> <str>debug</str>
> </arr>
> </requestHandler>
>
>
>
> From: Mark Miller<ma...@gmail.com>
> To: solr-user@lucene.apache.org
> Date: 25/02/2010 03:41 PM
> Subject: Re: Solr 1.4 distributed search configuration
>
>
>
> On 02/25/2010 03:32 PM, Jeffrey Zhao wrote:
>
>> How do define a new search handler with a shards parameter? I defined
>>
> as
>
>> following way but it doesn't work. If I put the shards parameter in
>> default handler, it seems I got an infinite loop.
>>
>>
>> <requestHandler name="standard" class="solr.SearchHandler"
>>
> default="true">
>
>> <!-- default values for query parameters -->
>> <lst name="defaults">
>> <str name="echoParams">explicit</str>
>> </lst>
>> </requestHandler>
>>
>> <requestHandler name="search" class="solr.SearchHandler">
>> <!-- default values for query parameters -->
>> <lst name="defaults">
>> <str
>> name="shards">202.161.196.189:8080/solr,localhost:8080/solr</str>
>> </lst>
>> <arr name="components">
>> <str>query</str>
>> <str>facet</str>
>> <str>spellcheck</str>
>> <str>debug</str>
>> </arr>
>> </requestHandler>
>>
>>
>> Thanks,
>>
>>
> Not seeing this on the wiki (it should be there), but you can't put the
> shards param on the default search handler without causing an infinite
> loop - you have to make a new request handler and put it on that.
>
>
--
- Mark
http://www.lucidimagination.com
Re: Solr 1.4 distributed search configuration
Posted by Jeffrey Zhao <Je...@metaLogic-inc.com>.
Hi Mark,
Thanks for your reply. I did make a new handler as following, but it does
not work, anything wrong with my configuration?
Thanks,
<requestHandler name="search" class="solr.SearchHandler">
<!-- default values for query parameters -->
<lst name="defaults">
<str>
name="shards">202.161.196.189:8080/solr,localhost:8080/solr</str>
</lst>
<arr name="components">
<str>query</str>
<str>facet</str>
<str>spellcheck</str>
<str>debug</str>
</arr>
</requestHandler>
From: Mark Miller <ma...@gmail.com>
To: solr-user@lucene.apache.org
Date: 25/02/2010 03:41 PM
Subject: Re: Solr 1.4 distributed search configuration
On 02/25/2010 03:32 PM, Jeffrey Zhao wrote:
> How do define a new search handler with a shards parameter? I defined
as
> following way but it doesn't work. If I put the shards parameter in
> default handler, it seems I got an infinite loop.
>
>
> <requestHandler name="standard" class="solr.SearchHandler"
default="true">
> <!-- default values for query parameters -->
> <lst name="defaults">
> <str name="echoParams">explicit</str>
> </lst>
> </requestHandler>
>
> <requestHandler name="search" class="solr.SearchHandler">
> <!-- default values for query parameters -->
> <lst name="defaults">
> <str
> name="shards">202.161.196.189:8080/solr,localhost:8080/solr</str>
> </lst>
> <arr name="components">
> <str>query</str>
> <str>facet</str>
> <str>spellcheck</str>
> <str>debug</str>
> </arr>
> </requestHandler>
>
>
> Thanks,
>
Not seeing this on the wiki (it should be there), but you can't put the
shards param on the default search handler without causing an infinite
loop - you have to make a new request handler and put it on that.
--
- Mark
http://www.lucidimagination.com
Re: Solr 1.4 distributed search configuration
Posted by Mark Miller <ma...@gmail.com>.
On 02/25/2010 03:32 PM, Jeffrey Zhao wrote:
> How do define a new search handler with a shards parameter? I defined as
> following way but it doesn't work. If I put the shards parameter in
> default handler, it seems I got an infinite loop.
>
>
> <requestHandler name="standard" class="solr.SearchHandler" default="true">
> <!-- default values for query parameters -->
> <lst name="defaults">
> <str name="echoParams">explicit</str>
> </lst>
> </requestHandler>
>
> <requestHandler name="search" class="solr.SearchHandler">
> <!-- default values for query parameters -->
> <lst name="defaults">
> <str
> name="shards">202.161.196.189:8080/solr,localhost:8080/solr</str>
> </lst>
> <arr name="components">
> <str>query</str>
> <str>facet</str>
> <str>spellcheck</str>
> <str>debug</str>
> </arr>
> </requestHandler>
>
>
> Thanks,
>
Not seeing this on the wiki (it should be there), but you can't put the
shards param on the default search handler without causing an infinite
loop - you have to make a new request handler and put it on that.
--
- Mark
http://www.lucidimagination.com
Solr 1.4 distributed search configuration
Posted by Jeffrey Zhao <Je...@metaLogic-inc.com>.
How do define a new search handler with a shards parameter? I defined as
following way but it doesn't work. If I put the shards parameter in
default handler, it seems I got an infinite loop.
<requestHandler name="standard" class="solr.SearchHandler" default="true">
<!-- default values for query parameters -->
<lst name="defaults">
<str name="echoParams">explicit</str>
</lst>
</requestHandler>
<requestHandler name="search" class="solr.SearchHandler">
<!-- default values for query parameters -->
<lst name="defaults">
<str
name="shards">202.161.196.189:8080/solr,localhost:8080/solr</str>
</lst>
<arr name="components">
<str>query</str>
<str>facet</str>
<str>spellcheck</str>
<str>debug</str>
</arr>
</requestHandler>
Thanks,
Re: Request assistance with distributed search multi shard/core setup
and configuration
Posted by Lance Norskog <go...@gmail.com>.
DS requires a bunch of shard names in the url. That's all. Note that a
ds does not use the data of the solr you call.
You can create an entry point for your distributed search by adding a
new <requestHandler> element in solrconfig.xml. You would add the
shard list parameter to the "defaults" list. Do not have it call the
same requesthandler path- you'll get an infinite loop.
On Tue, Nov 10, 2009 at 6:44 PM, Otis Gospodnetic
<ot...@yahoo.com> wrote:
> Hm, I don't follow. You don't need to create a custom (request) handler to make use of Solr's distributed search.
>
> Otis
> --
> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
>
>
>
> ----- Original Message ----
>> From: "Turner, Robbin J" <Ro...@boeing.com>
>> To: "solr-user@lucene.apache.org" <so...@lucene.apache.org>
>> Sent: Tue, November 10, 2009 6:41:32 PM
>> Subject: RE: Request assistance with distributed search multi shard/core setup and configuration
>>
>> Thanks, I had already read through this url. I guess my request was is there a
>> way to setup something that is already part of solr itself to pass the
>> URL[shard...] then having create a custom handler.
>>
>> thanks
>>
>> -----Original Message-----
>> From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
>> Sent: Tuesday, November 10, 2009 6:09 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Request assistance with distributed search multi shard/core setup
>> and configuration
>>
>> Right, that's http://wiki.apache.org/solr/DistributedSearch
>>
>> Otis
>> --
>> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
>> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
>>
>>
>>
>> ----- Original Message ----
>> > From: "Turner, Robbin J"
>> > To: "solr-user@lucene.apache.org"
>> > Sent: Tue, November 10, 2009 6:05:19 PM
>> > Subject: RE: Request assistance with distributed search multi
>> > shard/core setup and configuration
>> >
>> > I've already done the single Solr, that's why my request. I read on
>> > some site that there is a way to setup the configuration so I can send
>> > a query to one solr instance and have it pass it on or distribute it across
>> all the instances?
>> >
>> > Btw, thanks for the quick reply.
>> > RJ
>> >
>> > -----Original Message-----
>> > From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
>> > Sent: Tuesday, November 10, 2009 6:02 PM
>> > To: solr-user@lucene.apache.org
>> > Subject: Re: Request assistance with distributed search multi
>> > shard/core setup and configuration
>> >
>> > RJ,
>> >
>> > You may want to take a simpler step - single Solr core (no solr.xml
>> > needed) per machine. Then distributed search really only requires
>> > that you specify shard URLs in the URL of the search requests. In
>> > practice/production you rarely benefit from distributed search against
>> > multiple cores on the same server anyway.
>> >
>> > Otis
>> > --
>> > Sematext is hiring -- http://sematext.com/about/jobs.html?mls
>> > Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
>> >
>> >
>> >
>> >
>> > ________________________________
>> > From: "Turner, Robbin J"
>> > To: "solr-user@lucene.apache.org"
>> > Sent: Tue, November 10, 2009 5:58:52 PM
>> > Subject: Request assistance with distributed search multi shard/core
>> > setup and configuration
>> >
>> > I've been looking through all the documentation. I've set up a single
>> > solr instance, and one multicore instance. If someone would be
>> > willing to share some configuration examples and/or advise for setting
>> > up solr for distributing the search, I would really appreciate it.
>> > I've read that there is a way to do it, but most of the current
>> > documentation doesn't provide enough example on what to do with
>> > solr.xml, and the solrconfig.xml. Also, I'm using tomcat 6 for the servlet
>> container. I deployed the solr 1.4.0 released yesterday.
>> >
>> > Thanks
>> > RJ
>
>
--
Lance Norskog
goksron@gmail.com