You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Paul Rosen <pa...@performantsoftware.com> on 2009/09/09 18:38:56 UTC
multicore and ruby
Hi all,
I'd like to start experimenting with multicore in a ruby on rails app.
Right now, the app is using the solr-ruby-rails-0.0.5 to communicate
with solr and it doesn't appear to have direct support for multicore and
I didn't have any luck googling around for it.
We aren't necessarily wedded to using solr-ruby-rails-0.0.5, but I
looked at rsolr very briefly and didn't see any reference to multicore
there, either.
I can certainly hack something together, but it seems like this is a
common problem.
How are others doing multicore from ruby?
Thanks,
Paul
Re: multicore and ruby
Posted by Paul Rosen <pa...@performantsoftware.com>.
Hi Matt,
> What kinds of things were you hoping to find when looking for multicore
> support in either solr-ruby or rsolr?
I have a couple of uses for it:
1) Search and merge the results from multiple indexes:
http://localhost:8983/solr/core0/select?shards=localhost:8983/solr/core0,localhost:8983/solr/core1&q=XXXX
I assume the above would return documents containing XXXX in both cores.
How are the relevancy scores managed? Would the documents be merged
together?
(The reason I want two indexes here but in every respect look to the end
user as one index is that one index is huge and changes rarely, and the
other is small and changes more often, so I'd like to make the commits
on that take a reasonable amount of time. Also, it makes managing the
large index better, because I don't have to worry about the small
index's changes.)
2) Do automated tests during reindexing:
After reindexing to core1, I'll query both core0 and core1 separately
and compare the results to be sure only what I intended to change was
changed.
We can even create an interface so that an authorized user can switch
indexes in their session to test the changes out in an otherwise
completely live environment.
3) There are a few more similar uses that might be coming, but I think
the main point is to be able to query on one or the other cores, or
both, or possibly a third one in the future.
Thanks,
Paul
Re: multicore and ruby
Posted by Matt Mitchell <go...@gmail.com>.
Hey Paul,
In rsolr, you could use the #request method to set a request handler path:
solr.request('/core1/select', :q=>'*:*')
Alternatively, (rsolr and solr-ruby) you could probably handle this by
creating a new instance of a connection object per-core, and then have some
kind of factory to return connection objects by a core-name?
What kinds of things were you hoping to find when looking for multicore
support in either solr-ruby or rsolr?
Matt
On Wed, Sep 9, 2009 at 12:38 PM, Paul Rosen <pa...@performantsoftware.com>wrote:
> Hi all,
>
> I'd like to start experimenting with multicore in a ruby on rails app.
>
> Right now, the app is using the solr-ruby-rails-0.0.5 to communicate with
> solr and it doesn't appear to have direct support for multicore and I didn't
> have any luck googling around for it.
>
> We aren't necessarily wedded to using solr-ruby-rails-0.0.5, but I looked
> at rsolr very briefly and didn't see any reference to multicore there,
> either.
>
> I can certainly hack something together, but it seems like this is a common
> problem.
>
> How are others doing multicore from ruby?
>
> Thanks,
> Paul
>
Re: multicore and ruby
Posted by Greg Gershman <gr...@yahoo.com>.
Paul
I've been working with rsolr in a Rails app. In terms of querying from multiple indices/cores within a multicore setup of Solr, I'm managing it all on the Rails side, aggregating results from mutliple cores. In terms of core administration, I've been doing that all by hand as well.
Greg
________________________________
From: Paul Rosen <pa...@performantsoftware.com>
To: solr-user@lucene.apache.org
Sent: Wednesday, September 9, 2009 12:38:56 PM
Subject: multicore and ruby
Hi all,
I'd like to start experimenting with multicore in a ruby on rails app.
Right now, the app is using the solr-ruby-rails-0.0.5 to communicate with solr and it doesn't appear to have direct support for multicore and I didn't have any luck googling around for it.
We aren't necessarily wedded to using solr-ruby-rails-0.0.5, but I looked at rsolr very briefly and didn't see any reference to multicore there, either.
I can certainly hack something together, but it seems like this is a common problem.
How are others doing multicore from ruby?
Thanks,
Paul
Re: multicore and ruby
Posted by Matt Mitchell <go...@gmail.com>.
Yep same thing in rsolr and just use the :shards param. It'll return
whatever solr returns.
Matt
On Wed, Sep 9, 2009 at 4:17 PM, Paul Rosen <pa...@performantsoftware.com>wrote:
> Hi Erik,
>
> Yes, I've been doing that in my tests, but I also have the case of wanting
> to do a search over all the cores using the shards syntax. I was thinking
> that the following wouldn't work:
>
>
> solr = Solr::Connection.new('
> http://localhost:8983/solr/core0/select?shards=localhost:8983/solr/core0,localhost:8983/solr/core1
> ')
>
> because it has a "?" in it.
>
>
> Erik Hatcher wrote:
>
>> With solr-ruby, simply put the core name in the URL of the
>> Solr::Connection...
>>
>> solr = Solr::Connection.new('http://localhost:8983/solr/core_name')
>>
>> Erik
>>
>>
>> On Sep 9, 2009, at 6:38 PM, Paul Rosen wrote:
>>
>> Hi all,
>>>
>>> I'd like to start experimenting with multicore in a ruby on rails app.
>>>
>>> Right now, the app is using the solr-ruby-rails-0.0.5 to communicate with
>>> solr and it doesn't appear to have direct support for multicore and I didn't
>>> have any luck googling around for it.
>>>
>>> We aren't necessarily wedded to using solr-ruby-rails-0.0.5, but I looked
>>> at rsolr very briefly and didn't see any reference to multicore there,
>>> either.
>>>
>>> I can certainly hack something together, but it seems like this is a
>>> common problem.
>>>
>>> How are others doing multicore from ruby?
>>>
>>> Thanks,
>>> Paul
>>>
>>
>>
>
Re: multicore and ruby
Posted by Erik Hatcher <er...@gmail.com>.
The Connection is not for parameters, merely the base URL to the Solr
server (or core, which is effectively a Solr "server").
As of solr-ruby 0.0.6, the shards parameter is supported for the
Solr::Request::Standard and Dismax request objects, so you'd simply
specify :shards=>"...." for those queries.
Also note that you can specify the shards in solrconfig.xml for the
request handler mapping(s) and avoid having to send it from the client
(depends on your needs whether that makes sense or not).
Erik
On Sep 9, 2009, at 10:17 PM, Paul Rosen wrote:
> Hi Erik,
>
> Yes, I've been doing that in my tests, but I also have the case of
> wanting to do a search over all the cores using the shards syntax. I
> was thinking that the following wouldn't work:
>
>
> solr = Solr::Connection.new('http://localhost:8983/solr/core0/select?shards=localhost:8983/solr/core0,localhost:8983/solr/core1')
>
> because it has a "?" in it.
>
> Erik Hatcher wrote:
>> With solr-ruby, simply put the core name in the URL of the
>> Solr::Connection...
>> solr = Solr::Connection.new('http://localhost:8983/solr/core_name')
>> Erik
>> On Sep 9, 2009, at 6:38 PM, Paul Rosen wrote:
>>> Hi all,
>>>
>>> I'd like to start experimenting with multicore in a ruby on rails
>>> app.
>>>
>>> Right now, the app is using the solr-ruby-rails-0.0.5 to
>>> communicate with solr and it doesn't appear to have direct support
>>> for multicore and I didn't have any luck googling around for it.
>>>
>>> We aren't necessarily wedded to using solr-ruby-rails-0.0.5, but I
>>> looked at rsolr very briefly and didn't see any reference to
>>> multicore there, either.
>>>
>>> I can certainly hack something together, but it seems like this is
>>> a common problem.
>>>
>>> How are others doing multicore from ruby?
>>>
>>> Thanks,
>>> Paul
>
Re: multicore and ruby
Posted by Paul Rosen <pa...@performantsoftware.com>.
Hi Erik,
Yes, I've been doing that in my tests, but I also have the case of
wanting to do a search over all the cores using the shards syntax. I was
thinking that the following wouldn't work:
solr =
Solr::Connection.new('http://localhost:8983/solr/core0/select?shards=localhost:8983/solr/core0,localhost:8983/solr/core1')
because it has a "?" in it.
Erik Hatcher wrote:
> With solr-ruby, simply put the core name in the URL of the
> Solr::Connection...
>
> solr = Solr::Connection.new('http://localhost:8983/solr/core_name')
>
> Erik
>
>
> On Sep 9, 2009, at 6:38 PM, Paul Rosen wrote:
>
>> Hi all,
>>
>> I'd like to start experimenting with multicore in a ruby on rails app.
>>
>> Right now, the app is using the solr-ruby-rails-0.0.5 to communicate
>> with solr and it doesn't appear to have direct support for multicore
>> and I didn't have any luck googling around for it.
>>
>> We aren't necessarily wedded to using solr-ruby-rails-0.0.5, but I
>> looked at rsolr very briefly and didn't see any reference to multicore
>> there, either.
>>
>> I can certainly hack something together, but it seems like this is a
>> common problem.
>>
>> How are others doing multicore from ruby?
>>
>> Thanks,
>> Paul
>
Re: multicore and ruby
Posted by Erik Hatcher <er...@gmail.com>.
With solr-ruby, simply put the core name in the URL of the
Solr::Connection...
solr = Solr::Connection.new('http://localhost:8983/solr/core_name')
Erik
On Sep 9, 2009, at 6:38 PM, Paul Rosen wrote:
> Hi all,
>
> I'd like to start experimenting with multicore in a ruby on rails app.
>
> Right now, the app is using the solr-ruby-rails-0.0.5 to communicate
> with solr and it doesn't appear to have direct support for multicore
> and I didn't have any luck googling around for it.
>
> We aren't necessarily wedded to using solr-ruby-rails-0.0.5, but I
> looked at rsolr very briefly and didn't see any reference to
> multicore there, either.
>
> I can certainly hack something together, but it seems like this is a
> common problem.
>
> How are others doing multicore from ruby?
>
> Thanks,
> Paul