You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Matthew Runo <mr...@zappos.com> on 2007/04/19 18:46:55 UTC
Multiple indexes?
Hey there-
I was wondering if the following was possible, and, if so, how to set
it up...
I want to index two different types of data, and have them searchable
from the same interface.
For example, a group of products, with size, color, price, etc info.
And a group of brands, with brand, genre, brand description, etc info
So, the info does overlap some. But a lot of the fields for each
"type" don't matter to the other. Is there a way to set up two
different schema so that both types may be indexed with relative ease?
+--------------------------------------------------------+
| Matthew Runo
| Zappos Development
| mruno@zappos.com
| 702-943-7833
+--------------------------------------------------------+
Re: Multiple indexes?
Posted by Henrib <hb...@gmail.com>.
You can not have more than one Solr core per application (to be precise, per
class-loader since there are a few statics).
One way is thus to have 2 webapps - when & if indexes do not have the same
lifetime/radically different schema/etc.
However, the common wisdom is that you usually dont really need different
indexes (I discussed about this last week).
If you really are in desperate need of multiple cores, in the 'Multiple Solr
Cores' thread, you'll find (early state) patches that allow just that...
Cheers
Henri
Matthew Runo wrote:
>
> Hey there-
>
> I was wondering if the following was possible, and, if so, how to set
> it up...
>
> I want to index two different types of data, and have them searchable
> from the same interface.
>
> For example, a group of products, with size, color, price, etc info.
> And a group of brands, with brand, genre, brand description, etc info
>
> So, the info does overlap some. But a lot of the fields for each
> "type" don't matter to the other. Is there a way to set up two
> different schema so that both types may be indexed with relative ease?
>
> +--------------------------------------------------------+
> | Matthew Runo
> | Zappos Development
> | mruno@zappos.com
> | 702-943-7833
> +--------------------------------------------------------+
>
>
>
>
--
View this message in context: http://www.nabble.com/Multiple-indexes--tf3608429.html#a10083580
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Multiple indexes?
Posted by Matthew Runo <mr...@zappos.com>.
Ah. That makes sense then. I wasn't sure if that was the best way to
go about things or not. I didn't want to end up with a bunch of
fields that were not being used all the time if it would cause a
degradation in search quality.
+--------------------------------------------------------+
| Matthew Runo
| Zappos Development
| mruno@zappos.com
| 702-943-7833
+--------------------------------------------------------+
On Apr 19, 2007, at 12:59 PM, Erik Hatcher wrote:
> Matthew,
>
> All that is meant by "object_types" is an additional stored/indexed
> field in the Solr schema that gets added to every document
> providing context of which type it is (shoes or brands). Then you
> can limit searches to a particular area by just filtering on
> type:shoes, for example.
>
> Erik
>
> p.s. I could use some new shoes!
>
> On Apr 19, 2007, at 3:17 PM, Matthew Runo wrote:
>
>> I'll actually be doing this in Perl..
>>
>> any ideas on perl? heh
>>
>> +--------------------------------------------------------+
>> | Matthew Runo
>> | Zappos Development
>> | mruno@zappos.com
>> | 702-943-7833
>> +--------------------------------------------------------+
>>
>>
>> On Apr 19, 2007, at 11:59 AM, Cody Caughlan wrote:
>>
>>> If you're doing this in Ruby, there is an "acts_as_solr" plugin for
>>> Rails which takes exactly this approach to store all different kinds
>>> of Model objects in the same index...I just "took" the idea from
>>> there...
>>>
>>> /Cody
>>>
>>> On 4/19/07, Matthew Runo <mr...@zappos.com> wrote:
>>>> Ah hah! This appears to be what I'm interested in doing.
>>>>
>>>> I'll have to read up on object_types.
>>>>
>>>> +--------------------------------------------------------+
>>>> | Matthew Runo
>>>> | Zappos Development
>>>> | mruno@zappos.com
>>>> | 702-943-7833
>>>> +--------------------------------------------------------+
>>>>
>>>>
>>>> On Apr 19, 2007, at 10:04 AM, Cody Caughlan wrote:
>>>>
>>>> > Why not just store an additional "object_type" field which
>>>> > differentiates between the actual type of data you are looking
>>>> for?
>>>> >
>>>> > So if you're looking for some shoes:
>>>> >
>>>> > (size:8 AND color:'blue') AND object_type:'shoe'
>>>> >
>>>> > Or if you're searching on brands
>>>> >
>>>> > (genre:'skater' AND brand_desc:'skater boy') AND
>>>> object_type:'brand'
>>>> >
>>>> > I apologize if I misunderstood your question.
>>>> >
>>>> > /cody
>>>> >
>>>> > On 4/19/07, Matthew Runo <mr...@zappos.com> wrote:
>>>> >> Hey there-
>>>> >>
>>>> >> I was wondering if the following was possible, and, if so,
>>>> how to set
>>>> >> it up...
>>>> >>
>>>> >> I want to index two different types of data, and have them
>>>> searchable
>>>> >> from the same interface.
>>>> >>
>>>> >> For example, a group of products, with size, color, price,
>>>> etc info.
>>>> >> And a group of brands, with brand, genre, brand description,
>>>> etc info
>>>> >>
>>>> >> So, the info does overlap some. But a lot of the fields for each
>>>> >> "type" don't matter to the other. Is there a way to set up two
>>>> >> different schema so that both types may be indexed with relative
>>>> >> ease?
>>>> >>
>>>> >> +--------------------------------------------------------+
>>>> >> | Matthew Runo
>>>> >> | Zappos Development
>>>> >> | mruno@zappos.com
>>>> >> | 702-943-7833
>>>> >> +--------------------------------------------------------+
>>>> >>
>>>> >>
>>>> >>
>>>> >
>>>>
>>>>
>>>
>
Re: Multiple indexes?
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
Matthew,
All that is meant by "object_types" is an additional stored/indexed
field in the Solr schema that gets added to every document providing
context of which type it is (shoes or brands). Then you can limit
searches to a particular area by just filtering on type:shoes, for
example.
Erik
p.s. I could use some new shoes!
On Apr 19, 2007, at 3:17 PM, Matthew Runo wrote:
> I'll actually be doing this in Perl..
>
> any ideas on perl? heh
>
> +--------------------------------------------------------+
> | Matthew Runo
> | Zappos Development
> | mruno@zappos.com
> | 702-943-7833
> +--------------------------------------------------------+
>
>
> On Apr 19, 2007, at 11:59 AM, Cody Caughlan wrote:
>
>> If you're doing this in Ruby, there is an "acts_as_solr" plugin for
>> Rails which takes exactly this approach to store all different kinds
>> of Model objects in the same index...I just "took" the idea from
>> there...
>>
>> /Cody
>>
>> On 4/19/07, Matthew Runo <mr...@zappos.com> wrote:
>>> Ah hah! This appears to be what I'm interested in doing.
>>>
>>> I'll have to read up on object_types.
>>>
>>> +--------------------------------------------------------+
>>> | Matthew Runo
>>> | Zappos Development
>>> | mruno@zappos.com
>>> | 702-943-7833
>>> +--------------------------------------------------------+
>>>
>>>
>>> On Apr 19, 2007, at 10:04 AM, Cody Caughlan wrote:
>>>
>>> > Why not just store an additional "object_type" field which
>>> > differentiates between the actual type of data you are looking
>>> for?
>>> >
>>> > So if you're looking for some shoes:
>>> >
>>> > (size:8 AND color:'blue') AND object_type:'shoe'
>>> >
>>> > Or if you're searching on brands
>>> >
>>> > (genre:'skater' AND brand_desc:'skater boy') AND
>>> object_type:'brand'
>>> >
>>> > I apologize if I misunderstood your question.
>>> >
>>> > /cody
>>> >
>>> > On 4/19/07, Matthew Runo <mr...@zappos.com> wrote:
>>> >> Hey there-
>>> >>
>>> >> I was wondering if the following was possible, and, if so, how
>>> to set
>>> >> it up...
>>> >>
>>> >> I want to index two different types of data, and have them
>>> searchable
>>> >> from the same interface.
>>> >>
>>> >> For example, a group of products, with size, color, price, etc
>>> info.
>>> >> And a group of brands, with brand, genre, brand description,
>>> etc info
>>> >>
>>> >> So, the info does overlap some. But a lot of the fields for each
>>> >> "type" don't matter to the other. Is there a way to set up two
>>> >> different schema so that both types may be indexed with relative
>>> >> ease?
>>> >>
>>> >> +--------------------------------------------------------+
>>> >> | Matthew Runo
>>> >> | Zappos Development
>>> >> | mruno@zappos.com
>>> >> | 702-943-7833
>>> >> +--------------------------------------------------------+
>>> >>
>>> >>
>>> >>
>>> >
>>>
>>>
>>
Re: Multiple indexes?
Posted by Matthew Runo <mr...@zappos.com>.
I'll actually be doing this in Perl..
any ideas on perl? heh
+--------------------------------------------------------+
| Matthew Runo
| Zappos Development
| mruno@zappos.com
| 702-943-7833
+--------------------------------------------------------+
On Apr 19, 2007, at 11:59 AM, Cody Caughlan wrote:
> If you're doing this in Ruby, there is an "acts_as_solr" plugin for
> Rails which takes exactly this approach to store all different kinds
> of Model objects in the same index...I just "took" the idea from
> there...
>
> /Cody
>
> On 4/19/07, Matthew Runo <mr...@zappos.com> wrote:
>> Ah hah! This appears to be what I'm interested in doing.
>>
>> I'll have to read up on object_types.
>>
>> +--------------------------------------------------------+
>> | Matthew Runo
>> | Zappos Development
>> | mruno@zappos.com
>> | 702-943-7833
>> +--------------------------------------------------------+
>>
>>
>> On Apr 19, 2007, at 10:04 AM, Cody Caughlan wrote:
>>
>> > Why not just store an additional "object_type" field which
>> > differentiates between the actual type of data you are looking for?
>> >
>> > So if you're looking for some shoes:
>> >
>> > (size:8 AND color:'blue') AND object_type:'shoe'
>> >
>> > Or if you're searching on brands
>> >
>> > (genre:'skater' AND brand_desc:'skater boy') AND
>> object_type:'brand'
>> >
>> > I apologize if I misunderstood your question.
>> >
>> > /cody
>> >
>> > On 4/19/07, Matthew Runo <mr...@zappos.com> wrote:
>> >> Hey there-
>> >>
>> >> I was wondering if the following was possible, and, if so, how
>> to set
>> >> it up...
>> >>
>> >> I want to index two different types of data, and have them
>> searchable
>> >> from the same interface.
>> >>
>> >> For example, a group of products, with size, color, price, etc
>> info.
>> >> And a group of brands, with brand, genre, brand description,
>> etc info
>> >>
>> >> So, the info does overlap some. But a lot of the fields for each
>> >> "type" don't matter to the other. Is there a way to set up two
>> >> different schema so that both types may be indexed with relative
>> >> ease?
>> >>
>> >> +--------------------------------------------------------+
>> >> | Matthew Runo
>> >> | Zappos Development
>> >> | mruno@zappos.com
>> >> | 702-943-7833
>> >> +--------------------------------------------------------+
>> >>
>> >>
>> >>
>> >
>>
>>
>
Re: Multiple indexes?
Posted by Cody Caughlan <to...@gmail.com>.
If you're doing this in Ruby, there is an "acts_as_solr" plugin for
Rails which takes exactly this approach to store all different kinds
of Model objects in the same index...I just "took" the idea from
there...
/Cody
On 4/19/07, Matthew Runo <mr...@zappos.com> wrote:
> Ah hah! This appears to be what I'm interested in doing.
>
> I'll have to read up on object_types.
>
> +--------------------------------------------------------+
> | Matthew Runo
> | Zappos Development
> | mruno@zappos.com
> | 702-943-7833
> +--------------------------------------------------------+
>
>
> On Apr 19, 2007, at 10:04 AM, Cody Caughlan wrote:
>
> > Why not just store an additional "object_type" field which
> > differentiates between the actual type of data you are looking for?
> >
> > So if you're looking for some shoes:
> >
> > (size:8 AND color:'blue') AND object_type:'shoe'
> >
> > Or if you're searching on brands
> >
> > (genre:'skater' AND brand_desc:'skater boy') AND object_type:'brand'
> >
> > I apologize if I misunderstood your question.
> >
> > /cody
> >
> > On 4/19/07, Matthew Runo <mr...@zappos.com> wrote:
> >> Hey there-
> >>
> >> I was wondering if the following was possible, and, if so, how to set
> >> it up...
> >>
> >> I want to index two different types of data, and have them searchable
> >> from the same interface.
> >>
> >> For example, a group of products, with size, color, price, etc info.
> >> And a group of brands, with brand, genre, brand description, etc info
> >>
> >> So, the info does overlap some. But a lot of the fields for each
> >> "type" don't matter to the other. Is there a way to set up two
> >> different schema so that both types may be indexed with relative
> >> ease?
> >>
> >> +--------------------------------------------------------+
> >> | Matthew Runo
> >> | Zappos Development
> >> | mruno@zappos.com
> >> | 702-943-7833
> >> +--------------------------------------------------------+
> >>
> >>
> >>
> >
>
>
Re: Multiple indexes?
Posted by Matthew Runo <mr...@zappos.com>.
Ah hah! This appears to be what I'm interested in doing.
I'll have to read up on object_types.
+--------------------------------------------------------+
| Matthew Runo
| Zappos Development
| mruno@zappos.com
| 702-943-7833
+--------------------------------------------------------+
On Apr 19, 2007, at 10:04 AM, Cody Caughlan wrote:
> Why not just store an additional "object_type" field which
> differentiates between the actual type of data you are looking for?
>
> So if you're looking for some shoes:
>
> (size:8 AND color:'blue') AND object_type:'shoe'
>
> Or if you're searching on brands
>
> (genre:'skater' AND brand_desc:'skater boy') AND object_type:'brand'
>
> I apologize if I misunderstood your question.
>
> /cody
>
> On 4/19/07, Matthew Runo <mr...@zappos.com> wrote:
>> Hey there-
>>
>> I was wondering if the following was possible, and, if so, how to set
>> it up...
>>
>> I want to index two different types of data, and have them searchable
>> from the same interface.
>>
>> For example, a group of products, with size, color, price, etc info.
>> And a group of brands, with brand, genre, brand description, etc info
>>
>> So, the info does overlap some. But a lot of the fields for each
>> "type" don't matter to the other. Is there a way to set up two
>> different schema so that both types may be indexed with relative
>> ease?
>>
>> +--------------------------------------------------------+
>> | Matthew Runo
>> | Zappos Development
>> | mruno@zappos.com
>> | 702-943-7833
>> +--------------------------------------------------------+
>>
>>
>>
>
Re: Multiple indexes?
Posted by Ryan McKinley <ry...@gmail.com>.
As this question comes up so often, i put a new page on the wiki:
http://wiki.apache.org/solr/MultipleIndexes
We should fill in more details and link it to the front page.
Chris Hostetter wrote:
> : So if you're looking for some shoes:
> : (size:8 AND color:'blue') AND object_type:'shoe'
>
> : Or if you're searching on brands
> : (genre:'skater' AND brand_desc:'skater boy') AND object_type:'brand'
>
> a slight improvement on this: put your object_type restriction in a
> filter query (&fq=object_type:foo) and not in your main query ... that way
> it won't affect the scoring, and it will be cached uniquely so the work of
> identifying the set of all shows will only be done once per commit (and
> likewise for brands)
>
>
>
> -Hoss
>
>
Re: Multiple indexes?
Posted by Chris Hostetter <ho...@fucit.org>.
: So if you're looking for some shoes:
: (size:8 AND color:'blue') AND object_type:'shoe'
: Or if you're searching on brands
: (genre:'skater' AND brand_desc:'skater boy') AND object_type:'brand'
a slight improvement on this: put your object_type restriction in a
filter query (&fq=object_type:foo) and not in your main query ... that way
it won't affect the scoring, and it will be cached uniquely so the work of
identifying the set of all shows will only be done once per commit (and
likewise for brands)
-Hoss
Re: Multiple indexes?
Posted by Cody Caughlan <to...@gmail.com>.
Why not just store an additional "object_type" field which
differentiates between the actual type of data you are looking for?
So if you're looking for some shoes:
(size:8 AND color:'blue') AND object_type:'shoe'
Or if you're searching on brands
(genre:'skater' AND brand_desc:'skater boy') AND object_type:'brand'
I apologize if I misunderstood your question.
/cody
On 4/19/07, Matthew Runo <mr...@zappos.com> wrote:
> Hey there-
>
> I was wondering if the following was possible, and, if so, how to set
> it up...
>
> I want to index two different types of data, and have them searchable
> from the same interface.
>
> For example, a group of products, with size, color, price, etc info.
> And a group of brands, with brand, genre, brand description, etc info
>
> So, the info does overlap some. But a lot of the fields for each
> "type" don't matter to the other. Is there a way to set up two
> different schema so that both types may be indexed with relative ease?
>
> +--------------------------------------------------------+
> | Matthew Runo
> | Zappos Development
> | mruno@zappos.com
> | 702-943-7833
> +--------------------------------------------------------+
>
>
>