You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Zac Smith <za...@trinkit.com> on 2011/05/13 18:28:35 UTC
Schema Design Question
Let's say I have a data model that involves books and bookshelves. I have tens of thousands of books and thousands of bookshelves. There is a many-many relationship between books & bookshelves. All of the books are indexed by SOLR.
I need to be able to query SOLR and get all the books for a given bookshelf. I see two schema design options here:
1) Each book has a multi-value field that contains a list of all the bookshelf ID's. Many books will have thousands of bookshelf ID's. In this case the query is simple, I just send solr the bookshelf ID.
2) I send solr a query with each book on the bookshelf e.g. q=book_id:(1+OR+2+OR+3 ....). Many bookshelves will have thousands of book ID's so the query can get rather large.
Right now I am using option 2 and it seems to be working fine. I have had to crank 'maxBooleanClauses' right up but it does seem to be pretty fast.
Anyone have an opinion?
RE: Schema Design Question
Posted by Zac Smith <za...@trinkit.com>.
Ok thanks for the responses. My option #2 will be easier to implement than having the new doc with combinations so will give it a try. But that has opened my eyes to different possibilities!
-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com]
Sent: Sunday, May 15, 2011 8:55 AM
To: solr-user@lucene.apache.org
Subject: Re: Schema Design Question
Of your first two options, I'd go with a multi-valued field for each book (1).
But kenf_nc's suggestion is a good one too.
On Sun, May 15, 2011 at 3:54 AM, kenf_nc <ke...@realestate.com> wrote:
> create a separate document for each book-bookshelf combination.
> doc 1 = book 1,shelf 1
> doc 2 = book 1,shelf 3
> doc 3 = book 2,shelf 1
> etc.
>
> then your queries are q=book_id to get all bookshelfs a given book
> is on or q=shelf_id to get all books on a given bookshelf.
>
> Biggest problem people face with Solr schema design is thinking either
> object orientedly or RDBMs orientedly. You need to think differently.
> Solr/Lucene find text and they find it very fast over huge amounts of data.
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Schema-Design-Question-tp2939045p29
> 42809.html Sent from the Solr - User mailing list archive at
> Nabble.com.
>
Re: Schema Design Question
Posted by Erick Erickson <er...@gmail.com>.
Of your first two options, I'd go with a multi-valued field for each book (1).
But kenf_nc's suggestion is a good one too.
On Sun, May 15, 2011 at 3:54 AM, kenf_nc <ke...@realestate.com> wrote:
> create a separate document for each book-bookshelf combination.
> doc 1 = book 1,shelf 1
> doc 2 = book 1,shelf 3
> doc 3 = book 2,shelf 1
> etc.
>
> then your queries are q=book_id to get all bookshelfs a given book is on
> or q=shelf_id to get all books on a given bookshelf.
>
> Biggest problem people face with Solr schema design is thinking either
> object orientedly or RDBMs orientedly. You need to think differently.
> Solr/Lucene find text and they find it very fast over huge amounts of data.
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Schema-Design-Question-tp2939045p2942809.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
RE: Schema Design Question
Posted by kenf_nc <ke...@realestate.com>.
create a separate document for each book-bookshelf combination.
doc 1 = book 1,shelf 1
doc 2 = book 1,shelf 3
doc 3 = book 2,shelf 1
etc.
then your queries are q=book_id to get all bookshelfs a given book is on
or q=shelf_id to get all books on a given bookshelf.
Biggest problem people face with Solr schema design is thinking either
object orientedly or RDBMs orientedly. You need to think differently.
Solr/Lucene find text and they find it very fast over huge amounts of data.
--
View this message in context: http://lucene.472066.n3.nabble.com/Schema-Design-Question-tp2939045p2942809.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: Schema Design Question
Posted by Zac Smith <za...@trinkit.com>.
Thanks that looks interesting. Don't think it helps my situation though as I would have to index all the bookshelves and will still end up having to put thousands of Book ID values in a multi-value field.
I guess the question I have is: Is it more appropriate to load a multi-value field with a large number of values or should you pass a large number of values in as a Boolean clause?
Zac
-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
Sent: Friday, May 13, 2011 10:37 AM
To: solr-user@lucene.apache.org
Subject: Re: Schema Design Question
Hi Zac,
Solr 4.0 (trunk) has support for relationships/JOIN. Have a look:
http://search-lucene.com/?q=solr+join
Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/
----- Original Message ----
> From: Zac Smith <za...@trinkit.com>
> To: "solr-user@lucene.apache.org" <so...@lucene.apache.org>
> Sent: Fri, May 13, 2011 12:28:35 PM
> Subject: Schema Design Question
>
> Let's say I have a data model that involves books and bookshelves. I
>have tens of thousands of books and thousands of bookshelves. There is
>a many-many relationship between books & bookshelves. All of the books are indexed by SOLR.
>
> I need to be able to query SOLR and get all the books for a given
>bookshelf. I see two schema design options here:
>
>
> 1) Each book has a multi-value field that contains a list of all the
>bookshelf ID's. Many books will have thousands of bookshelf ID's. In
>this case the query is simple, I just send solr the bookshelf ID.
>
> 2) I send solr a query with each book on the bookshelf e.g.
>q=book_id:(1+OR+2+OR+3 ....). Many bookshelves will have thousands of
>book ID's so the query can get rather large.
>
> Right now I am using option 2 and it seems to be working fine. I have
>had to crank 'maxBooleanClauses' right up but it does seem to be pretty fast.
>
> Anyone have an opinion?
>
>
Re: Schema Design Question
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Hi Zac,
Solr 4.0 (trunk) has support for relationships/JOIN. Have a look:
http://search-lucene.com/?q=solr+join
Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/
----- Original Message ----
> From: Zac Smith <za...@trinkit.com>
> To: "solr-user@lucene.apache.org" <so...@lucene.apache.org>
> Sent: Fri, May 13, 2011 12:28:35 PM
> Subject: Schema Design Question
>
> Let's say I have a data model that involves books and bookshelves. I have tens
>of thousands of books and thousands of bookshelves. There is a many-many
>relationship between books & bookshelves. All of the books are indexed by SOLR.
>
> I need to be able to query SOLR and get all the books for a given bookshelf. I
>see two schema design options here:
>
>
> 1) Each book has a multi-value field that contains a list of all the
>bookshelf ID's. Many books will have thousands of bookshelf ID's. In this case
>the query is simple, I just send solr the bookshelf ID.
>
> 2) I send solr a query with each book on the bookshelf e.g.
>q=book_id:(1+OR+2+OR+3 ....). Many bookshelves will have thousands of book ID's
>so the query can get rather large.
>
> Right now I am using option 2 and it seems to be working fine. I have had to
>crank 'maxBooleanClauses' right up but it does seem to be pretty fast.
>
> Anyone have an opinion?
>
>