You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Frédéric Malo <fr...@gmail.com> on 2009/08/16 10:29:58 UTC
How to code an advanced search
Hello,
I would like to create an "advanced search" for a couchDB database, and I
don't know how to do it without reduce. The documentation warns me not to
follow this strategy :
A common mistake new CouchDB users make, is attempting to construct complex
aggregate values with a reduce function. Full reductions should result in a
scalar value, like 5, not, for instance, a JSON hash with the set of unique
keys, and the count of each. The problem with this approach is that you’ll
end up with a very very large final value. The number of unique keys can be
nearly as large as the number of total keys, even for a large set. It is
fine to combine a few scalar calculations into one reduce function, for
instance to find the total, average, and standard deviation of a set of
numbers in a single function.
http://books.couchdb.org/relax/design-documents/views
For example, imagine a music database as iTunes. I would like to create a
script to query such as :
Sort all songs by genre then album :
- with 3 or 5 stars
- artist name matching "floyd"
- duration between 5 and 8 minutes
- heard last week (perhaps another couch database, logging user actions)
- ....
All the parameters are optional. Users can choose to filter with only 2 or 5
parameters. Other parameters should be added.
My first reflex is to think like sql with index and joins.
But how to code it with views and reduce ?
Thank you for all, couchDB users !
Frédéric
Re: How to code an advanced search
Posted by Frédéric Malo <fr...@gmail.com>.
Thank you Adam,
So the JOIN logic is not hosted in couchDB but in the application layer. The
benefit is scalability, but a little more work :-)
Frédéric
On Mon, Aug 17, 2009 at 23:11, Adam Jacob <ad...@opscode.com> wrote:
> On Sun, Aug 16, 2009 at 1:29 AM, Frédéric Malo<fr...@gmail.com>
> wrote:
> > For example, imagine a music database as iTunes. I would like to create a
> > script to query such as :
> >
> > Sort all songs by genre then album :
> > - with 3 or 5 stars
> > - artist name matching "floyd"
> > - duration between 5 and 8 minutes
> > - heard last week (perhaps another couch database, logging user actions)
> > - ....
> >
> > All the parameters are optional. Users can choose to filter with only 2
> or 5
> > parameters. Other parameters should be added.
>
> The simple answer is "you can't get there from here", at least not
> like you are thinking. You have a couple of options:
>
> * Refactor each of those questions into a single view for each, with
> the key pointing to a list of songs that match.
> * As the user adds more options, you are actually adding an extra
> request for each view
> * Use the built in sorting to get the sort order
> * Query all the various required views and return the final result set
>
> This is following a similar pattern to the ones used by Full text
> search engines - you are using CouchDB to create the indexes for you,
> which return light-weight data that gets culled based on your
> requirements.
>
> Another option would be to actually put all this data into a full text
> engine, and use that. (Solr is a nice option here, and perhaps
> couchdb-lucene.)
>
> Don't be afraid to do a little more work on the systems that talk to
> CouchDB - often when you find yourself reaching for a Join, instead
> reach for multiple queries and doing the computation of the final
> match set yourself.
>
> Adam
>
> (I've seen a system that does just this, only with CDB indexes rather
> than CouchDB, do very complex queries and filtering against every
> hotel in America at very, very high speed.)
>
> --
> Opscode, Inc.
> Adam Jacob, CTO
> T: (206) 508-4759 E: adam@opscode.com
>
Re: How to code an advanced search
Posted by Adam Jacob <ad...@opscode.com>.
On Sun, Aug 16, 2009 at 1:29 AM, Frédéric Malo<fr...@gmail.com> wrote:
> For example, imagine a music database as iTunes. I would like to create a
> script to query such as :
>
> Sort all songs by genre then album :
> - with 3 or 5 stars
> - artist name matching "floyd"
> - duration between 5 and 8 minutes
> - heard last week (perhaps another couch database, logging user actions)
> - ....
>
> All the parameters are optional. Users can choose to filter with only 2 or 5
> parameters. Other parameters should be added.
The simple answer is "you can't get there from here", at least not
like you are thinking. You have a couple of options:
* Refactor each of those questions into a single view for each, with
the key pointing to a list of songs that match.
* As the user adds more options, you are actually adding an extra
request for each view
* Use the built in sorting to get the sort order
* Query all the various required views and return the final result set
This is following a similar pattern to the ones used by Full text
search engines - you are using CouchDB to create the indexes for you,
which return light-weight data that gets culled based on your
requirements.
Another option would be to actually put all this data into a full text
engine, and use that. (Solr is a nice option here, and perhaps
couchdb-lucene.)
Don't be afraid to do a little more work on the systems that talk to
CouchDB - often when you find yourself reaching for a Join, instead
reach for multiple queries and doing the computation of the final
match set yourself.
Adam
(I've seen a system that does just this, only with CDB indexes rather
than CouchDB, do very complex queries and filtering against every
hotel in America at very, very high speed.)
--
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-4759 E: adam@opscode.com