You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Daniele Testa <da...@gmail.com> on 2011/06/10 11:41:03 UTC
One view per "query"?
Hi all,
I just want to check with you guys that I understand things correctly.
Lets say I have a database that stores music songs.
Each song has a "title", "insert_date", "album", "genre".
Now, if I want to do these "queries":
"All songs that belongs to album=X ordered by title"
"All songs that belongs to genre=X ordered by insert_date descending"
"All songs that belongs to genre=X ordered by title"
"All songs ordered by insert_date descending"
Etc..
Is it true that I will have to create an unique view for each of these cases?
I assume I cannot do something like
emit( [doc.album, doc.genre, doc.insert_date, doc.title], doc);
And then just choose what index in that array I want to use? It is
ordered left-to-right, right?
That would mean that I cannot do something like
startkey=[null, null, 0, null]&endkey=[null, null, {}, null]
.. to get all songs ordered by "insert_date"?
Regards,
Daniele
Re: One view per "query"?
Posted by MK <mk...@cognitivedissonance.ca>.
On Fri, 10 Jun 2011 17:10:13 +0200
Daniele Testa <da...@gmail.com> wrote:
> Hmm, if I have 10 million songs in my database, wouldn't that make
> couchDB send all those 10 million documents to the client for the
> sort?
> That sounds crazy to me :)
None of what I was suggesting involves any client side code at all;
it's all about interaction between a database (couch) and an http server
(apache, node, whatever) on the same machine. The server function was
js-esque for purposes of illustration but it could be PHP, perl, C/C+
+, anything at all. It's server side. You can do it however you
want. If doing everything with couch is what you want, then you
might be stuck with 3*4 views. But you do not have to do that.
WRT "sending all 10 million documents", if you use the 3 views and
"?key=", couch is only going to pass on what is relevant to the server.
Anyway, IMHO: couch is a great, easy to use database but very awkward
as a be all and end all front-end server. So why make trouble for
yourself that way?
> 2011/6/10 MK <mk...@cognitivedissonance.ca>:
> > On Fri, 10 Jun 2011 12:05:20 +0200
> > Daniele Testa <da...@gmail.com> wrote:
> >> That would mean that I need 3*4=12 views to accomodate all those
> >> different "queries"?
> >
> > Only if you have to have your application 100% served directly
> > from couch. However, IMO the view and shows make a lot more sense as
> > ancillary functions for use by an external server. Working that
> > way, you could cover these twelve cases with a single view and a
> > single function in the server proper, or 3 views and a single
> > function, or 3 views and 4 function functions in the server -- ie,
> > you are free to choose how to do it. For example, if you want to
> > maintain the least amount of code, you'd go with the first option,
> > a single view with a map defined:
> >
> > function (doc) {
> > emit (doc.title, [doc.artist, doc.genre, doc.insert_date]);
> > }
> >
> > Dead simple, returns an array of all albums keyed by title. Your
> > server uses this like (js-esque pseudo code):
> >
> > function (criteria, value, sortby) {
> > var rawJSON = [whatever method you use to query couch];
> > // most languages have a means of turning JSON strings into
> > data structures:
> > var data = parse rawJSON;
> > var selected; // array type
> > for (e in data.rows) {
> > // you could also use a map type function here if you or your
> > language have such
> > if (data.rows.criteria == value)
> > selected.push(data.rows.criteria);
> > }
> > return Sort(selected, sortby);
> > }
> >
> > How the "Sort" works depends on what lang you are programming in,
> > sortby would probably be a comparison (eg: a.title > b.title ? b :
> > a).
> >
> > If you want efficiency and you believe couch will do the most
> > efficient job of selected data (which it probably will in most
> > settings), then you might want the 3 views (these are all one
> > liners, nb), and possibly even 3 views and 4 ordering methods in
> > the server.
> >
> > From this perspective, views and shows make a lot of sense and are
> > very useful -- they are like being able to incorporate simple,
> > customized SQL queries into an sql db, providing you with a nice
> > separation of concerns (you don't need any SQL in your server code,
> > the db itself incorporates a "design" specific to the form of its
> > content). But understood as the final points in a query, they just
> > seem awkward, and you have no such separation.
> >
> >
> > --
> > "Enthusiasm is not the enemy of the intellect." (said of Irving
> > Howe) "The angel of history[...]is turned toward the past." (Walter
> > Benjamin)
> >
> >
>
--
"Enthusiasm is not the enemy of the intellect." (said of Irving Howe)
"The angel of history[...]is turned toward the past." (Walter Benjamin)
Re: One view per "query"?
Posted by Daniele Testa <da...@gmail.com>.
Hmm, if I have 10 million songs in my database, wouldn't that make
couchDB send all those 10 million documents to the client for the
sort?
That sounds crazy to me :)
Regards,
Daniele
2011/6/10 MK <mk...@cognitivedissonance.ca>:
> On Fri, 10 Jun 2011 12:05:20 +0200
> Daniele Testa <da...@gmail.com> wrote:
>> That would mean that I need 3*4=12 views to accomodate all those
>> different "queries"?
>
> Only if you have to have your application 100% served directly
> from couch. However, IMO the view and shows make a lot more sense as
> ancillary functions for use by an external server. Working that way,
> you could cover these twelve cases with a single view and a single
> function in the server proper, or 3 views and a single function, or 3
> views and 4 function functions in the server -- ie, you are free to
> choose how to do it. For example, if you want to maintain the least
> amount of code, you'd go with the first option, a single view with a
> map defined:
>
> function (doc) {
> emit (doc.title, [doc.artist, doc.genre, doc.insert_date]);
> }
>
> Dead simple, returns an array of all albums keyed by title. Your
> server uses this like (js-esque pseudo code):
>
> function (criteria, value, sortby) {
> var rawJSON = [whatever method you use to query couch];
> // most languages have a means of turning JSON strings into
> data structures:
> var data = parse rawJSON;
> var selected; // array type
> for (e in data.rows) {
> // you could also use a map type function here if you or your
> language have such
> if (data.rows.criteria == value)
> selected.push(data.rows.criteria);
> }
> return Sort(selected, sortby);
> }
>
> How the "Sort" works depends on what lang you are programming in,
> sortby would probably be a comparison (eg: a.title > b.title ? b : a).
>
> If you want efficiency and you believe couch will do the most efficient
> job of selected data (which it probably will in most settings), then you
> might want the 3 views (these are all one liners, nb), and possibly
> even 3 views and 4 ordering methods in the server.
>
> From this perspective, views and shows make a lot of sense and are very
> useful -- they are like being able to incorporate simple, customized SQL
> queries into an sql db, providing you with a nice separation of
> concerns (you don't need any SQL in your server code, the db itself
> incorporates a "design" specific to the form of its content). But
> understood as the final points in a query, they just seem awkward, and
> you have no such separation.
>
>
> --
> "Enthusiasm is not the enemy of the intellect." (said of Irving Howe)
> "The angel of history[...]is turned toward the past." (Walter Benjamin)
>
>
RE: One view per "query"?
Posted by Nils Breunese <N....@vpro.nl>.
The query arguments are startkey and endkey, not startKey and endKey.
You might want to just use key= in this case though.
http://wiki.apache.org/couchdb/HTTP_view_API#Querying_Options
Nils.
________________________________________
Van: Peter Nolan [peterwnolan@gmail.com]
Verzonden: vrijdag 10 juni 2011 17:27
Aan: user@couchdb.apache.org
Onderwerp: Re: One view per "query"?
you can create a view called byArtist whose code looks like the following.
function (doc) {
if (doc.artist) {
emit(doc.artist,otherStuffYouWantToKnow)
}
}
then you can query this view with startKey='dylan' and endKey='dylan'
and receive those docs that only have the field of artist set to 'dylan'.
------------------------------------------------------------------------
VPRO www.vpro.nl
------------------------------------------------------------------------
Re: One view per "query"?
Posted by Daniele Testa <da...@gmail.com>.
Ok, I will try some different things and see how it goes :)
No, the music thing was just an example. I am working on something
completely different.
Regards,
Daniele
2011/6/10 Peter Nolan <pe...@gmail.com>:
>>
>> in regards to '10 million documents'
>>
>
>
>
>> not necessarily. Say you want to find a doc whose field of 'artist' is
>> 'dylan'
>>
>
> you can create a view called byArtist whose code looks like the following.
>
> function (doc) {
>
> if (doc.artist) {
> emit(doc.artist,otherStuffYouWantToKnow)
> }
>
> }
>
> then you can query this view with startKey='dylan' and endKey='dylan'
> and receive those docs that only have the field of artist set to 'dylan'.
> then you can do some kind of filtering on the client side.
>
> yes, you'll have to do a little copypasta for each field you're interested
> in querying on.
>
> Again, if you have more complicated search algorithms (like regexps and
> whatnot) check out the filter functions from the changes feed.
>
> out of curiosity, what kind of project are you working? Cause i did a
> little music app myself and would love to see code if you're willing to
> share.
>
> -good luck
>
> -Pete
>
Re: One view per "query"?
Posted by Peter Nolan <pe...@gmail.com>.
>
> in regards to '10 million documents'
>
> not necessarily. Say you want to find a doc whose field of 'artist' is
> 'dylan'
>
you can create a view called byArtist whose code looks like the following.
function (doc) {
if (doc.artist) {
emit(doc.artist,otherStuffYouWantToKnow)
}
}
then you can query this view with startKey='dylan' and endKey='dylan'
and receive those docs that only have the field of artist set to 'dylan'.
then you can do some kind of filtering on the client side.
yes, you'll have to do a little copypasta for each field you're interested
in querying on.
Again, if you have more complicated search algorithms (like regexps and
whatnot) check out the filter functions from the changes feed.
out of curiosity, what kind of project are you working? Cause i did a
little music app myself and would love to see code if you're willing to
share.
-good luck
-Pete
Re: One view per "query"?
Posted by Peter Nolan <pe...@gmail.com>.
you might want to check out filter functions from the changes feed - this
would allow you to do more 'intense' querying from the database.
http://guide.couchdb.org/draft/notifications.html
<http://guide.couchdb.org/draft/notifications.html>though, from my
experience, it seems that using the filters function from the changes
database is slower than querying from a view. Has anyone else noticed slow
downs between views and changes/filters ?
Re: One view per "query"?
Posted by MK <mk...@cognitivedissonance.ca>.
On Fri, 10 Jun 2011 12:05:20 +0200
Daniele Testa <da...@gmail.com> wrote:
> That would mean that I need 3*4=12 views to accomodate all those
> different "queries"?
Only if you have to have your application 100% served directly
from couch. However, IMO the view and shows make a lot more sense as
ancillary functions for use by an external server. Working that way,
you could cover these twelve cases with a single view and a single
function in the server proper, or 3 views and a single function, or 3
views and 4 function functions in the server -- ie, you are free to
choose how to do it. For example, if you want to maintain the least
amount of code, you'd go with the first option, a single view with a
map defined:
function (doc) {
emit (doc.title, [doc.artist, doc.genre, doc.insert_date]);
}
Dead simple, returns an array of all albums keyed by title. Your
server uses this like (js-esque pseudo code):
function (criteria, value, sortby) {
var rawJSON = [whatever method you use to query couch];
// most languages have a means of turning JSON strings into
data structures:
var data = parse rawJSON;
var selected; // array type
for (e in data.rows) {
// you could also use a map type function here if you or your
language have such
if (data.rows.criteria == value)
selected.push(data.rows.criteria);
}
return Sort(selected, sortby);
}
How the "Sort" works depends on what lang you are programming in,
sortby would probably be a comparison (eg: a.title > b.title ? b : a).
If you want efficiency and you believe couch will do the most efficient
job of selected data (which it probably will in most settings), then you
might want the 3 views (these are all one liners, nb), and possibly
even 3 views and 4 ordering methods in the server.
>From this perspective, views and shows make a lot of sense and are very
useful -- they are like being able to incorporate simple, customized SQL
queries into an sql db, providing you with a nice separation of
concerns (you don't need any SQL in your server code, the db itself
incorporates a "design" specific to the form of its content). But
understood as the final points in a query, they just seem awkward, and
you have no such separation.
--
"Enthusiasm is not the enemy of the intellect." (said of Irving Howe)
"The angel of history[...]is turned toward the past." (Walter Benjamin)
Re: One view per "query"?
Posted by Sean Copenhaver <se...@gmail.com>.
You can also hit the views with a 'stale=update_after' if on 1.1.
On Fri, Jun 10, 2011 at 9:15 AM, Owen Marshall <om...@facilityone.com>wrote:
> At Fri, 10 Jun 2011 12:05:20 +0200,
> Daniele Testa wrote:
>
> > Another question I have is what benefit do I get when putting several
> > views in one design document?
>
> Views within the same design document are indexed together, saving space
> and speeding up view updates.
>
> > As I understood, this just causes problems like that all views needs
> > to update, thus locking all the views in the design document.
>
> I was also concerned about this at first, but in my experience, CouchDB
> updates views *very* quickly. My application's write rate is significantly
> less than CouchDB's update rate.
>
> If your application doesn't behave like that, you can always:
>
> * query with stale=ok as appropriate
> * hit the views more often, reducing latency
>
> --
> Owen Marshall
> FacilityONE
> http://www.facilityone.com | (502) 805-2126
>
>
--
“The limits of language are the limits of one's world. “ -Ludwig von
Wittgenstein
Re: One view per "query"?
Posted by Owen Marshall <om...@facilityone.com>.
At Fri, 10 Jun 2011 12:05:20 +0200,
Daniele Testa wrote:
> Another question I have is what benefit do I get when putting several
> views in one design document?
Views within the same design document are indexed together, saving space and speeding up view updates.
> As I understood, this just causes problems like that all views needs
> to update, thus locking all the views in the design document.
I was also concerned about this at first, but in my experience, CouchDB updates views *very* quickly. My application's write rate is significantly less than CouchDB's update rate.
If your application doesn't behave like that, you can always:
* query with stale=ok as appropriate
* hit the views more often, reducing latency
--
Owen Marshall
FacilityONE
http://www.facilityone.com | (502) 805-2126
Re: One view per "query"?
Posted by Daniele Testa <da...@gmail.com>.
Ok, thanks. It will be _a lot_ of views then.
I mean, you might want to select songs based on title, album or genre
at the same time as you can order by title, album, genre or
insert_date
That would mean that I need 3*4=12 views to accomodate all those
different "queries"?
Another question I have is what benefit do I get when putting several
views in one design document?
As I understood, this just causes problems like that all views needs
to update, thus locking all the views in the design document.
Why is it even possible to put several views in one design document
then? I am sure I am missing some important point :)
Regards,
Daniele
2011/6/10 Clement Hallet <cl...@challet.eu>:
> I think you're right
>
>> startkey=[null, null, 0, null]&endkey=[null, null, {}, null]
>
> that query would still return results, the ones wich match all the conditions :
>
> - doc.album = null
> - doc.genre = null
> - some range with doc.insert_date, doc.title (not sure what it would really use)
>
> So if you want to order by insert_date, you have to create a view where the key (or the first element of it) is the insert_date.
>
>
> --
> Clément
>
>
>
> Le 10 juin 2011 à 11:41, Daniele Testa a écrit :
>
>> Hi all,
>>
>> I just want to check with you guys that I understand things correctly.
>>
>> Lets say I have a database that stores music songs.
>>
>> Each song has a "title", "insert_date", "album", "genre".
>>
>> Now, if I want to do these "queries":
>>
>> "All songs that belongs to album=X ordered by title"
>> "All songs that belongs to genre=X ordered by insert_date descending"
>> "All songs that belongs to genre=X ordered by title"
>> "All songs ordered by insert_date descending"
>>
>> Etc..
>>
>> Is it true that I will have to create an unique view for each of these cases?
>>
>> I assume I cannot do something like
>>
>> emit( [doc.album, doc.genre, doc.insert_date, doc.title], doc);
>>
>> And then just choose what index in that array I want to use? It is
>> ordered left-to-right, right?
>>
>> That would mean that I cannot do something like
>>
>> startkey=[null, null, 0, null]&endkey=[null, null, {}, null]
>>
>> .. to get all songs ordered by "insert_date"?
>>
>> Regards,
>> Daniele
>
>
Re: One view per "query"?
Posted by Clement Hallet <cl...@challet.eu>.
I think you're right
> startkey=[null, null, 0, null]&endkey=[null, null, {}, null]
that query would still return results, the ones wich match all the conditions :
- doc.album = null
- doc.genre = null
- some range with doc.insert_date, doc.title (not sure what it would really use)
So if you want to order by insert_date, you have to create a view where the key (or the first element of it) is the insert_date.
--
Clément
Le 10 juin 2011 à 11:41, Daniele Testa a écrit :
> Hi all,
>
> I just want to check with you guys that I understand things correctly.
>
> Lets say I have a database that stores music songs.
>
> Each song has a "title", "insert_date", "album", "genre".
>
> Now, if I want to do these "queries":
>
> "All songs that belongs to album=X ordered by title"
> "All songs that belongs to genre=X ordered by insert_date descending"
> "All songs that belongs to genre=X ordered by title"
> "All songs ordered by insert_date descending"
>
> Etc..
>
> Is it true that I will have to create an unique view for each of these cases?
>
> I assume I cannot do something like
>
> emit( [doc.album, doc.genre, doc.insert_date, doc.title], doc);
>
> And then just choose what index in that array I want to use? It is
> ordered left-to-right, right?
>
> That would mean that I cannot do something like
>
> startkey=[null, null, 0, null]&endkey=[null, null, {}, null]
>
> .. to get all songs ordered by "insert_date"?
>
> Regards,
> Daniele