Suggestions required on app design

[Ramkrishna Kulkarni <ra...@gmail.com>]

Hello All,

Please bear with me, I'm new. I have been playing with CouchDB
recently and I find it kool! I would appreciate if someone from the
list could answer few of my questions below.

Consider I have a database-per-user design and I want to be RESTful.

a) How do I list all users by just typing a URL like http://server/users/?
b) Is there a way to specify startkey and endkey while listing such
databases? For example: I want to list only databases of users
starting with "a"

I know this is possible if there is sinlge "users" database with a
document for each user with _id=user-id.

Further, consider I have database-per-blog as well (assume this is a
simple blog application).
a) How do I create something like view (views I believe cannot be run
across databases) so that whenever a user resource is requested, I
return all blogs written by that user in addition to user information?
Basically links to all blog resources.

Probably this logic should be in the application. Fire two queries,
one to get user data and another to get blogs?

The reason I'm trying to do this is because I'm bit scared about
having data in just couple of huge files. I would rather have data
spread out in several smaller files.

Probably there are better ways to do this. Please let me know.

Thanks,
Ram

Re: Suggestions required on app design

[Ramkrishna Kulkarni <ra...@gmail.com>]

On Wed, Mar 4, 2009 at 11:53 PM, Dean Landolt <de...@deanlandolt.com> wrote:
> On Wed, Mar 4, 2009 at 12:34 PM, Ramkrishna Kulkarni <
> ramkrishna.kulkarni@gmail.com> wrote:
>
>> Hello All,
>>
>> Please bear with me, I'm new. I have been playing with CouchDB
>> recently and I find it kool! I would appreciate if someone from the
>> list could answer few of my questions below.
>>
>> Consider I have a database-per-user design and I want to be RESTful.
>>
>> a) How do I list all users by just typing a URL like http://server/users/?
>> b) Is there a way to specify startkey and endkey while listing such
>> databases? For example: I want to list only databases of users
>> starting with "a"
>
>
> As it stands now you can use a proxy to rewrite urls to specifics, including
> startkey and endkey as querystring args.
>
> There's been some conversation on implementing routing inside the design doc
> itself to make it easy (and replicatible) to have pretty urls...
>
> But I'd still argue you'll need some sort of rewriting at the proxy level to
> allow http://server/users -- if there's routing in the design doc, it should
> probably be limited to http://server/<db>/<designdoc>/... to prevent
> malicious (or accidental) httpd hijacking.
>

I see. Makes sense. But having routing inside design doc would be great.

>>
>>
>> I know this is possible if there is sinlge "users" database with a
>> document for each user with _id=user-id.
>>
>> Further, consider I have database-per-blog as well (assume this is a
>> simple blog application).
>> a) How do I create something like view (views I believe cannot be run
>> across databases) so that whenever a user resource is requested, I
>> return all blogs written by that user in addition to user information?
>> Basically links to all blog resources.
>>
>> Probably this logic should be in the application. Fire two queries,
>> one to get user data and another to get blogs?
>
>
> If you plan on splitting these across dbs, it sounds like you'll need an
> administrative db to keep the users/blogs metadata. You could map over that
> with ease.
>

I'm leaning towards making multiple client requests instead. But your
idea sounds interesting.

>>
>>
>> The reason I'm trying to do this is because I'm bit scared about
>> having data in just couple of huge files. I would rather have data
>> spread out in several smaller files.
>
>
> What's the worry?
>

I'm not a file system expert but I'm worried about file corruption. In
the worst case, if data is split across files, its not that bad
compared to huge files getting corrupted. I'm not talking about
CouchDB. Let's say bad blocks. I have also read in one of the mail
threads that its good from performance point of view as it makes
concurrent writes possible.

>>
>>
>> Probably there are better ways to do this. Please let me know.
>>
>> Thanks,
>> Ram
>>
>

Re: Suggestions required on app design

[Dean Landolt <de...@deanlandolt.com>]

On Wed, Mar 4, 2009 at 12:34 PM, Ramkrishna Kulkarni <
ramkrishna.kulkarni@gmail.com> wrote:

> Hello All,
>
> Please bear with me, I'm new. I have been playing with CouchDB
> recently and I find it kool! I would appreciate if someone from the
> list could answer few of my questions below.
>
> Consider I have a database-per-user design and I want to be RESTful.
>
> a) How do I list all users by just typing a URL like http://server/users/?
> b) Is there a way to specify startkey and endkey while listing such
> databases? For example: I want to list only databases of users
> starting with "a"


As it stands now you can use a proxy to rewrite urls to specifics, including
startkey and endkey as querystring args.

There's been some conversation on implementing routing inside the design doc
itself to make it easy (and replicatible) to have pretty urls...

But I'd still argue you'll need some sort of rewriting at the proxy level to
allow http://server/users -- if there's routing in the design doc, it should
probably be limited to http://server/<db>/<designdoc>/... to prevent
malicious (or accidental) httpd hijacking.

>
>
> I know this is possible if there is sinlge "users" database with a
> document for each user with _id=user-id.
>
> Further, consider I have database-per-blog as well (assume this is a
> simple blog application).
> a) How do I create something like view (views I believe cannot be run
> across databases) so that whenever a user resource is requested, I
> return all blogs written by that user in addition to user information?
> Basically links to all blog resources.
>
> Probably this logic should be in the application. Fire two queries,
> one to get user data and another to get blogs?


If you plan on splitting these across dbs, it sounds like you'll need an
administrative db to keep the users/blogs metadata. You could map over that
with ease.

>
>
> The reason I'm trying to do this is because I'm bit scared about
> having data in just couple of huge files. I would rather have data
> spread out in several smaller files.


What's the worry?

>
>
> Probably there are better ways to do this. Please let me know.
>
> Thanks,
> Ram
>

Re: Suggestions required on app design

[Ramkrishna Kulkarni <ra...@gmail.com>]

On Wed, Mar 4, 2009 at 11:40 PM, Troy Kruthoff <tk...@gmail.com> wrote:
>
> On Mar 4, 2009, at 9:34 AM, Ramkrishna Kulkarni wrote:
>
>> Hello All,
>>
>> Please bear with me, I'm new. I have been playing with CouchDB
>> recently and I find it kool! I would appreciate if someone from the
>> list could answer few of my questions below.
>>
>> Consider I have a database-per-user design and I want to be RESTful.
>>
>> a) How do I list all users by just typing a URL like http://server/users/?
>
> There is a built-in view http://host:5984/_all_dbs
>

That would list all databases including those of blogs. I want to list
only user databases.

>>
>> b) Is there a way to specify startkey and endkey while listing such
>> databases? For example: I want to list only databases of users
>> starting with "a"
>
> I believe the _all_dbs view can be filtered just like any other view (see
> query options http://wiki.apache.org/couchdb/HTTP_view_API)
>

Well, I tried that. _all_dbs list all dbs and ignores query options it seems.

>>
>>
>> I know this is possible if there is sinlge "users" database with a
>> document for each user with _id=user-id.
>>
>> Further, consider I have database-per-blog as well (assume this is a
>> simple blog application).
>> a) How do I create something like view (views I believe cannot be run
>> across databases) so that whenever a user resource is requested, I
>> return all blogs written by that user in addition to user information?
>> Basically links to all blog resources.
>>
>> Probably this logic should be in the application. Fire two queries,
>> one to get user data and another to get blogs?
>
> Your on the right track.  In the users database you can store the name of
> the blog databases.
>

Thanks for your inputs. I'll look into it.

>>
>>
>> The reason I'm trying to do this is because I'm bit scared about
>> having data in just couple of huge files. I would rather have data
>> spread out in several smaller files.
>>
>> Probably there are better ways to do this. Please let me know.
>>
>> Thanks,
>> Ram
>
>

Re: Suggestions required on app design

[Troy Kruthoff <tk...@gmail.com>]

On Mar 4, 2009, at 9:34 AM, Ramkrishna Kulkarni wrote:

> Hello All,
>
> Please bear with me, I'm new. I have been playing with CouchDB
> recently and I find it kool! I would appreciate if someone from the
> list could answer few of my questions below.
>
> Consider I have a database-per-user design and I want to be RESTful.
>
> a) How do I list all users by just typing a URL like http://server/users/?

There is a built-in view http://host:5984/_all_dbs

>
> b) Is there a way to specify startkey and endkey while listing such
> databases? For example: I want to list only databases of users
> starting with "a"

I believe the _all_dbs view can be filtered just like any other view  
(see query options http://wiki.apache.org/couchdb/HTTP_view_API)

>
>
> I know this is possible if there is sinlge "users" database with a
> document for each user with _id=user-id.
>
> Further, consider I have database-per-blog as well (assume this is a
> simple blog application).
> a) How do I create something like view (views I believe cannot be run
> across databases) so that whenever a user resource is requested, I
> return all blogs written by that user in addition to user information?
> Basically links to all blog resources.
>
> Probably this logic should be in the application. Fire two queries,
> one to get user data and another to get blogs?

Your on the right track.  In the users database you can store the name  
of the blog databases.

>
>
> The reason I'm trying to do this is because I'm bit scared about
> having data in just couple of huge files. I would rather have data
> spread out in several smaller files.
>
> Probably there are better ways to do this. Please let me know.
>
> Thanks,
> Ram