You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by Audrey Lee <au...@gmail.com> on 2010/03/29 07:41:23 UTC
Replicating a subset of data(data for user of a website)
Hello List,
I'm starting work on a website for tourists.
The basic use-case is that a tourist uses the site to store and
retrieve information about itineraries.
In addition to the website I want to create a netbook application
which the tourist can use while she is disconnected from the web.
I want the netbook application-data-store to synchronize with the
data-store of the website.
The replication abilities of couchDB interest me.
I'm curious about some deployment issues.
How easy is it to implement couchDB such that all of the data for all
of the tourists is in a data-center?
Obviously I want each tourist to have a local copy of only her data.
Is it easy to deploy couchDB on each netbook so the netbook has only
data related to 1 user?
Based on the small amount of reading that I've done, I gather that it
is easy to replicate an entire couchDB between hosts.
For my use case, I dont want to replicate an entire couchDB from a
data-center to a netbook.
Instead I want to replicate a subset of a couchDB.
Has anyone on this list dealt with a use-case similar to my
tourist-site-and-app use-case?
--Audrey
Re: Replicating a subset of data(data for user of a website)
Posted by David Goodlad <da...@goodlad.ca>.
On Tue, Mar 30, 2010 at 2:42 AM, Audrey Lee <au...@gmail.com> wrote:
> At the risk of appearing lazy about RTFM, I'll add another question to
> the thread.
>
> If I have 1,000 users then my use-case would convince me to create a
> couchDB database for each user.
>
> So, if I have 1,000 couchDB databases installed on my server in the
> datacenter, does that mean that I'd have 1,000 couchDB
> server-processes?
No, one CouchDB "node" (process, essentially) can have many databases.
Think of mysql - one mysql daemon process can serve up many
independent databases.
Dave
Re: Replicating a subset of data(data for user of a website)
Posted by Audrey Lee <au...@gmail.com>.
At the risk of appearing lazy about RTFM, I'll add another question to
the thread.
If I have 1,000 users then my use-case would convince me to create a
couchDB database for each user.
So, if I have 1,000 couchDB databases installed on my server in the
datacenter, does that mean that I'd have 1,000 couchDB
server-processes?
Or could I configure couchDB on my server such that it is more like a
webserver (which might be only a few processes).
While waiting for a possible response to my question, I'll start
reading this book:
- http://books.couchdb.org/relax/
Perhaps the answer is in there.
Thanks.
On 3/28/10, Jan Lehnardt <ja...@apache.org> wrote:
> Hi Audrey,
>
> CouchDB is well suited for your needs :)
>
> There are two approaches you can take:
>
> 1) Create a database per user (or itinerary). That way,
> replicating a database will only include the data that's
> interesting to a single user. On the server side, you
> don't end up with a ginormous database that has all
> users' data mashed together.
>
> 2) If you like ginormous databases, you can use a new
> technique called "filtered replication" that lets you replicate
> only a subset to your users netbooks. It is described in more
> detail on the Couchio blog*. Note that this is only available
> in CouchDB 0.11.0 which is about to be released.
>
> *
> http://blog.couch.io/post/468392274/whats-new-in-apache-couchdb-0-11-part-three-new
>
> 3) Bonus version: For completeness sake, there is another
> possible approach that might work as a stop-gap but is not,
> I think feasible. You can define a document update validation
> function on your users' netbooks to block all writes that include
> data they are not interested in. If you replicate now, the netbook
> will only include the user's data, but for each client you will send
> all your data down and the client's CouchDB rejects the writes.
> Like I said, very likely not feasible in your case.
>
> I'd go with 1) if you don't need the big database and 2) if you do.
>
> Cheers
> Jan
> --
>
>
>
> On 28 Mar 2010, at 22:41, Audrey Lee wrote:
>
>> Hello List,
>>
>> I'm starting work on a website for tourists.
>>
>> The basic use-case is that a tourist uses the site to store and
>> retrieve information about itineraries.
>>
>> In addition to the website I want to create a netbook application
>> which the tourist can use while she is disconnected from the web.
>>
>> I want the netbook application-data-store to synchronize with the
>> data-store of the website.
>>
>> The replication abilities of couchDB interest me.
>>
>> I'm curious about some deployment issues.
>>
>> How easy is it to implement couchDB such that all of the data for all
>> of the tourists is in a data-center?
>>
>> Obviously I want each tourist to have a local copy of only her data.
>>
>> Is it easy to deploy couchDB on each netbook so the netbook has only
>> data related to 1 user?
>>
>> Based on the small amount of reading that I've done, I gather that it
>> is easy to replicate an entire couchDB between hosts.
>>
>> For my use case, I dont want to replicate an entire couchDB from a
>> data-center to a netbook.
>>
>> Instead I want to replicate a subset of a couchDB.
>>
>> Has anyone on this list dealt with a use-case similar to my
>> tourist-site-and-app use-case?
>>
>> --Audrey
>
>
Re: Replicating a subset of data(data for user of a website)
Posted by Mirsal Ennaime <mi...@gmail.com>.
I'd go for 1)
You can then fetch data from your user databases to populate a
centralized one if you need to run calculations on the data of
multiple users.
Re: Replicating a subset of data(data for user of a website)
Posted by Jan Lehnardt <ja...@apache.org>.
Hi Audrey,
CouchDB is well suited for your needs :)
There are two approaches you can take:
1) Create a database per user (or itinerary). That way,
replicating a database will only include the data that's
interesting to a single user. On the server side, you
don't end up with a ginormous database that has all
users' data mashed together.
2) If you like ginormous databases, you can use a new
technique called "filtered replication" that lets you replicate
only a subset to your users netbooks. It is described in more
detail on the Couchio blog*. Note that this is only available
in CouchDB 0.11.0 which is about to be released.
* http://blog.couch.io/post/468392274/whats-new-in-apache-couchdb-0-11-part-three-new
3) Bonus version: For completeness sake, there is another
possible approach that might work as a stop-gap but is not,
I think feasible. You can define a document update validation
function on your users' netbooks to block all writes that include
data they are not interested in. If you replicate now, the netbook
will only include the user's data, but for each client you will send
all your data down and the client's CouchDB rejects the writes.
Like I said, very likely not feasible in your case.
I'd go with 1) if you don't need the big database and 2) if you do.
Cheers
Jan
--
On 28 Mar 2010, at 22:41, Audrey Lee wrote:
> Hello List,
>
> I'm starting work on a website for tourists.
>
> The basic use-case is that a tourist uses the site to store and
> retrieve information about itineraries.
>
> In addition to the website I want to create a netbook application
> which the tourist can use while she is disconnected from the web.
>
> I want the netbook application-data-store to synchronize with the
> data-store of the website.
>
> The replication abilities of couchDB interest me.
>
> I'm curious about some deployment issues.
>
> How easy is it to implement couchDB such that all of the data for all
> of the tourists is in a data-center?
>
> Obviously I want each tourist to have a local copy of only her data.
>
> Is it easy to deploy couchDB on each netbook so the netbook has only
> data related to 1 user?
>
> Based on the small amount of reading that I've done, I gather that it
> is easy to replicate an entire couchDB between hosts.
>
> For my use case, I dont want to replicate an entire couchDB from a
> data-center to a netbook.
>
> Instead I want to replicate a subset of a couchDB.
>
> Has anyone on this list dealt with a use-case similar to my
> tourist-site-and-app use-case?
>
> --Audrey
Re: Replicating a subset of data(data for user of a website)
Posted by David Goodlad <da...@goodlad.ca>.
On Mon, Mar 29, 2010 at 4:41 PM, Audrey Lee <au...@gmail.com> wrote:
> Hello List,
>
> I'm starting work on a website for tourists.
>
> The basic use-case is that a tourist uses the site to store and
> retrieve information about itineraries.
>
> In addition to the website I want to create a netbook application
> which the tourist can use while she is disconnected from the web.
>
> I want the netbook application-data-store to synchronize with the
> data-store of the website.
>
> The replication abilities of couchDB interest me.
>
> I'm curious about some deployment issues.
>
> How easy is it to implement couchDB such that all of the data for all
> of the tourists is in a data-center?
>
> Obviously I want each tourist to have a local copy of only her data.
>
> Is it easy to deploy couchDB on each netbook so the netbook has only
> data related to 1 user?
>
> Based on the small amount of reading that I've done, I gather that it
> is easy to replicate an entire couchDB between hosts.
>
> For my use case, I dont want to replicate an entire couchDB from a
> data-center to a netbook.
>
> Instead I want to replicate a subset of a couchDB.
>
> Has anyone on this list dealt with a use-case similar to my
> tourist-site-and-app use-case?
Hi Audrey
Have you considered having multiple databases on your server, one for
each user? This would allow you easily replicate the entire contents
of that db to the netbook. This is quite a common thing in Couch:
multiple dbs for the same app, separated by user, or groups of users,
or...
Dave