scaling couchdb

[Dale Johnson <da...@gmail.com>]

Greetings,

I'm trying to implement a large scale assessment of CouchDB.

CouchDB shows a great deal of promise, and I'm busy learning
erlang so I can read the source code.

I'm plowing ahead with a wiki entry:

http://wiki.apache.org/couchdb/ConfiguringDistributedSystems

That page is in an 'alpha' state. :)

So my basic plan is to read the source code to see how it
works, and try to get something running in a couple of days.
I'm more than willing to fill out the wiki page, even if it's to say
it doesn't to this and it doesn't do that.

But, the 64k$ question is -- is there any project roadmap
in general, or scaling documents in particular that would
guide potential adopters?

-- 
Dale Johnson (dale.johnson@gmail.com)

Fwd: scaling couchdb

[Dale Johnson <da...@gmail.com>]

No takers?

---------- Forwarded message ----------
From: Dale Johnson <da...@gmail.com>
Date: Mon, Jul 7, 2008 at 4:37 PM
Subject: scaling couchdb
To: couchdb-dev@incubator.apache.org


Greetings,

I'm trying to implement a large scale assessment of CouchDB.

CouchDB shows a great deal of promise, and I'm busy learning
erlang so I can read the source code.

I'm plowing ahead with a wiki entry:

http://wiki.apache.org/couchdb/ConfiguringDistributedSystems

That page is in an 'alpha' state. :)

So my basic plan is to read the source code to see how it
works, and try to get something running in a couple of days.
I'm more than willing to fill out the wiki page, even if it's to say
it doesn't to this and it doesn't do that.

But, the 64k$ question is -- is there any project roadmap
in general, or scaling documents in particular that would
guide potential adopters?

--
Dale Johnson (dale.johnson@gmail.com)



-- 
Dale Johnson (dale.johnson@gmail.com)

Re: scaling couchdb

[Dale Johnson <da...@gmail.com>]

Jan,

Thanks for getting back to me.  Probably the core team has been
at this awhile, and when newbies come by asking the same old
questions, they probably seem a bit thick ;)  Obviously I need
to go through the mailing lists, and RTFM, as they say.  Thanks
for linking me to the obviously placed road map.

So about the roadmap, obviously the "next release" is 0.9.  I'm
going to guess that "server storage partitioning" being in that
third section puts it 1.1-ish, putting us into perhaps next year?

Please don't think I'm trying to be pushy or anything, I just
need to figure out if the timeline (and system design) for CouchDB
fits into the timeline and constraints for the project that I'm
working on.

I think this statement from the FAQ might have got me on the
wrong track:

"How Much Stuff can I Store in CouchDB?

With node partitioning, virtually unlimited. For a single database
instance, the practical scaling limits aren't yet known."

I think it suggests that CouchDB supports node partitioning,
which is non-trivial for applications to implement in a way that's
easy for operations to manage.

Dale.

On Wed, Jul 9, 2008 at 12:42 AM, Jan Lehnardt <ja...@apache.org> wrote:
> Heya Dale,
> On Jul 8, 2008, at 01:37, Dale Johnson wrote:
>>
>> I'm trying to implement a large scale assessment of CouchDB.
>
> Cool! :)
>
>
>> http://wiki.apache.org/couchdb/ConfiguringDistributedSystems
>> That page is in an 'alpha' state. :)
>
> Thanks for putting this stuff together.
>
>
>> But, the 64k$ question is -- is there any project roadmap
>> in general, or scaling documents in particular that would
>> guide potential adopters?
>
> Not to sound too grumpy, but
> http://www.google.com/search?q=couchdb%20roadmap
> has a good first result :) The Roadmap needs updating,
> but we are in the process to discuss what things we need
> and which version they should go. We'll update the roadmap
> page then.
>
> You can see that the scaling parts are Future Feature work.
> A couple of people have voiced interest in contributing there
> especially the database partitioning, but nothing has come
> out of that yet.
>
> At the moment, CouchDB runs best on a single machine
> with multiple machines for a cluster using replication to
> synchronise data. Erlang allows a VM to run on multiple
> machines and we do not yet take advantage of that fact.
> This is an aera that is worth investigating.
>
> Another one is making the view server actually use the
> map/reduce paradigm in parallel. At the moment things
> are computed serially.
>
> There is no guide though, since we are not yet focussing
> on the scaling (which doesn't mean we don't have it in
> mind :). Any contributions to this area are highly welcome.
>
> Feel free to follow up with any questions you might have.
>
> Cheers
> Jan
> --
>



-- 
Dale Johnson (dale.johnson@gmail.com)

Re: scaling couchdb

[Jan Lehnardt <ja...@apache.org>]

Heya Dale,
On Jul 8, 2008, at 01:37, Dale Johnson wrote:
> I'm trying to implement a large scale assessment of CouchDB.

Cool! :)


> http://wiki.apache.org/couchdb/ConfiguringDistributedSystems
> That page is in an 'alpha' state. :)

Thanks for putting this stuff together.


> But, the 64k$ question is -- is there any project roadmap
> in general, or scaling documents in particular that would
> guide potential adopters?

Not to sound too grumpy, but
http://www.google.com/search?q=couchdb%20roadmap
has a good first result :) The Roadmap needs updating,
but we are in the process to discuss what things we need
and which version they should go. We'll update the roadmap
page then.

You can see that the scaling parts are Future Feature work.
A couple of people have voiced interest in contributing there
especially the database partitioning, but nothing has come
out of that yet.

At the moment, CouchDB runs best on a single machine
with multiple machines for a cluster using replication to
synchronise data. Erlang allows a VM to run on multiple
machines and we do not yet take advantage of that fact.
This is an aera that is worth investigating.

Another one is making the view server actually use the
map/reduce paradigm in parallel. At the moment things
are computed serially.

There is no guide though, since we are not yet focussing
on the scaling (which doesn't mean we don't have it in
mind :). Any contributions to this area are highly welcome.

Feel free to follow up with any questions you might have.

Cheers
Jan
--

Re: scaling couchdb

[Peter Schade <l-...@gmx.net>]

----- Original Message ----- 
From: "Dale Johnson" <da...@gmail.com>
To: <co...@incubator.apache.org>
Sent: Tuesday, July 08, 2008 1:37 AM
Subject: scaling couchdb


> Greetings,
>
> I'm trying to implement a large scale assessment of CouchDB.
>
> CouchDB shows a great deal of promise, and I'm busy learning
> erlang so I can read the source code.
>
> I'm plowing ahead with a wiki entry:
>
> http://wiki.apache.org/couchdb/ConfiguringDistributedSystems
>

Yes that interesting questions that other user would also interrest. The
first time i saw couchdb i thought cool, but then ... not much information
on the scaling ... and so on...

Bye Peter