modular or monolithic: how do you envision Apache CouchDB?

[Benoit Chesneau <bc...@gmail.com>]

Hi,

Last day we had a quick discussion on IRC about splitting code in
different apps or not. I wasn't totally crystal clear about my position
and I would like to clarify it a little. Imo everything turn around the
question , "how do you envision Apache CouchDB".

It appears for me that some wants to distribute it as a full product,
everything is embedded in one archive. In that case the distributiuon
system is only targeting this goal and nothing in the core allows
alternatives.

While I'm agree we should offer a "canonical" release of Apache CouchDB,
I'm thinking more Apache CouchDB as a collections of modules with a
core. For example CouchDB could be:

a k/v server that can live on one or more machine : the core
with an HTTP api
with a M/R api also accessible with HTT
with a _changes API
with a replicaton API
with a couchapp engine
with javascript support

So anyone can eventually replace, remove or add a module in its own
couchdb distribution. The release system could also be enough smart to
allws a distributor to customize its own couchdb.

What do you think about it? How do you envision CouchDB ?

- benoîr

Re: modular or monolithic: how do you envision Apache CouchDB?

[Benoit Chesneau <bc...@gmail.com>]

On Sun, Feb 5, 2012 at 3:33 PM, Randall Leeds <ra...@gmail.com> wrote:
> I agree with Paul that modularity is not tightly bound to git
> structure. I'm fine to not break out the git structure into multiple
> repos, and would maybe even prefer it. The main advantage of doing it
> would be to merge commits from external projects that decide to pull
> in individual CouchDB components, but that's possible to do in the
> monolithic setup, too. Not white belt git-fu, but it certainly isn't
> dan-level.
>
> If there are components in the Apache CouchDB repo which are optional,
> or can be made optional, patches welcome for turning them off at build
> time.
>
> I am 100% for breaking apart the various modules, though. Code
> organization would benefit greatly, as well as the possibility for
> transclusion of components into other projects.
>
> -R
>

Patches are welcome. of course. but I don't see how it's related to this topic.

Anyway I didn't speak about git or rebar at that point. However it
seems important for me that modules could be removed easily from a
final release and that the distributor don't have to distribute it
with useless code.

Having modules managed as standalone projects offer different advantages:
- history could be managed apart, an it's a lot easier to merge/rebase
etc by doing this.
- removing a module (code and binary) is just a matter of "not
including it in the release"
- modules are really optional.

About using different modules with the core to provide a release, i
don't see where is the real problem. The way to test integration could
be easily managed imo. As far as i know it's a matter of process.
Speaking about bigcouch I can feel the pain, but that's mostly because
the process isn't strict as it can be, and indeed some modules aren't
really modules in bigcouch case. I can see in that case bigcouch as
couch+fabric+mem3 = 1 core app, while chttpd, view, rexi, and maybe
the attachment service could be standalone apps if they can works as
standalone app (rexi) or replaced (chttpd,  view server).

And we could do that for couch too.

- benoît

Re: modular or monolithic: how do you envision Apache CouchDB?

[Randall Leeds <ra...@gmail.com>]

I agree with Paul that modularity is not tightly bound to git
structure. I'm fine to not break out the git structure into multiple
repos, and would maybe even prefer it. The main advantage of doing it
would be to merge commits from external projects that decide to pull
in individual CouchDB components, but that's possible to do in the
monolithic setup, too. Not white belt git-fu, but it certainly isn't
dan-level.

If there are components in the Apache CouchDB repo which are optional,
or can be made optional, patches welcome for turning them off at build
time.

I am 100% for breaking apart the various modules, though. Code
organization would benefit greatly, as well as the possibility for
transclusion of components into other projects.

-R

On Sun, Feb 5, 2012 at 10:45, Paul Davis <pa...@gmail.com> wrote:
> On Sun, Feb 5, 2012 at 5:33 AM, Benoit Chesneau <bc...@gmail.com> wrote:
>> Hi,
>>
>> Last day we had a quick discussion on IRC about splitting code in
>> different apps or not. I wasn't totally crystal clear about my position
>> and I would like to clarify it a little. Imo everything turn around the
>> question , "how do you envision Apache CouchDB".
>>
>> It appears for me that some wants to distribute it as a full product,
>> everything is embedded in one archive. In that case the distributiuon
>> system is only targeting this goal and nothing in the core allows
>> alternatives.
>>
>> While I'm agree we should offer a "canonical" release of Apache CouchDB,
>> I'm thinking more Apache CouchDB as a collections of modules with a
>> core. For example CouchDB could be:
>>
>> a k/v server that can live on one or more machine : the core
>> with an HTTP api
>> with a M/R api also accessible with HTT
>> with a _changes API
>> with a replicaton API
>> with a couchapp engine
>> with javascript support
>>
>> So anyone can eventually replace, remove or add a module in its own
>> couchdb distribution. The release system could also be enough smart to
>> allws a distributor to customize its own couchdb.
>>
>> What do you think about it? How do you envision CouchDB ?
>>
>> - benoîr
>
> I think there are a couple issues here that are intertwined that need not be.
>
> First, I don't think that having code in multiple apps is necessarily
> incompatible with having the code in a single repository. BigCouch was
> split out into multiple repositories with one Git repo per app and I
> can't say that I'm a fan. There are definitely arguments for it, but I
> find coordinating merges and pull requests across multiple git repos
> as well as the extra difficulty in preparing single tarball source
> releases to be a bigger negative than the nominal benefit of making
> sure changes to apps are in separate commits (and the perceived
> benefit that this forces less coupling).
>
> Secondly, the idea of allowing users/packagers to customize what
> modules are included in the "canonical" CouchDB seems like its just
> going to open up a world of hurt. If part of the promise of CouchDB is
> "run anywhere" then having things out there called "CouchDB" that
> don't include basic functionality like m/r views and _changes feeds is
> going to cause a lot of breakage.
>
> That said, I think that there definitely is a use case for making
> CouchDB more modular such that advanced Erlang uses are possible. I
> think we should have our code modularized internally so that it's
> possible (and much much easier) for Erlangers to pull just a KV core
> if they want. The distinction here is that people that just want to
> pull specific modules don't call their end result "CouchDB". Now, this
> may require some effort to integrate directly into tools like rebar,
> but I view this as a secondary goal that shouldn't interfere with our
> primary mission of releasing "CouchDB".
>
> Or, more succinctly, "CouchDB" should be what is built from our source
> release/repo but we should make it possible for people to reuse parts
> of CouchDB (as long as they don't call their end result CouchDB to
> avoid user confusion).

Re: modular or monolithic: how do you envision Apache CouchDB?

[Paul Davis <pa...@gmail.com>]

On Sun, Feb 5, 2012 at 5:33 AM, Benoit Chesneau <bc...@gmail.com> wrote:
> Hi,
>
> Last day we had a quick discussion on IRC about splitting code in
> different apps or not. I wasn't totally crystal clear about my position
> and I would like to clarify it a little. Imo everything turn around the
> question , "how do you envision Apache CouchDB".
>
> It appears for me that some wants to distribute it as a full product,
> everything is embedded in one archive. In that case the distributiuon
> system is only targeting this goal and nothing in the core allows
> alternatives.
>
> While I'm agree we should offer a "canonical" release of Apache CouchDB,
> I'm thinking more Apache CouchDB as a collections of modules with a
> core. For example CouchDB could be:
>
> a k/v server that can live on one or more machine : the core
> with an HTTP api
> with a M/R api also accessible with HTT
> with a _changes API
> with a replicaton API
> with a couchapp engine
> with javascript support
>
> So anyone can eventually replace, remove or add a module in its own
> couchdb distribution. The release system could also be enough smart to
> allws a distributor to customize its own couchdb.
>
> What do you think about it? How do you envision CouchDB ?
>
> - benoîr

I think there are a couple issues here that are intertwined that need not be.

First, I don't think that having code in multiple apps is necessarily
incompatible with having the code in a single repository. BigCouch was
split out into multiple repositories with one Git repo per app and I
can't say that I'm a fan. There are definitely arguments for it, but I
find coordinating merges and pull requests across multiple git repos
as well as the extra difficulty in preparing single tarball source
releases to be a bigger negative than the nominal benefit of making
sure changes to apps are in separate commits (and the perceived
benefit that this forces less coupling).

Secondly, the idea of allowing users/packagers to customize what
modules are included in the "canonical" CouchDB seems like its just
going to open up a world of hurt. If part of the promise of CouchDB is
"run anywhere" then having things out there called "CouchDB" that
don't include basic functionality like m/r views and _changes feeds is
going to cause a lot of breakage.

That said, I think that there definitely is a use case for making
CouchDB more modular such that advanced Erlang uses are possible. I
think we should have our code modularized internally so that it's
possible (and much much easier) for Erlangers to pull just a KV core
if they want. The distinction here is that people that just want to
pull specific modules don't call their end result "CouchDB". Now, this
may require some effort to integrate directly into tools like rebar,
but I view this as a secondary goal that shouldn't interfere with our
primary mission of releasing "CouchDB".

Or, more succinctly, "CouchDB" should be what is built from our source
release/repo but we should make it possible for people to reuse parts
of CouchDB (as long as they don't call their end result CouchDB to
avoid user confusion).