merge, where is the doc?

[Benoit Chesneau <bc...@gmail.com>]

Sometimes in the past we blamed some people because they were committing
big chunks of code not atomically and not really documented. And I am
afraid this is happening right now again.

This is not the first time I asked for it but could the merge be more
documented? Commits message are not enough and having to pass 1 day or more
to follow all the changes, why they are done, what are their intents, is
not pleasant/fun at all. More important I find myself in waiting everything
is finished to know what can I do with it. And I don't know where we are
going exactly (just a vague idea of what we will have). Some discussion
with others outside about it shows I am not alone.

Don't get me wrong, I like to see the source code moving. But I think it's
more important to integrate others developers (and not only committers) in
the loop when a change happen. And we shouldn't wait that a branch is
finished before knowing where we are going. We shouldn't have to dive in
the source code of the current master to understand how each modules
interacts, understanding the way decisions are done on the cluster, .....
Such things should already be documented before to go further. So other
devs can again help if they want to, without losing their times too much in
the basement. The knowledge should be shared with the community.

I am sure all these chunks of code have been discussed before between
people inside Cloudant, that some design documents have been exchanged, and
some tickets describe more exactly the solved problem. If people don't have
the time to document the wip, maybe at least such material can be shared?

I hope this situation may be sorted soon.

- benoit

Re: merge, where is the doc?

[Andy Wenk <an...@apache.org>]

+1 for putting davisp' whirle wind tour into the docs repo ....


On 17 August 2014 12:51, Robert Samuel Newson <rn...@apache.org> wrote:

> I’ve no objection to putting that doc from davisp into the documentation
> repo. I was referring to other docs like INSTALL and so on. A rough
> structural overview is a fine start.
>
> B.
>
> On 16 Aug 2014, at 22:59, Robert Samuel Newson <rn...@apache.org> wrote:
>
> > I agree we need docs, I ask that you bear with us while we get the code
> working first.
> >
> > B.
> >
> > On 16 Aug 2014, at 22:47, Dirkjan Ochtman <di...@ochtman.nl> wrote:
> >
> >> On Fri, Aug 15, 2014 at 3:26 PM, Alexander Shorin <kx...@gmail.com>
> wrote:
> >>> On Fri, Aug 15, 2014 at 6:21 PM, Paul Davis <
> paul.joseph.davis@gmail.com> wrote:
> >>>> Here's a first pass at things. I'm not entirely sure where this sort
> >>>> of thing should go in the source tree, suggestions welcome.
> >>>>
> >>>> https://gist.github.com/davisp/4013dd3880d3c4ffc600
> >>>
> >>> This can go to docs. I have a plans for developers section which
> >>> exactly have to describe our apps, how they works, their api etc.
> >>> Bookmarked for future usage, thanks for sharing!
> >>
> >> I think it would be great to put this into the docs tree as a rough
> >> cut for now, and have it be iteratively improved later on.
> >>
> >> Cheers,
> >>
> >> Dirkjan
> >
>
>


-- 
Andy Wenk
Hamburg - Germany
RockIt!

GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3 9588

 https://people.apache.org/keys/committer/andywenk.asc

Re: merge, where is the doc?

[Robert Samuel Newson <rn...@apache.org>]

I’ve no objection to putting that doc from davisp into the documentation repo. I was referring to other docs like INSTALL and so on. A rough structural overview is a fine start.

B.

On 16 Aug 2014, at 22:59, Robert Samuel Newson <rn...@apache.org> wrote:

> I agree we need docs, I ask that you bear with us while we get the code working first.
> 
> B.
> 
> On 16 Aug 2014, at 22:47, Dirkjan Ochtman <di...@ochtman.nl> wrote:
> 
>> On Fri, Aug 15, 2014 at 3:26 PM, Alexander Shorin <kx...@gmail.com> wrote:
>>> On Fri, Aug 15, 2014 at 6:21 PM, Paul Davis <pa...@gmail.com> wrote:
>>>> Here's a first pass at things. I'm not entirely sure where this sort
>>>> of thing should go in the source tree, suggestions welcome.
>>>> 
>>>> https://gist.github.com/davisp/4013dd3880d3c4ffc600
>>> 
>>> This can go to docs. I have a plans for developers section which
>>> exactly have to describe our apps, how they works, their api etc.
>>> Bookmarked for future usage, thanks for sharing!
>> 
>> I think it would be great to put this into the docs tree as a rough
>> cut for now, and have it be iteratively improved later on.
>> 
>> Cheers,
>> 
>> Dirkjan
> 

Re: merge, where is the doc?

[Robert Samuel Newson <rn...@apache.org>]

I agree we need docs, I ask that you bear with us while we get the code working first.

B.

On 16 Aug 2014, at 22:47, Dirkjan Ochtman <di...@ochtman.nl> wrote:

> On Fri, Aug 15, 2014 at 3:26 PM, Alexander Shorin <kx...@gmail.com> wrote:
>> On Fri, Aug 15, 2014 at 6:21 PM, Paul Davis <pa...@gmail.com> wrote:
>>> Here's a first pass at things. I'm not entirely sure where this sort
>>> of thing should go in the source tree, suggestions welcome.
>>> 
>>> https://gist.github.com/davisp/4013dd3880d3c4ffc600
>> 
>> This can go to docs. I have a plans for developers section which
>> exactly have to describe our apps, how they works, their api etc.
>> Bookmarked for future usage, thanks for sharing!
> 
> I think it would be great to put this into the docs tree as a rough
> cut for now, and have it be iteratively improved later on.
> 
> Cheers,
> 
> Dirkjan

Re: merge, where is the doc?

[Dirkjan Ochtman <di...@ochtman.nl>]

On Fri, Aug 15, 2014 at 3:26 PM, Alexander Shorin <kx...@gmail.com> wrote:
> On Fri, Aug 15, 2014 at 6:21 PM, Paul Davis <pa...@gmail.com> wrote:
>> Here's a first pass at things. I'm not entirely sure where this sort
>> of thing should go in the source tree, suggestions welcome.
>>
>> https://gist.github.com/davisp/4013dd3880d3c4ffc600
>
> This can go to docs. I have a plans for developers section which
> exactly have to describe our apps, how they works, their api etc.
> Bookmarked for future usage, thanks for sharing!

I think it would be great to put this into the docs tree as a rough
cut for now, and have it be iteratively improved later on.

Cheers,

Dirkjan

Re: merge, where is the doc?

[Alexander Shorin <kx...@gmail.com>]

On Fri, Aug 15, 2014 at 6:21 PM, Paul Davis <pa...@gmail.com> wrote:
> Here's a first pass at things. I'm not entirely sure where this sort
> of thing should go in the source tree, suggestions welcome.
>
> https://gist.github.com/davisp/4013dd3880d3c4ffc600

This can go to docs. I have a plans for developers section which
exactly have to describe our apps, how they works, their api etc.
Bookmarked for future usage, thanks for sharing!

--
,,,^..^,,,

Re: merge, where is the doc?

[Paul Davis <pa...@gmail.com>]

Here's a first pass at things. I'm not entirely sure where this sort
of thing should go in the source tree, suggestions welcome.

https://gist.github.com/davisp/4013dd3880d3c4ffc600

On Fri, Aug 15, 2014 at 7:44 AM, Paul Davis <pa...@gmail.com> wrote:
> On Fri, Aug 15, 2014 at 2:10 AM, Benoit Chesneau <bc...@gmail.com> wrote:
>> On Wed, Aug 13, 2014 at 7:19 PM, Paul Davis <pa...@gmail.com>
>> wrote:
>>
>>> Benoit,
>>>
>>> I'm not exactly sure what you're asking for here. As I read it, it
>>> sounds like you're wanting documentation both on the merge process
>>> itself and then documentation on all the various things the merge
>>> introduces. As to the merge process, there's really not much to
>>> document other than "import code as agreed, apply patches that were
>>> voted on, fix bugs". The applying of patches and bug fixes its what's
>>> been happening in the windsor-merge branches the last few weeks. If
>>> instead you're wanting documentation on all the things the merge
>>> introduces then you'll have to be more specific on which parts. I
>>> would be more than happy to write documentation for major portions of
>>> the clustering from an internal perspective. I agree that it would be
>>> quite helpful to others that are just starting to learn how this code
>>> works and fits together.
>>>
>>> Unfortunately there is no trove of documentation inside Cloudant.
>>> Historically most of our "design" happens over IRC or as code review.
>>> As Bob has said, we obviously can give unrestricted access to our
>>> ticketing system but if there are any patches that are curious we can
>>> go back and get the historical context/reasoning for changes that seem
>>> opaque. On the other hand you seem to be wanting a wider focus
>>> discussion on the various sectiosn of code and how they fit together
>>> of which there really isn't anything at all. But as I mentioned I'd be
>>> more than happy to sit down and write up anything you've got questions
>>> on.
>>>
>>> Let me know what I can do to help.
>>>
>>> Paul
>>>
>>>
>> Hi Paul,
>>
>> Thanks for your answer. To be more specific, what could be really useful
>> right now is the following:
>>
>> - layout of the new arch and how apps (and module eventually) interacts
>> - technical description of the consensus algorithm: it would be interesting
>> to have a general overview of the algorithms and how they are used on
>> update/read/query of views
>> - what is the API used to handle the cluster at the lowest level. Imo
>> providing a simple example "ala" riak core would help.
>> - document changes in records, macros and other stuffs done in a doc. A 1.0
>> -> 2.0 doc for devs in short/
>>
>> These points would help anyone that want to help on our code and would also
>> maybe interest the community in general. More eyes we have, the best it is.
>>
>> Among other things i really wish we could do right now is to document all
>> the public internal api we have, put in private others so we could
>> eventually generate edocs. Also having specs defined so we can do more
>> testing and know which data type could be used in a function. This is the
>> hardest to do, so if it could be done on new code it would save a lot of
>> time...
>>
>> For the rest, if full detail can't be given, at least maybe some commits
>> could provide more context. I had an example in mind sometimes ago but I
>> forget. One sure thing for example is that the why of adding couch_event,
>> couch_io could be more detailed in a README.
>>
>> Hope it's more cleat,
>>
>> - benoit
>
> Benoit,
>
> That's quite reasonable and something I think we should consider doing
> for all of CouchDB and not just the clustered bits. Some of that I can
> do more quickly than others obviously. I'll try and get a quick write
> up of each app and its main purpose and some of the important modules.
> Some of the other bits will have to wait till I have more time but
> hopefully a quick pass will at least be good enough to give an idea on
> how everything fits together both at the code level and on the
> behavior level.
>
> BigCouch has fabric which is the clustered API. Beyond that though I
> don't really think that CouchDB has a public API to document. I'd very
> much like to have an API to document though. :D
>
> I'll try and get something quick written up this morning to at least
> give some general directions on things. Will post back with anything I
> come up with.

Re: merge, where is the doc?

[Paul Davis <pa...@gmail.com>]

On Fri, Aug 15, 2014 at 2:10 AM, Benoit Chesneau <bc...@gmail.com> wrote:
> On Wed, Aug 13, 2014 at 7:19 PM, Paul Davis <pa...@gmail.com>
> wrote:
>
>> Benoit,
>>
>> I'm not exactly sure what you're asking for here. As I read it, it
>> sounds like you're wanting documentation both on the merge process
>> itself and then documentation on all the various things the merge
>> introduces. As to the merge process, there's really not much to
>> document other than "import code as agreed, apply patches that were
>> voted on, fix bugs". The applying of patches and bug fixes its what's
>> been happening in the windsor-merge branches the last few weeks. If
>> instead you're wanting documentation on all the things the merge
>> introduces then you'll have to be more specific on which parts. I
>> would be more than happy to write documentation for major portions of
>> the clustering from an internal perspective. I agree that it would be
>> quite helpful to others that are just starting to learn how this code
>> works and fits together.
>>
>> Unfortunately there is no trove of documentation inside Cloudant.
>> Historically most of our "design" happens over IRC or as code review.
>> As Bob has said, we obviously can give unrestricted access to our
>> ticketing system but if there are any patches that are curious we can
>> go back and get the historical context/reasoning for changes that seem
>> opaque. On the other hand you seem to be wanting a wider focus
>> discussion on the various sectiosn of code and how they fit together
>> of which there really isn't anything at all. But as I mentioned I'd be
>> more than happy to sit down and write up anything you've got questions
>> on.
>>
>> Let me know what I can do to help.
>>
>> Paul
>>
>>
> Hi Paul,
>
> Thanks for your answer. To be more specific, what could be really useful
> right now is the following:
>
> - layout of the new arch and how apps (and module eventually) interacts
> - technical description of the consensus algorithm: it would be interesting
> to have a general overview of the algorithms and how they are used on
> update/read/query of views
> - what is the API used to handle the cluster at the lowest level. Imo
> providing a simple example "ala" riak core would help.
> - document changes in records, macros and other stuffs done in a doc. A 1.0
> -> 2.0 doc for devs in short/
>
> These points would help anyone that want to help on our code and would also
> maybe interest the community in general. More eyes we have, the best it is.
>
> Among other things i really wish we could do right now is to document all
> the public internal api we have, put in private others so we could
> eventually generate edocs. Also having specs defined so we can do more
> testing and know which data type could be used in a function. This is the
> hardest to do, so if it could be done on new code it would save a lot of
> time...
>
> For the rest, if full detail can't be given, at least maybe some commits
> could provide more context. I had an example in mind sometimes ago but I
> forget. One sure thing for example is that the why of adding couch_event,
> couch_io could be more detailed in a README.
>
> Hope it's more cleat,
>
> - benoit

Benoit,

That's quite reasonable and something I think we should consider doing
for all of CouchDB and not just the clustered bits. Some of that I can
do more quickly than others obviously. I'll try and get a quick write
up of each app and its main purpose and some of the important modules.
Some of the other bits will have to wait till I have more time but
hopefully a quick pass will at least be good enough to give an idea on
how everything fits together both at the code level and on the
behavior level.

BigCouch has fabric which is the clustered API. Beyond that though I
don't really think that CouchDB has a public API to document. I'd very
much like to have an API to document though. :D

I'll try and get something quick written up this morning to at least
give some general directions on things. Will post back with anything I
come up with.

Re: merge, where is the doc?

[Benoit Chesneau <bc...@gmail.com>]

On Fri, Aug 15, 2014 at 11:46 AM, Robert Samuel Newson <rn...@apache.org>
wrote:

> I can quickly answer one question: there is no consensus algorithm.
> bigcouch / couchdb 2.0 is explicitly AP not CP.
>

Hrm OK, thanks for the info.

But, my question was more about an overview of the algorithms used to
balance the data over the shard, also rebalance (or whatever the purpose of
the `mem3_rebalance` module and other, how nodes are choosed to handle
shards and such things :) Anything that could help to have an overview and
possibly reuse the code to extend couch :)

- benoit

>
> B.
>
> On 15 Aug 2014, at 08:10, Benoit Chesneau <bc...@gmail.com> wrote:
>
> > On Wed, Aug 13, 2014 at 7:19 PM, Paul Davis <paul.joseph.davis@gmail.com
> >
> > wrote:
> >
> >> Benoit,
> >>
> >> I'm not exactly sure what you're asking for here. As I read it, it
> >> sounds like you're wanting documentation both on the merge process
> >> itself and then documentation on all the various things the merge
> >> introduces. As to the merge process, there's really not much to
> >> document other than "import code as agreed, apply patches that were
> >> voted on, fix bugs". The applying of patches and bug fixes its what's
> >> been happening in the windsor-merge branches the last few weeks. If
> >> instead you're wanting documentation on all the things the merge
> >> introduces then you'll have to be more specific on which parts. I
> >> would be more than happy to write documentation for major portions of
> >> the clustering from an internal perspective. I agree that it would be
> >> quite helpful to others that are just starting to learn how this code
> >> works and fits together.
> >>
> >> Unfortunately there is no trove of documentation inside Cloudant.
> >> Historically most of our "design" happens over IRC or as code review.
> >> As Bob has said, we obviously can give unrestricted access to our
> >> ticketing system but if there are any patches that are curious we can
> >> go back and get the historical context/reasoning for changes that seem
> >> opaque. On the other hand you seem to be wanting a wider focus
> >> discussion on the various sectiosn of code and how they fit together
> >> of which there really isn't anything at all. But as I mentioned I'd be
> >> more than happy to sit down and write up anything you've got questions
> >> on.
> >>
> >> Let me know what I can do to help.
> >>
> >> Paul
> >>
> >>
> > Hi Paul,
> >
> > Thanks for your answer. To be more specific, what could be really useful
> > right now is the following:
> >
> > - layout of the new arch and how apps (and module eventually) interacts
> > - technical description of the consensus algorithm: it would be
> interesting
> > to have a general overview of the algorithms and how they are used on
> > update/read/query of views
> > - what is the API used to handle the cluster at the lowest level. Imo
> > providing a simple example "ala" riak core would help.
> > - document changes in records, macros and other stuffs done in a doc. A
> 1.0
> > -> 2.0 doc for devs in short/
> >
> > These points would help anyone that want to help on our code and would
> also
> > maybe interest the community in general. More eyes we have, the best it
> is.
> >
> > Among other things i really wish we could do right now is to document all
> > the public internal api we have, put in private others so we could
> > eventually generate edocs. Also having specs defined so we can do more
> > testing and know which data type could be used in a function. This is the
> > hardest to do, so if it could be done on new code it would save a lot of
> > time...
> >
> > For the rest, if full detail can't be given, at least maybe some commits
> > could provide more context. I had an example in mind sometimes ago but I
> > forget. One sure thing for example is that the why of adding couch_event,
> > couch_io could be more detailed in a README.
> >
> > Hope it's more cleat,
> >
> > - benoit
>
>

Re: merge, where is the doc?

[Robert Samuel Newson <rn...@apache.org>]

I can quickly answer one question: there is no consensus algorithm. bigcouch / couchdb 2.0 is explicitly AP not CP.

B.

On 15 Aug 2014, at 08:10, Benoit Chesneau <bc...@gmail.com> wrote:

> On Wed, Aug 13, 2014 at 7:19 PM, Paul Davis <pa...@gmail.com>
> wrote:
> 
>> Benoit,
>> 
>> I'm not exactly sure what you're asking for here. As I read it, it
>> sounds like you're wanting documentation both on the merge process
>> itself and then documentation on all the various things the merge
>> introduces. As to the merge process, there's really not much to
>> document other than "import code as agreed, apply patches that were
>> voted on, fix bugs". The applying of patches and bug fixes its what's
>> been happening in the windsor-merge branches the last few weeks. If
>> instead you're wanting documentation on all the things the merge
>> introduces then you'll have to be more specific on which parts. I
>> would be more than happy to write documentation for major portions of
>> the clustering from an internal perspective. I agree that it would be
>> quite helpful to others that are just starting to learn how this code
>> works and fits together.
>> 
>> Unfortunately there is no trove of documentation inside Cloudant.
>> Historically most of our "design" happens over IRC or as code review.
>> As Bob has said, we obviously can give unrestricted access to our
>> ticketing system but if there are any patches that are curious we can
>> go back and get the historical context/reasoning for changes that seem
>> opaque. On the other hand you seem to be wanting a wider focus
>> discussion on the various sectiosn of code and how they fit together
>> of which there really isn't anything at all. But as I mentioned I'd be
>> more than happy to sit down and write up anything you've got questions
>> on.
>> 
>> Let me know what I can do to help.
>> 
>> Paul
>> 
>> 
> Hi Paul,
> 
> Thanks for your answer. To be more specific, what could be really useful
> right now is the following:
> 
> - layout of the new arch and how apps (and module eventually) interacts
> - technical description of the consensus algorithm: it would be interesting
> to have a general overview of the algorithms and how they are used on
> update/read/query of views
> - what is the API used to handle the cluster at the lowest level. Imo
> providing a simple example "ala" riak core would help.
> - document changes in records, macros and other stuffs done in a doc. A 1.0
> -> 2.0 doc for devs in short/
> 
> These points would help anyone that want to help on our code and would also
> maybe interest the community in general. More eyes we have, the best it is.
> 
> Among other things i really wish we could do right now is to document all
> the public internal api we have, put in private others so we could
> eventually generate edocs. Also having specs defined so we can do more
> testing and know which data type could be used in a function. This is the
> hardest to do, so if it could be done on new code it would save a lot of
> time...
> 
> For the rest, if full detail can't be given, at least maybe some commits
> could provide more context. I had an example in mind sometimes ago but I
> forget. One sure thing for example is that the why of adding couch_event,
> couch_io could be more detailed in a README.
> 
> Hope it's more cleat,
> 
> - benoit

Re: merge, where is the doc?

[Benoit Chesneau <bc...@gmail.com>]

On Wed, Aug 13, 2014 at 7:19 PM, Paul Davis <pa...@gmail.com>
wrote:

> Benoit,
>
> I'm not exactly sure what you're asking for here. As I read it, it
> sounds like you're wanting documentation both on the merge process
> itself and then documentation on all the various things the merge
> introduces. As to the merge process, there's really not much to
> document other than "import code as agreed, apply patches that were
> voted on, fix bugs". The applying of patches and bug fixes its what's
> been happening in the windsor-merge branches the last few weeks. If
> instead you're wanting documentation on all the things the merge
> introduces then you'll have to be more specific on which parts. I
> would be more than happy to write documentation for major portions of
> the clustering from an internal perspective. I agree that it would be
> quite helpful to others that are just starting to learn how this code
> works and fits together.
>
> Unfortunately there is no trove of documentation inside Cloudant.
> Historically most of our "design" happens over IRC or as code review.
> As Bob has said, we obviously can give unrestricted access to our
> ticketing system but if there are any patches that are curious we can
> go back and get the historical context/reasoning for changes that seem
> opaque. On the other hand you seem to be wanting a wider focus
> discussion on the various sectiosn of code and how they fit together
> of which there really isn't anything at all. But as I mentioned I'd be
> more than happy to sit down and write up anything you've got questions
> on.
>
> Let me know what I can do to help.
>
> Paul
>
>
Hi Paul,

Thanks for your answer. To be more specific, what could be really useful
right now is the following:

- layout of the new arch and how apps (and module eventually) interacts
- technical description of the consensus algorithm: it would be interesting
to have a general overview of the algorithms and how they are used on
update/read/query of views
- what is the API used to handle the cluster at the lowest level. Imo
providing a simple example "ala" riak core would help.
- document changes in records, macros and other stuffs done in a doc. A 1.0
-> 2.0 doc for devs in short/

These points would help anyone that want to help on our code and would also
maybe interest the community in general. More eyes we have, the best it is.

Among other things i really wish we could do right now is to document all
the public internal api we have, put in private others so we could
eventually generate edocs. Also having specs defined so we can do more
testing and know which data type could be used in a function. This is the
hardest to do, so if it could be done on new code it would save a lot of
time...

For the rest, if full detail can't be given, at least maybe some commits
could provide more context. I had an example in mind sometimes ago but I
forget. One sure thing for example is that the why of adding couch_event,
couch_io could be more detailed in a README.

Hope it's more cleat,

- benoit

Re: merge, where is the doc?

[Paul Davis <pa...@gmail.com>]

Derp. Should've been "obviously *can't* give unrestricted access".

On Wed, Aug 13, 2014 at 12:19 PM, Paul Davis
<pa...@gmail.com> wrote:
> Benoit,
>
> I'm not exactly sure what you're asking for here. As I read it, it
> sounds like you're wanting documentation both on the merge process
> itself and then documentation on all the various things the merge
> introduces. As to the merge process, there's really not much to
> document other than "import code as agreed, apply patches that were
> voted on, fix bugs". The applying of patches and bug fixes its what's
> been happening in the windsor-merge branches the last few weeks. If
> instead you're wanting documentation on all the things the merge
> introduces then you'll have to be more specific on which parts. I
> would be more than happy to write documentation for major portions of
> the clustering from an internal perspective. I agree that it would be
> quite helpful to others that are just starting to learn how this code
> works and fits together.
>
> Unfortunately there is no trove of documentation inside Cloudant.
> Historically most of our "design" happens over IRC or as code review.
> As Bob has said, we obviously can give unrestricted access to our
> ticketing system but if there are any patches that are curious we can
> go back and get the historical context/reasoning for changes that seem
> opaque. On the other hand you seem to be wanting a wider focus
> discussion on the various sectiosn of code and how they fit together
> of which there really isn't anything at all. But as I mentioned I'd be
> more than happy to sit down and write up anything you've got questions
> on.
>
> Let me know what I can do to help.
>
> Paul
>
> On Wed, Aug 13, 2014 at 1:00 AM, Benoit Chesneau <bc...@gmail.com> wrote:
>> Sometimes in the past we blamed some people because they were committing
>> big chunks of code not atomically and not really documented. And I am
>> afraid this is happening right now again.
>>
>> This is not the first time I asked for it but could the merge be more
>> documented? Commits message are not enough and having to pass 1 day or more
>> to follow all the changes, why they are done, what are their intents, is
>> not pleasant/fun at all. More important I find myself in waiting everything
>> is finished to know what can I do with it. And I don't know where we are
>> going exactly (just a vague idea of what we will have). Some discussion
>> with others outside about it shows I am not alone.
>>
>> Don't get me wrong, I like to see the source code moving. But I think it's
>> more important to integrate others developers (and not only committers) in
>> the loop when a change happen. And we shouldn't wait that a branch is
>> finished before knowing where we are going. We shouldn't have to dive in
>> the source code of the current master to understand how each modules
>> interacts, understanding the way decisions are done on the cluster, .....
>> Such things should already be documented before to go further. So other
>> devs can again help if they want to, without losing their times too much in
>> the basement. The knowledge should be shared with the community.
>>
>> I am sure all these chunks of code have been discussed before between
>> people inside Cloudant, that some design documents have been exchanged, and
>> some tickets describe more exactly the solved problem. If people don't have
>> the time to document the wip, maybe at least such material can be shared?
>>
>> I hope this situation may be sorted soon.
>>
>> - benoit

Re: merge, where is the doc?

[Paul Davis <pa...@gmail.com>]

Benoit,

I'm not exactly sure what you're asking for here. As I read it, it
sounds like you're wanting documentation both on the merge process
itself and then documentation on all the various things the merge
introduces. As to the merge process, there's really not much to
document other than "import code as agreed, apply patches that were
voted on, fix bugs". The applying of patches and bug fixes its what's
been happening in the windsor-merge branches the last few weeks. If
instead you're wanting documentation on all the things the merge
introduces then you'll have to be more specific on which parts. I
would be more than happy to write documentation for major portions of
the clustering from an internal perspective. I agree that it would be
quite helpful to others that are just starting to learn how this code
works and fits together.

Unfortunately there is no trove of documentation inside Cloudant.
Historically most of our "design" happens over IRC or as code review.
As Bob has said, we obviously can give unrestricted access to our
ticketing system but if there are any patches that are curious we can
go back and get the historical context/reasoning for changes that seem
opaque. On the other hand you seem to be wanting a wider focus
discussion on the various sectiosn of code and how they fit together
of which there really isn't anything at all. But as I mentioned I'd be
more than happy to sit down and write up anything you've got questions
on.

Let me know what I can do to help.

Paul

On Wed, Aug 13, 2014 at 1:00 AM, Benoit Chesneau <bc...@gmail.com> wrote:
> Sometimes in the past we blamed some people because they were committing
> big chunks of code not atomically and not really documented. And I am
> afraid this is happening right now again.
>
> This is not the first time I asked for it but could the merge be more
> documented? Commits message are not enough and having to pass 1 day or more
> to follow all the changes, why they are done, what are their intents, is
> not pleasant/fun at all. More important I find myself in waiting everything
> is finished to know what can I do with it. And I don't know where we are
> going exactly (just a vague idea of what we will have). Some discussion
> with others outside about it shows I am not alone.
>
> Don't get me wrong, I like to see the source code moving. But I think it's
> more important to integrate others developers (and not only committers) in
> the loop when a change happen. And we shouldn't wait that a branch is
> finished before knowing where we are going. We shouldn't have to dive in
> the source code of the current master to understand how each modules
> interacts, understanding the way decisions are done on the cluster, .....
> Such things should already be documented before to go further. So other
> devs can again help if they want to, without losing their times too much in
> the basement. The knowledge should be shared with the community.
>
> I am sure all these chunks of code have been discussed before between
> people inside Cloudant, that some design documents have been exchanged, and
> some tickets describe more exactly the solved problem. If people don't have
> the time to document the wip, maybe at least such material can be shared?
>
> I hope this situation may be sorted soon.
>
> - benoit