Contribution Process before PR

[Thomas Weise <th...@apache.org>]

Hi,

I want to propose additions to the contributor guidelines that place
stronger emphasis on open collaboration and the early part of the
contribution process.

Specifically, I would like to suggest that *thought process* and *design
discussion* are more important than the final code produced. It is
necessary to develop the community and invest in the future of the project.

I start this discussion based on observation over time. I have seen cases
(non trivial changes) where code and JIRAs appear at the same time, where
the big picture is discussed after the PR is already open, or where
information that would be valuable to other contributors or users isn't on
record.

Let's consider a non-trivial change or a feature. It would normally start
with engagement on the mailing list to ensure time is well spent and the
proposal is welcomed by the community, does not conflict with other
initiatives etc.

Once that is cleared, we would want to think about design, the how in the
larger picture. In many cases that would involve discussion, questions,
suggestions, consensus building towards agreed approach. Or maybe it is
done through prototyping. In any case, before a PR is raised, it will be
good to have as prerequisite that *thought process and approach have been
documented*. I would prefer to see that on the JIRA, what do others think?

Benefits:

* Contributor does not waste time and there is no frustration due to a PR
being turned down for reasons that could be avoided with upfront
communication.

* Contributor benefits from suggestions, questions, guidance of those with
in depth knowledge of particular areas.

* Other community members have an opportunity to learn from discussion, the
knowledge base broadens.

* Information gets indexed, user later looking at JIRAs will find valuable
information on how certain problems were solved that they would never
obtain from a PR.

The ASF and "Apache Way", a read for the bigger picture with more links in
it:
http://krzysztof-sobkowiak.net/blog/celebrating-17-years-of-the-apache-software-foundation/

Looking forward to feedback and discussion,
Thomas

Re: Contribution Process before PR

[Chinmay Kolhatkar <ch...@apache.org>]

I agree that the discussions are important. There are currently discussions
being happening over mailing list but some topics might be get missed and
we should improve there. The tasks should be brought to community before
the design starts and then mature the design through community discussion.
Though there are few of my observations that I would like to point out:

1. If I understand correctly, any development happening should still be
bound by a time. This includes the discussions over mailing list OR on the
PR as well. If the design discussion are happening over PR, then it means
the mailing list communication is failing somewhere.

2. Moreover, the nature of discussion should be conclusive. There are some
discussions which keeps happening over and over and takes time to reach a
conclusion. IMO this is blocking the development and does not benefit
anyone.

3. There are mailing threads (few but important ones) which have as much as
30+ email exchanges. After a point of time, I believe the crux of the topic
is somewhere missed. If an email thread is going beyond 10 mails to reach a
conclusion, then there is something wrong in way the discussion is heading
and its is the responsibility of everyone and not just original author to
"hold the thought" and re-align the discussion to reach a conclusion and
consensus.

4. At the same time, there are possibly cases where PR comments gets a
shape of design discussion because the original thread is not followed up
completely. Everyone has enough to catch up hence one might miss following
a certain mail thread but might point of some crucial points in PR comment.
In such case, I think there should be an alternate way (maybe an offline
call for author to clear the idea) might help. This is just to make sure
that the progress is made in current task and should not be followed as an
standard practice.

5. We as community should emphasis on the fact that the quality code is
generated but also should be mindful if time it is taking. Its hard to get
the right balance, hence it might be a good practice to break things down
to phases instead of trying to do in a single go. In such case, the
community should be highlighted that what is the first phase, second phase
and so on....


On Tue, Jan 17, 2017 at 12:04 PM, Bhupesh Chawda <bh...@datatorrent.com>
wrote:

> I agree.
> The discussion on requirements very much helps folks to understand the need
> for a feature and sometimes much more about related topics. From my own
> experience, the discussion thread on requirements before the implementation
> is what helps the design discussions and allows even people not aware of
> the internals, to quickly pickup things with little effort.
>
> At the same time, I do feel the need for a prototype implementation to
> clarify things, most of the times. But, I think this should be
> supplementary to the discussions and not a replacement.
>
> ~ Bhupesh
>
> On Tue, Jan 17, 2017 at 7:50 AM, Sandesh Hegde <sa...@datatorrent.com>
> wrote:
>
> > I do see value in having the review PR along with the discussion in
> > the mailing list/jira.
> >
> > It shows the commitment of the Author to the idea and to the project.
> > There is some truth in what Linus Torvalds says
> > https://lkml.org/lkml/2000/8/25/132
> >
> > Thanks
> >
> >
> > On Mon, Jan 16, 2017 at 8:11 AM Amol Kekre <am...@datatorrent.com> wrote:
> >
> > > I do see folks discussing issues for most part now. For me this does
> not
> > > look like an issue that is hurting Apex community. I do however want to
> > > discuss cases where code gets blocked and get spilled into larger
> > context.
> > > As a culture and a process, we have a very high overhead that deters
> > > contributions.
> > >
> > > Thks
> > > Amol
> > >
> > >
> > > On Sun, Jan 15, 2017 at 8:50 PM, Pramod Immaneni <
> pramod@datatorrent.com
> > >
> > > wrote:
> > >
> > > > Yes, it will be good to have these points added to the contributor
> > > > guidelines but I also see for the most part folks do bring up issues
> > for
> > > > discussion, try to address concerns and come to a consensus and in
> > > > generally participate in the community. I also think we should have
> > some
> > > > latitude in the process when it comes to bug fixes that are contained
> > and
> > > > don't spill into re-design of components otherwise, the overhead will
> > > deter
> > > > contributions especially from folks who are new to the project and
> want
> > > to
> > > > start contributing by fixing low hanging bugs.
> > > >
> > > > Thanks
> > > >
> > > > On Sun, Jan 15, 2017 at 7:50 PM, Thomas Weise <th...@apache.org>
> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I want to propose additions to the contributor guidelines that
> place
> > > > > stronger emphasis on open collaboration and the early part of the
> > > > > contribution process.
> > > > >
> > > > > Specifically, I would like to suggest that *thought process* and
> > > *design
> > > > > discussion* are more important than the final code produced. It is
> > > > > necessary to develop the community and invest in the future of the
> > > > project.
> > > > >
> > > > > I start this discussion based on observation over time. I have seen
> > > cases
> > > > > (non trivial changes) where code and JIRAs appear at the same time,
> > > where
> > > > > the big picture is discussed after the PR is already open, or where
> > > > > information that would be valuable to other contributors or users
> > isn't
> > > > on
> > > > > record.
> > > > >
> > > > > Let's consider a non-trivial change or a feature. It would normally
> > > start
> > > > > with engagement on the mailing list to ensure time is well spent
> and
> > > the
> > > > > proposal is welcomed by the community, does not conflict with other
> > > > > initiatives etc.
> > > > >
> > > > > Once that is cleared, we would want to think about design, the how
> in
> > > the
> > > > > larger picture. In many cases that would involve discussion,
> > questions,
> > > > > suggestions, consensus building towards agreed approach. Or maybe
> it
> > is
> > > > > done through prototyping. In any case, before a PR is raised, it
> will
> > > be
> > > > > good to have as prerequisite that *thought process and approach
> have
> > > been
> > > > > documented*. I would prefer to see that on the JIRA, what do others
> > > > think?
> > > > >
> > > > > Benefits:
> > > > >
> > > > > * Contributor does not waste time and there is no frustration due
> to
> > a
> > > PR
> > > > > being turned down for reasons that could be avoided with upfront
> > > > > communication.
> > > > >
> > > > > * Contributor benefits from suggestions, questions, guidance of
> those
> > > > with
> > > > > in depth knowledge of particular areas.
> > > > >
> > > > > * Other community members have an opportunity to learn from
> > discussion,
> > > > the
> > > > > knowledge base broadens.
> > > > >
> > > > > * Information gets indexed, user later looking at JIRAs will find
> > > > valuable
> > > > > information on how certain problems were solved that they would
> never
> > > > > obtain from a PR.
> > > > >
> > > > > The ASF and "Apache Way", a read for the bigger picture with more
> > links
> > > > in
> > > > > it:
> > > > > http://krzysztof-sobkowiak.net/blog/celebrating-17-years-
> > > > > of-the-apache-software-foundation/
> > > > >
> > > > > Looking forward to feedback and discussion,
> > > > > Thomas
> > > > >
> > > >
> > >
> >
>

Re: Contribution Process before PR

[Bhupesh Chawda <bh...@datatorrent.com>]

I agree.
The discussion on requirements very much helps folks to understand the need
for a feature and sometimes much more about related topics. From my own
experience, the discussion thread on requirements before the implementation
is what helps the design discussions and allows even people not aware of
the internals, to quickly pickup things with little effort.

At the same time, I do feel the need for a prototype implementation to
clarify things, most of the times. But, I think this should be
supplementary to the discussions and not a replacement.

~ Bhupesh

On Tue, Jan 17, 2017 at 7:50 AM, Sandesh Hegde <sa...@datatorrent.com>
wrote:

> I do see value in having the review PR along with the discussion in
> the mailing list/jira.
>
> It shows the commitment of the Author to the idea and to the project.
> There is some truth in what Linus Torvalds says
> https://lkml.org/lkml/2000/8/25/132
>
> Thanks
>
>
> On Mon, Jan 16, 2017 at 8:11 AM Amol Kekre <am...@datatorrent.com> wrote:
>
> > I do see folks discussing issues for most part now. For me this does not
> > look like an issue that is hurting Apex community. I do however want to
> > discuss cases where code gets blocked and get spilled into larger
> context.
> > As a culture and a process, we have a very high overhead that deters
> > contributions.
> >
> > Thks
> > Amol
> >
> >
> > On Sun, Jan 15, 2017 at 8:50 PM, Pramod Immaneni <pramod@datatorrent.com
> >
> > wrote:
> >
> > > Yes, it will be good to have these points added to the contributor
> > > guidelines but I also see for the most part folks do bring up issues
> for
> > > discussion, try to address concerns and come to a consensus and in
> > > generally participate in the community. I also think we should have
> some
> > > latitude in the process when it comes to bug fixes that are contained
> and
> > > don't spill into re-design of components otherwise, the overhead will
> > deter
> > > contributions especially from folks who are new to the project and want
> > to
> > > start contributing by fixing low hanging bugs.
> > >
> > > Thanks
> > >
> > > On Sun, Jan 15, 2017 at 7:50 PM, Thomas Weise <th...@apache.org> wrote:
> > >
> > > > Hi,
> > > >
> > > > I want to propose additions to the contributor guidelines that place
> > > > stronger emphasis on open collaboration and the early part of the
> > > > contribution process.
> > > >
> > > > Specifically, I would like to suggest that *thought process* and
> > *design
> > > > discussion* are more important than the final code produced. It is
> > > > necessary to develop the community and invest in the future of the
> > > project.
> > > >
> > > > I start this discussion based on observation over time. I have seen
> > cases
> > > > (non trivial changes) where code and JIRAs appear at the same time,
> > where
> > > > the big picture is discussed after the PR is already open, or where
> > > > information that would be valuable to other contributors or users
> isn't
> > > on
> > > > record.
> > > >
> > > > Let's consider a non-trivial change or a feature. It would normally
> > start
> > > > with engagement on the mailing list to ensure time is well spent and
> > the
> > > > proposal is welcomed by the community, does not conflict with other
> > > > initiatives etc.
> > > >
> > > > Once that is cleared, we would want to think about design, the how in
> > the
> > > > larger picture. In many cases that would involve discussion,
> questions,
> > > > suggestions, consensus building towards agreed approach. Or maybe it
> is
> > > > done through prototyping. In any case, before a PR is raised, it will
> > be
> > > > good to have as prerequisite that *thought process and approach have
> > been
> > > > documented*. I would prefer to see that on the JIRA, what do others
> > > think?
> > > >
> > > > Benefits:
> > > >
> > > > * Contributor does not waste time and there is no frustration due to
> a
> > PR
> > > > being turned down for reasons that could be avoided with upfront
> > > > communication.
> > > >
> > > > * Contributor benefits from suggestions, questions, guidance of those
> > > with
> > > > in depth knowledge of particular areas.
> > > >
> > > > * Other community members have an opportunity to learn from
> discussion,
> > > the
> > > > knowledge base broadens.
> > > >
> > > > * Information gets indexed, user later looking at JIRAs will find
> > > valuable
> > > > information on how certain problems were solved that they would never
> > > > obtain from a PR.
> > > >
> > > > The ASF and "Apache Way", a read for the bigger picture with more
> links
> > > in
> > > > it:
> > > > http://krzysztof-sobkowiak.net/blog/celebrating-17-years-
> > > > of-the-apache-software-foundation/
> > > >
> > > > Looking forward to feedback and discussion,
> > > > Thomas
> > > >
> > >
> >
>

Re: Contribution Process before PR

[Sandesh Hegde <sa...@datatorrent.com>]

I do see value in having the review PR along with the discussion in
the mailing list/jira.

It shows the commitment of the Author to the idea and to the project.
There is some truth in what Linus Torvalds says
https://lkml.org/lkml/2000/8/25/132

Thanks


On Mon, Jan 16, 2017 at 8:11 AM Amol Kekre <am...@datatorrent.com> wrote:

> I do see folks discussing issues for most part now. For me this does not
> look like an issue that is hurting Apex community. I do however want to
> discuss cases where code gets blocked and get spilled into larger context.
> As a culture and a process, we have a very high overhead that deters
> contributions.
>
> Thks
> Amol
>
>
> On Sun, Jan 15, 2017 at 8:50 PM, Pramod Immaneni <pr...@datatorrent.com>
> wrote:
>
> > Yes, it will be good to have these points added to the contributor
> > guidelines but I also see for the most part folks do bring up issues for
> > discussion, try to address concerns and come to a consensus and in
> > generally participate in the community. I also think we should have some
> > latitude in the process when it comes to bug fixes that are contained and
> > don't spill into re-design of components otherwise, the overhead will
> deter
> > contributions especially from folks who are new to the project and want
> to
> > start contributing by fixing low hanging bugs.
> >
> > Thanks
> >
> > On Sun, Jan 15, 2017 at 7:50 PM, Thomas Weise <th...@apache.org> wrote:
> >
> > > Hi,
> > >
> > > I want to propose additions to the contributor guidelines that place
> > > stronger emphasis on open collaboration and the early part of the
> > > contribution process.
> > >
> > > Specifically, I would like to suggest that *thought process* and
> *design
> > > discussion* are more important than the final code produced. It is
> > > necessary to develop the community and invest in the future of the
> > project.
> > >
> > > I start this discussion based on observation over time. I have seen
> cases
> > > (non trivial changes) where code and JIRAs appear at the same time,
> where
> > > the big picture is discussed after the PR is already open, or where
> > > information that would be valuable to other contributors or users isn't
> > on
> > > record.
> > >
> > > Let's consider a non-trivial change or a feature. It would normally
> start
> > > with engagement on the mailing list to ensure time is well spent and
> the
> > > proposal is welcomed by the community, does not conflict with other
> > > initiatives etc.
> > >
> > > Once that is cleared, we would want to think about design, the how in
> the
> > > larger picture. In many cases that would involve discussion, questions,
> > > suggestions, consensus building towards agreed approach. Or maybe it is
> > > done through prototyping. In any case, before a PR is raised, it will
> be
> > > good to have as prerequisite that *thought process and approach have
> been
> > > documented*. I would prefer to see that on the JIRA, what do others
> > think?
> > >
> > > Benefits:
> > >
> > > * Contributor does not waste time and there is no frustration due to a
> PR
> > > being turned down for reasons that could be avoided with upfront
> > > communication.
> > >
> > > * Contributor benefits from suggestions, questions, guidance of those
> > with
> > > in depth knowledge of particular areas.
> > >
> > > * Other community members have an opportunity to learn from discussion,
> > the
> > > knowledge base broadens.
> > >
> > > * Information gets indexed, user later looking at JIRAs will find
> > valuable
> > > information on how certain problems were solved that they would never
> > > obtain from a PR.
> > >
> > > The ASF and "Apache Way", a read for the bigger picture with more links
> > in
> > > it:
> > > http://krzysztof-sobkowiak.net/blog/celebrating-17-years-
> > > of-the-apache-software-foundation/
> > >
> > > Looking forward to feedback and discussion,
> > > Thomas
> > >
> >
>

Re: Contribution Process before PR

[Amol Kekre <am...@datatorrent.com>]

I do see folks discussing issues for most part now. For me this does not
look like an issue that is hurting Apex community. I do however want to
discuss cases where code gets blocked and get spilled into larger context.
As a culture and a process, we have a very high overhead that deters
contributions.

Thks
Amol


On Sun, Jan 15, 2017 at 8:50 PM, Pramod Immaneni <pr...@datatorrent.com>
wrote:

> Yes, it will be good to have these points added to the contributor
> guidelines but I also see for the most part folks do bring up issues for
> discussion, try to address concerns and come to a consensus and in
> generally participate in the community. I also think we should have some
> latitude in the process when it comes to bug fixes that are contained and
> don't spill into re-design of components otherwise, the overhead will deter
> contributions especially from folks who are new to the project and want to
> start contributing by fixing low hanging bugs.
>
> Thanks
>
> On Sun, Jan 15, 2017 at 7:50 PM, Thomas Weise <th...@apache.org> wrote:
>
> > Hi,
> >
> > I want to propose additions to the contributor guidelines that place
> > stronger emphasis on open collaboration and the early part of the
> > contribution process.
> >
> > Specifically, I would like to suggest that *thought process* and *design
> > discussion* are more important than the final code produced. It is
> > necessary to develop the community and invest in the future of the
> project.
> >
> > I start this discussion based on observation over time. I have seen cases
> > (non trivial changes) where code and JIRAs appear at the same time, where
> > the big picture is discussed after the PR is already open, or where
> > information that would be valuable to other contributors or users isn't
> on
> > record.
> >
> > Let's consider a non-trivial change or a feature. It would normally start
> > with engagement on the mailing list to ensure time is well spent and the
> > proposal is welcomed by the community, does not conflict with other
> > initiatives etc.
> >
> > Once that is cleared, we would want to think about design, the how in the
> > larger picture. In many cases that would involve discussion, questions,
> > suggestions, consensus building towards agreed approach. Or maybe it is
> > done through prototyping. In any case, before a PR is raised, it will be
> > good to have as prerequisite that *thought process and approach have been
> > documented*. I would prefer to see that on the JIRA, what do others
> think?
> >
> > Benefits:
> >
> > * Contributor does not waste time and there is no frustration due to a PR
> > being turned down for reasons that could be avoided with upfront
> > communication.
> >
> > * Contributor benefits from suggestions, questions, guidance of those
> with
> > in depth knowledge of particular areas.
> >
> > * Other community members have an opportunity to learn from discussion,
> the
> > knowledge base broadens.
> >
> > * Information gets indexed, user later looking at JIRAs will find
> valuable
> > information on how certain problems were solved that they would never
> > obtain from a PR.
> >
> > The ASF and "Apache Way", a read for the bigger picture with more links
> in
> > it:
> > http://krzysztof-sobkowiak.net/blog/celebrating-17-years-
> > of-the-apache-software-foundation/
> >
> > Looking forward to feedback and discussion,
> > Thomas
> >
>

Re: Contribution Process before PR

[Pramod Immaneni <pr...@datatorrent.com>]

Yes, it will be good to have these points added to the contributor
guidelines but I also see for the most part folks do bring up issues for
discussion, try to address concerns and come to a consensus and in
generally participate in the community. I also think we should have some
latitude in the process when it comes to bug fixes that are contained and
don't spill into re-design of components otherwise, the overhead will deter
contributions especially from folks who are new to the project and want to
start contributing by fixing low hanging bugs.

Thanks

On Sun, Jan 15, 2017 at 7:50 PM, Thomas Weise <th...@apache.org> wrote:

> Hi,
>
> I want to propose additions to the contributor guidelines that place
> stronger emphasis on open collaboration and the early part of the
> contribution process.
>
> Specifically, I would like to suggest that *thought process* and *design
> discussion* are more important than the final code produced. It is
> necessary to develop the community and invest in the future of the project.
>
> I start this discussion based on observation over time. I have seen cases
> (non trivial changes) where code and JIRAs appear at the same time, where
> the big picture is discussed after the PR is already open, or where
> information that would be valuable to other contributors or users isn't on
> record.
>
> Let's consider a non-trivial change or a feature. It would normally start
> with engagement on the mailing list to ensure time is well spent and the
> proposal is welcomed by the community, does not conflict with other
> initiatives etc.
>
> Once that is cleared, we would want to think about design, the how in the
> larger picture. In many cases that would involve discussion, questions,
> suggestions, consensus building towards agreed approach. Or maybe it is
> done through prototyping. In any case, before a PR is raised, it will be
> good to have as prerequisite that *thought process and approach have been
> documented*. I would prefer to see that on the JIRA, what do others think?
>
> Benefits:
>
> * Contributor does not waste time and there is no frustration due to a PR
> being turned down for reasons that could be avoided with upfront
> communication.
>
> * Contributor benefits from suggestions, questions, guidance of those with
> in depth knowledge of particular areas.
>
> * Other community members have an opportunity to learn from discussion, the
> knowledge base broadens.
>
> * Information gets indexed, user later looking at JIRAs will find valuable
> information on how certain problems were solved that they would never
> obtain from a PR.
>
> The ASF and "Apache Way", a read for the bigger picture with more links in
> it:
> http://krzysztof-sobkowiak.net/blog/celebrating-17-years-
> of-the-apache-software-foundation/
>
> Looking forward to feedback and discussion,
> Thomas
>

Re: Contribution Process before PR

[Munagala Ramanath <ra...@datatorrent.com>]

Yes, I think an initial discussion that includes some or all of the
following
would be invaluable both for feature implementations and for bug fixes:
1. Discussion of current implementation and how/why it fails to meet
current need.
2. Possible approaches and tradeoffs (if any) for each.
3. Recommended best approach if author has reached such a conclusion.

However, some contributors may want to throw in a prototype implementation
as a
PR as part of the discussion since code has a clarity and precision that,
ultimately, cannot be matched by natural language. We should allow and even
encourage this since it helps, in many cases, to avoid a long chain of Q&A.
It also helps others to try out working code to make their own assessment
of its strengths and weaknesses. Additionally, it helps contributors whose
English may not be as strong as their coding skills, to participate in the
discussion.

In such cases, it should be clearly understood that the PR is potentially
throwaway code, intended to further the discussion and not necessarily
intended to be merged.

Ram

On Mon, Jan 16, 2017 at 9:20 AM, Thomas Weise <th...@apache.org> wrote:

> Hi,
>
> I want to propose additions to the contributor guidelines that place
> stronger emphasis on open collaboration and the early part of the
> contribution process.
>
> Specifically, I would like to suggest that *thought process* and *design
> discussion* are more important than the final code produced. It is
> necessary to develop the community and invest in the future of the project.
>
> I start this discussion based on observation over time. I have seen cases
> (non trivial changes) where code and JIRAs appear at the same time, where
> the big picture is discussed after the PR is already open, or where
> information that would be valuable to other contributors or users isn't on
> record.
>
> Let's consider a non-trivial change or a feature. It would normally start
> with engagement on the mailing list to ensure time is well spent and the
> proposal is welcomed by the community, does not conflict with other
> initiatives etc.
>
> Once that is cleared, we would want to think about design, the how in the
> larger picture. In many cases that would involve discussion, questions,
> suggestions, consensus building towards agreed approach. Or maybe it is
> done through prototyping. In any case, before a PR is raised, it will be
> good to have as prerequisite that *thought process and approach have been
> documented*. I would prefer to see that on the JIRA, what do others think?
>
> Benefits:
>
> * Contributor does not waste time and there is no frustration due to a PR
> being turned down for reasons that could be avoided with upfront
> communication.
>
> * Contributor benefits from suggestions, questions, guidance of those with
> in depth knowledge of particular areas.
>
> * Other community members have an opportunity to learn from discussion, the
> knowledge base broadens.
>
> * Information gets indexed, user later looking at JIRAs will find valuable
> information on how certain problems were solved that they would never
> obtain from a PR.
>
> The ASF and "Apache Way", a read for the bigger picture with more links in
> it:
> http://krzysztof-sobkowiak.net/blog/celebrating-17-years-
> of-the-apache-software-foundation/
>
> Looking forward to feedback and discussion,
> Thomas
>