Vetting & Testing Contributor’s Code Changes

[Pierre Smits <pi...@apache.org>]

Hi all,

Currently we have Git (and GitHub) in full play for the master branch and
the 2 active release branches. The transition to this methodology of
distributed version control and collaboration has been adopted by our
fellow contributors without any major impediments and/or destructions. At
least none have been reported. It seems that everybody participating in
this project and submitting code changes has an experience with Git and/or
Github prior to the project’s move.

In the past (during the svn era) the only option for fellow contributors to
deliver code improvements for bug fixes and other enhancements was to
deliver a file to a ticket in our Jira.
Nowadays, the contributor can -besides still using that upload option- use
the option to submit a Pull Request from his private clone or public fork
to one of our official repositories on Github.
In that past, and still to date, any code contribution uploaded to a Jira
ticket needed to be downloaded by a committer and applied by him to a test
branch (based on either the master or the active release branches) in order
to vet the code changes and test the code changes whether these would break
(or integrate with) the codebase. This vetting and testing was and is an
arduous and  time consuming task, especially when the handed code changes
span multiple components, languages and/or files. The committer must vet
the changes with in his mind:

   - does it have unnecessary whitespacing
   - does it comply with checkstyle
   - is the code change to the point
   - is the code compliant to our coding and other best practices
   - does the code change pass system integration tests
   - etc., etc.

Vetting and testing these manually and on the development machine, takes te
committer away from enhancing our codebase himself. And when a vetted and
tested code change doesn’t pass muster, the testing committer needs to
update the ticket (but also when the change pases muster).

Maybe, all of this is one of the reasons why contributors and/or committer
lose interest and the project doesn’t get done much?

However, this should not be. With code enhancements coming in via patch
files in tickets and via Pull Request, we should adopt a method that will
help our committers (and fellow contributors too) by offering an automated
vetting and testing process that will deliver a report about the code and
its compliance to technical requirements.

This automated vetting and testing process can be done on arrival of a Pull
Request in the project’s public repository on Github, as well on newly
added patch files to a ticket. And such a report would , after the process
has completed, be incorporated in the Pull Request or the ticket. This way
the submitter, the potential committer and anyone else participating in
this project and interested in the issue at hand and the solution would be
informed in a consistent way. With such an automated process, where a
contributor can rework the code of an initially submitted set of change(s)
and submit again, and the vetting and testing can be done over and over
without any burden on a committer.Currently such automated vetting and
testing can include:

   - ASF license checks
   - checkstyle
   - whitespace checks
   - bug hangs (via findbugs or spotbugs)
   - javadoc compliance
   - java compilation testing
   - script validation
   - groovy compilation testing
   - system integration tests
   - etc.

I have some questions for you:

   - What are your thoughts on this?
   - Should we adopt this for our project?
   - Before adopting this for our project, should we define a PoC and
   execute it if acceptable?

Best regards,

Pierre Smits
*Proud* *contributor* (but without privileges)* of* Apache OFBiz
<https://ofbiz.apache.org/>, since 2008

*Apache Trafodion <https://trafodion.apache.org>, Vice President*
*Apache Directory <https://directory.apache.org>, PMC Member*
Apache Incubator <https://incubator.apache.org>, committer
Apache Steve <https://steve.apache.org>, committer

Re: Vetting & Testing Contributor’s Code Changes

[Pierre Smits <pi...@apache.org>]

Given that no objections have come forward, I assume lazy consensus and
will move forward with this.

Met vriendelijke groet,

Pierre Smits
*Proud* *contributor** of* Apache OFBiz <https://ofbiz.apache.org/> since
2008 (without privileges)

*Apache Trafodion <https://trafodion.apache.org>, Vice President*
*Apache Directory <https://directory.apache.org>, PMC Member*
Apache Incubator <https://incubator.apache.org>, committer
Apache Steve <https://steve.apache.org>, committer


On Sat, Feb 22, 2020 at 9:50 AM Pierre Smits <pi...@apache.org> wrote:

> Hi all,
>
> Currently we have Git (and GitHub) in full play for the master branch and
> the 2 active release branches. The transition to this methodology of
> distributed version control and collaboration has been adopted by our
> fellow contributors without any major impediments and/or destructions. At
> least none have been reported. It seems that everybody participating in
> this project and submitting code changes has an experience with Git and/or
> Github prior to the project’s move.
>
> In the past (during the svn era) the only option for fellow contributors
> to deliver code improvements for bug fixes and other enhancements was to
> deliver a file to a ticket in our Jira.
> Nowadays, the contributor can -besides still using that upload option- use
> the option to submit a Pull Request from his private clone or public fork
> to one of our official repositories on Github.
> In that past, and still to date, any code contribution uploaded to a Jira
> ticket needed to be downloaded by a committer and applied by him to a test
> branch (based on either the master or the active release branches) in order
> to vet the code changes and test the code changes whether these would break
> (or integrate with) the codebase. This vetting and testing was and is an
> arduous and  time consuming task, especially when the handed code changes
> span multiple components, languages and/or files. The committer must vet
> the changes with in his mind:
>
>    - does it have unnecessary whitespacing
>    - does it comply with checkstyle
>    - is the code change to the point
>    - is the code compliant to our coding and other best practices
>    - does the code change pass system integration tests
>    - etc., etc.
>
> Vetting and testing these manually and on the development machine, takes
> te committer away from enhancing our codebase himself. And when a vetted
> and tested code change doesn’t pass muster, the testing committer needs to
> update the ticket (but also when the change pases muster).
>
> Maybe, all of this is one of the reasons why contributors and/or committer
> lose interest and the project doesn’t get done much?
>
> However, this should not be. With code enhancements coming in via patch
> files in tickets and via Pull Request, we should adopt a method that will
> help our committers (and fellow contributors too) by offering an automated
> vetting and testing process that will deliver a report about the code and
> its compliance to technical requirements.
>
> This automated vetting and testing process can be done on arrival of a
> Pull Request in the project’s public repository on Github, as well on newly
> added patch files to a ticket. And such a report would , after the process
> has completed, be incorporated in the Pull Request or the ticket. This way
> the submitter, the potential committer and anyone else participating in
> this project and interested in the issue at hand and the solution would be
> informed in a consistent way. With such an automated process, where a
> contributor can rework the code of an initially submitted set of change(s)
> and submit again, and the vetting and testing can be done over and over
> without any burden on a committer.Currently such automated vetting and
> testing can include:
>
>    - ASF license checks
>    - checkstyle
>    - whitespace checks
>    - bug hangs (via findbugs or spotbugs)
>    - javadoc compliance
>    - java compilation testing
>    - script validation
>    - groovy compilation testing
>    - system integration tests
>    - etc.
>
> I have some questions for you:
>
>    - What are your thoughts on this?
>    - Should we adopt this for our project?
>    - Before adopting this for our project, should we define a PoC and
>    execute it if acceptable?
>
> Best regards,
>
> Pierre Smits
> *Proud* *contributor* (but without privileges)* of* Apache OFBiz
> <https://ofbiz.apache.org/>, since 2008
>
> *Apache Trafodion <https://trafodion.apache.org>, Vice President*
> *Apache Directory <https://directory.apache.org>, PMC Member*
> Apache Incubator <https://incubator.apache.org>, committer
> Apache Steve <https://steve.apache.org>, committer
>