[DISCUSS] Skip tests for documentation related changes

[Sijie Guo <si...@apache.org>]

Hi all,

Currently for every documentation change, we have run 3 precommit jobs,
java, c++ and integrationt tests. None of them is actually testing the
documentation change and it is wasting jenkins resources and make the merge
process for documentation changes take much longer time.

So I am proposing :

- add a separate precommit job for documentation-only changes. e.g.
`Jenkins: Documentation Tests`

- provide a checkbox in description
  * [ ] documentation-only change
  * [ ] code-only change

- if `[x] documentation-only` is checked, then java, c++ and integration
tests will be skipped.
- if `[x] code-only` is checked, then documentation tests will be skipped.


I would like to see what other people think about this.

- Sijie

Re: [DISCUSS] Skip tests for documentation related changes

[Ali Ahmed <ah...@gmail.com>]

I support it maybe a tag in the commit Header ie [Documentation] .

On Mon, Jan 28, 2019 at 12:12 PM Sijie Guo <si...@apache.org> wrote:

> Hi all,
>
> Currently for every documentation change, we have run 3 precommit jobs,
> java, c++ and integrationt tests. None of them is actually testing the
> documentation change and it is wasting jenkins resources and make the merge
> process for documentation changes take much longer time.
>
> So I am proposing :
>
> - add a separate precommit job for documentation-only changes. e.g.
> `Jenkins: Documentation Tests`
>
> - provide a checkbox in description
>   * [ ] documentation-only change
>   * [ ] code-only change
>
> - if `[x] documentation-only` is checked, then java, c++ and integration
> tests will be skipped.
> - if `[x] code-only` is checked, then documentation tests will be skipped.
>
>
> I would like to see what other people think about this.
>
> - Sijie
>
-- 
-Ali

Re: [DISCUSS] Skip tests for documentation related changes

[Sijie Guo <gu...@gmail.com>]

I have talked to a few Flink committers. They are using a `flinkbot` for
automating their review process.
It sounds very interesting. It is a process that they might be looking for.

The `flinkbot` code is here -
https://github.com/rmetzger/flink-community-tools
There is a discussion thread in flink mailing list about improving the
flinkbot.

If anyone has time, it might be worth checking out `flinkbot`. It can be a
good weekend project to prototype
a process to help improve pulsar review and ci process :-)

- Sijie

On Tue, Jan 29, 2019 at 6:29 AM Dave Fisher <da...@comcast.net> wrote:

> Hi -
>
> Inline
>
> Sent from my iPhone
>
> > On Jan 28, 2019, at 1:12 PM, Sijie Guo <gu...@gmail.com> wrote:
> >
> > both adding tag or checkbox are developer driven. but it is easier to
> > achieve with current github pull request builder in Jenkins.
> > so I think the responsibility of this approach is kind of spreading
> across
> > contributors and committers. It is also a way to educate
> > pulsar contributors to understand better pulsar codebase and its build
> > system.
> >
> > However if we want to automate the process, one way I can think of is
> > introducing some kind of `pulsarbot`.
>
> One suggestion is to bring this question to builds@apache.org mailing
> list and see if others have solved this problem in some way.
>
> >
> > - DONT run precommit jobs automatically (java, cpp, integration and docs)
> > - use Github hook to listen on activities of a pull request
> > - pulsarbot check the Github diff when a pull request is created or
> updated
> > - pulsarbot trigger corresponding precommit jobs to run
> >
> > But that is going to be a huge task to implement this mechanism. I am not
> > sure it is worth to do initially and who has enough time dedicated to
> this.
>
> Regards,
> Dave
>
> >
> > - Sijie
> >
> >> On Mon, Jan 28, 2019 at 12:26 PM Joe F <jo...@gmail.com> wrote:
> >>
> >> +1 to Sanjeev's suggestion
> >>
> >> On Mon, Jan 28, 2019 at 12:15 PM Sanjeev Kulkarni <sa...@gmail.com>
> >> wrote:
> >>
> >>> If developers are in charge of checking the checkbox, it might lead to
> >>> errors. Any way to make it automatic? Since docs are restricted to
> >> certain
> >>> areas of repo, maybe we can have some rules around that?
> >>>
> >>>> On Mon, Jan 28, 2019 at 12:12 PM Sijie Guo <si...@apache.org> wrote:
> >>>>
> >>>> Hi all,
> >>>>
> >>>> Currently for every documentation change, we have run 3 precommit
> jobs,
> >>>> java, c++ and integrationt tests. None of them is actually testing the
> >>>> documentation change and it is wasting jenkins resources and make the
> >>> merge
> >>>> process for documentation changes take much longer time.
> >>>>
> >>>> So I am proposing :
> >>>>
> >>>> - add a separate precommit job for documentation-only changes. e.g.
> >>>> `Jenkins: Documentation Tests`
> >>>>
> >>>> - provide a checkbox in description
> >>>>  * [ ] documentation-only change
> >>>>  * [ ] code-only change
> >>>>
> >>>> - if `[x] documentation-only` is checked, then java, c++ and
> >> integration
> >>>> tests will be skipped.
> >>>> - if `[x] code-only` is checked, then documentation tests will be
> >>> skipped.
> >>>>
> >>>>
> >>>> I would like to see what other people think about this.
> >>>>
> >>>> - Sijie
> >>>>
> >>>
> >>
>
>

Re: [DISCUSS] Skip tests for documentation related changes

[Dave Fisher <da...@comcast.net>]

Hi -

Inline

Sent from my iPhone

> On Jan 28, 2019, at 1:12 PM, Sijie Guo <gu...@gmail.com> wrote:
> 
> both adding tag or checkbox are developer driven. but it is easier to
> achieve with current github pull request builder in Jenkins.
> so I think the responsibility of this approach is kind of spreading across
> contributors and committers. It is also a way to educate
> pulsar contributors to understand better pulsar codebase and its build
> system.
> 
> However if we want to automate the process, one way I can think of is
> introducing some kind of `pulsarbot`.

One suggestion is to bring this question to builds@apache.org mailing list and see if others have solved this problem in some way.

> 
> - DONT run precommit jobs automatically (java, cpp, integration and docs)
> - use Github hook to listen on activities of a pull request
> - pulsarbot check the Github diff when a pull request is created or updated
> - pulsarbot trigger corresponding precommit jobs to run
> 
> But that is going to be a huge task to implement this mechanism. I am not
> sure it is worth to do initially and who has enough time dedicated to this.

Regards,
Dave

> 
> - Sijie
> 
>> On Mon, Jan 28, 2019 at 12:26 PM Joe F <jo...@gmail.com> wrote:
>> 
>> +1 to Sanjeev's suggestion
>> 
>> On Mon, Jan 28, 2019 at 12:15 PM Sanjeev Kulkarni <sa...@gmail.com>
>> wrote:
>> 
>>> If developers are in charge of checking the checkbox, it might lead to
>>> errors. Any way to make it automatic? Since docs are restricted to
>> certain
>>> areas of repo, maybe we can have some rules around that?
>>> 
>>>> On Mon, Jan 28, 2019 at 12:12 PM Sijie Guo <si...@apache.org> wrote:
>>>> 
>>>> Hi all,
>>>> 
>>>> Currently for every documentation change, we have run 3 precommit jobs,
>>>> java, c++ and integrationt tests. None of them is actually testing the
>>>> documentation change and it is wasting jenkins resources and make the
>>> merge
>>>> process for documentation changes take much longer time.
>>>> 
>>>> So I am proposing :
>>>> 
>>>> - add a separate precommit job for documentation-only changes. e.g.
>>>> `Jenkins: Documentation Tests`
>>>> 
>>>> - provide a checkbox in description
>>>>  * [ ] documentation-only change
>>>>  * [ ] code-only change
>>>> 
>>>> - if `[x] documentation-only` is checked, then java, c++ and
>> integration
>>>> tests will be skipped.
>>>> - if `[x] code-only` is checked, then documentation tests will be
>>> skipped.
>>>> 
>>>> 
>>>> I would like to see what other people think about this.
>>>> 
>>>> - Sijie
>>>> 
>>> 
>> 

Re: [DISCUSS] Skip tests for documentation related changes

[Sijie Guo <gu...@gmail.com>]

both adding tag or checkbox are developer driven. but it is easier to
achieve with current github pull request builder in Jenkins.
so I think the responsibility of this approach is kind of spreading across
contributors and committers. It is also a way to educate
pulsar contributors to understand better pulsar codebase and its build
system.

However if we want to automate the process, one way I can think of is
introducing some kind of `pulsarbot`.

- DONT run precommit jobs automatically (java, cpp, integration and docs)
- use Github hook to listen on activities of a pull request
- pulsarbot check the Github diff when a pull request is created or updated
- pulsarbot trigger corresponding precommit jobs to run

But that is going to be a huge task to implement this mechanism. I am not
sure it is worth to do initially and who has enough time dedicated to this.

- Sijie

On Mon, Jan 28, 2019 at 12:26 PM Joe F <jo...@gmail.com> wrote:

> +1 to Sanjeev's suggestion
>
> On Mon, Jan 28, 2019 at 12:15 PM Sanjeev Kulkarni <sa...@gmail.com>
> wrote:
>
> > If developers are in charge of checking the checkbox, it might lead to
> > errors. Any way to make it automatic? Since docs are restricted to
> certain
> > areas of repo, maybe we can have some rules around that?
> >
> > On Mon, Jan 28, 2019 at 12:12 PM Sijie Guo <si...@apache.org> wrote:
> >
> > > Hi all,
> > >
> > > Currently for every documentation change, we have run 3 precommit jobs,
> > > java, c++ and integrationt tests. None of them is actually testing the
> > > documentation change and it is wasting jenkins resources and make the
> > merge
> > > process for documentation changes take much longer time.
> > >
> > > So I am proposing :
> > >
> > > - add a separate precommit job for documentation-only changes. e.g.
> > > `Jenkins: Documentation Tests`
> > >
> > > - provide a checkbox in description
> > >   * [ ] documentation-only change
> > >   * [ ] code-only change
> > >
> > > - if `[x] documentation-only` is checked, then java, c++ and
> integration
> > > tests will be skipped.
> > > - if `[x] code-only` is checked, then documentation tests will be
> > skipped.
> > >
> > >
> > > I would like to see what other people think about this.
> > >
> > > - Sijie
> > >
> >
>

Re: [DISCUSS] Skip tests for documentation related changes

[Joe F <jo...@gmail.com>]

+1 to Sanjeev's suggestion

On Mon, Jan 28, 2019 at 12:15 PM Sanjeev Kulkarni <sa...@gmail.com>
wrote:

> If developers are in charge of checking the checkbox, it might lead to
> errors. Any way to make it automatic? Since docs are restricted to certain
> areas of repo, maybe we can have some rules around that?
>
> On Mon, Jan 28, 2019 at 12:12 PM Sijie Guo <si...@apache.org> wrote:
>
> > Hi all,
> >
> > Currently for every documentation change, we have run 3 precommit jobs,
> > java, c++ and integrationt tests. None of them is actually testing the
> > documentation change and it is wasting jenkins resources and make the
> merge
> > process for documentation changes take much longer time.
> >
> > So I am proposing :
> >
> > - add a separate precommit job for documentation-only changes. e.g.
> > `Jenkins: Documentation Tests`
> >
> > - provide a checkbox in description
> >   * [ ] documentation-only change
> >   * [ ] code-only change
> >
> > - if `[x] documentation-only` is checked, then java, c++ and integration
> > tests will be skipped.
> > - if `[x] code-only` is checked, then documentation tests will be
> skipped.
> >
> >
> > I would like to see what other people think about this.
> >
> > - Sijie
> >
>

Re: [DISCUSS] Skip tests for documentation related changes

[Sanjeev Kulkarni <sa...@gmail.com>]

If developers are in charge of checking the checkbox, it might lead to
errors. Any way to make it automatic? Since docs are restricted to certain
areas of repo, maybe we can have some rules around that?

On Mon, Jan 28, 2019 at 12:12 PM Sijie Guo <si...@apache.org> wrote:

> Hi all,
>
> Currently for every documentation change, we have run 3 precommit jobs,
> java, c++ and integrationt tests. None of them is actually testing the
> documentation change and it is wasting jenkins resources and make the merge
> process for documentation changes take much longer time.
>
> So I am proposing :
>
> - add a separate precommit job for documentation-only changes. e.g.
> `Jenkins: Documentation Tests`
>
> - provide a checkbox in description
>   * [ ] documentation-only change
>   * [ ] code-only change
>
> - if `[x] documentation-only` is checked, then java, c++ and integration
> tests will be skipped.
> - if `[x] code-only` is checked, then documentation tests will be skipped.
>
>
> I would like to see what other people think about this.
>
> - Sijie
>