Re: Pull request builds

[Richard Zowalla <rz...@apache.org>]

Hi,

I got some updates from builds@

Currently, it is not possible with Jenkins to run jobs from forked
repositories or for contributors, who do not have committer access to
the TomEE repositories.

The workaround is to push the branch to the asf repo and let it build
(as we are doing it).

However, they are discussing the possibility of having temporary nodes
(on k8s) in the future, so these (security) restrictions can be
dropped.

Another possibility would be to explore the use of GitHub actions for
running or (quick) tests in a PR build. I remember, that we
experimented with it in the past and it wasn't a good option due to
limitations / timing / resources. Maybe time to revisit and try the
"quick" build for PRs / commits again?

Gruß
Richard

Am Freitag, dem 28.10.2022 um 06:09 +0000 schrieb Zowalla, Richard:
> Hi,
> 
> I dropped a message in the builds slack channel to get some thoughts
> on
> our PR issue. Maybe they can give us some hints to docs. 
> 
> If the topic is to broad / wide for a quick slack chat, I will sent a
> mail to builds@ to get some thoughts / ideas. 
> 
> I agree with David, that we cannot be the only project, which suffers
> from that issue.
> 
> Gruß
> Richard
> 
> 
> Am Donnerstag, dem 27.10.2022 um 10:38 +0200 schrieb Jean-Louis
> Monteiro:
> > It only works if you push the PR straight to Apache repo I think
> > Richard
> > mentioned.
> > So every non committer submitting a PR is out of the builder. We
> > need
> > to
> > ask Infra maybe. I'd be surprised if we would be the first project
> > to
> > face
> > it.
> > --
> > Jean-Louis Monteiro
> > http://twitter.com/jlouismonteiro
> > http://www.tomitribe.com
> > 
> > 
> > On Thu, Oct 27, 2022 at 12:47 AM David Blevins <
> > david.blevins@gmail.com>
> > wrote:
> > 
> > > > On Oct 26, 2022, at 2:13 PM, David Blevins <
> > > > david.blevins@gmail.com>
> > > wrote:
> > > > I wonder if there's some way this can be improved so Richard
> > > > doesn't
> > > have to create CI jobs for individual PRs.  At minimum, maybe we
> > > can have a
> > > PR builder job that takes the PR number as a parameter?
> > > 
> > > I wonder also if there's some way with a Github Action that a
> > > committer
> > > could comment "test this please" and have that PR build kicked
> > > off
> > > in
> > > Apache's Jenkins?
> > > 
> > > 
> > > -David
> > > 
> > > 

Re: Pull request builds

[Richard Zowalla <rz...@apache.org>]

Yes. Setting up an entire job is cumbersome.

I think, that it would be possible to have interactive user input (i.e.
the branch) for a Jenkins job using Jenkins pipelines without
additional plugins. Some solutions seem to require additional plugins.
So we need to check, which plugins are available on the ASF instance.

This might be a good first time contribution, if someone has access to
a Jenkins installation. It might be even possible to setup a test
instance locally via docker compose.

Gruß
Richard


Am Freitag, dem 11.11.2022 um 09:24 -0800 schrieb David Blevins:
> > On Nov 10, 2022, at 10:34 PM, Richard Zowalla <rz...@apache.org>
> > wrote:
> > 
> > Currently, it is not possible with Jenkins to run jobs from forked
> > repositories or for contributors, who do not have committer access
> > to
> > the TomEE repositories.
> 
> Thanks for getting that confirmed.
> 
> > However, they are discussing the possibility of having temporary
> > nodes
> > (on k8s) in the future, so these (security) restrictions can be
> > dropped.
> > 
> > Another possibility would be to explore the use of GitHub actions
> > for
> > running or (quick) tests in a PR build. I remember, that we
> > experimented with it in the past and it wasn't a good option due to
> > limitations / timing / resources. Maybe time to revisit and try the
> > "quick" build for PRs / commits again?
> 
> Speaking for myself only, I personally would not want to see PRs
> merged with only a quick build.  I always do a quick build before
> pushing any code and I routinely break the build.
> 
> Similarly, I don't like to merge PRs that do not have full clean
> build as I would feel it's my responsibility to find/fix any issues
> they caused.  Library upgrades in particular more often than not
> break something in the build, so definitely need a full build before
> merging.
> 
> > The workaround is to push the branch to the asf repo and let it
> > build
> > (as we are doing it).
> 
> Along these lines, we should be able to setup a manually run PR
> builder where we just enter the PR number and it does the rest.
> 
> The git portions might have to be a shell script, but it should be a
> little better than having to make an entire job definition for one PR
> (and also having to delete those old job definitions later)
> 
> I don't really have the time to work on anything like that now, but
> seems promising if someone wanted to volunteer.  Theoretically,
> anyone with access to a jenkins install could try it out, get
> something that works and then we can set it up in the Apache install.
> 
> 
> -David
> 

Re: Pull request builds

[David Blevins <da...@gmail.com>]

> On Nov 10, 2022, at 10:34 PM, Richard Zowalla <rz...@apache.org> wrote:
> 
> Currently, it is not possible with Jenkins to run jobs from forked
> repositories or for contributors, who do not have committer access to
> the TomEE repositories.

Thanks for getting that confirmed.

> However, they are discussing the possibility of having temporary nodes
> (on k8s) in the future, so these (security) restrictions can be
> dropped.
> 
> Another possibility would be to explore the use of GitHub actions for
> running or (quick) tests in a PR build. I remember, that we
> experimented with it in the past and it wasn't a good option due to
> limitations / timing / resources. Maybe time to revisit and try the
> "quick" build for PRs / commits again?

Speaking for myself only, I personally would not want to see PRs merged with only a quick build.  I always do a quick build before pushing any code and I routinely break the build.

Similarly, I don't like to merge PRs that do not have full clean build as I would feel it's my responsibility to find/fix any issues they caused.  Library upgrades in particular more often than not break something in the build, so definitely need a full build before merging.

> The workaround is to push the branch to the asf repo and let it build
> (as we are doing it).

Along these lines, we should be able to setup a manually run PR builder where we just enter the PR number and it does the rest.

The git portions might have to be a shell script, but it should be a little better than having to make an entire job definition for one PR (and also having to delete those old job definitions later)

I don't really have the time to work on anything like that now, but seems promising if someone wanted to volunteer.  Theoretically, anyone with access to a jenkins install could try it out, get something that works and then we can set it up in the Apache install.


-David