Jenkins jobs

[Harald Wellmann <hw...@gmail.com>]

Do we have any defined policy regarding Jenkins jobs?

Looking at https://builds.apache.org/view/A-D/view/DeltaSpike/. it seems 
a number of jobs are disabled or have been failing for months.

It is not quite clear if a given job is supported, obsolete or experimental.

What about a simple rule like "don't release until all balls are blue", 
except for a few jobs to be marked as experimental, like WildFly 10 and 
TomEE 7?

Some jobs fail deterministically due to bugs in older CDI or app server 
versions. If there's a good reason to test on these old versions, then 
we should at least disable the tests known to fail on a given version 
with excludes in a version-specific Maven profile.

Other jobs fail sporadically due to port conflicts. This could be 
addressed with the build-helper-maven-plugin (reserve-network-ports).

Regarding build triggers, I can't look at the job configurations, but it 
seems that some jobs are directly triggered by a commit, while others 
are triggered by completion of the DeltaSpike Deploy job, which results 
in some jobs being run twice.

Wouldn't it be better to have a single gatekeeper job triggered by 
commits (e.g. DeltaSpike Deploy) and treat all other jobs as downstream 
dependencies that are only triggered by a _stable_ run of the gatekeeper 
job?

Regards,
Harald

Re: Jenkins jobs

[Gerhard Petracek <ge...@gmail.com>]

hi harald,

you mentioned some points already - i just summarize the intended approach
(independent of any improvement which might be possible).

jenkins-jobs are disabled if they are known to be broken completely, e.g.
due to an issue with the arquillian-adapter or once we don't support an old
version of owb/weld/... any longer.
failing jenkins-jobs are still "ok" in some cases, because we need to
observe if they fail due to a known issue (e.g. special constellations with
ear-files - see e.g. GF 4.0.x vs 4.1.x) and users can use it to check which
parts are compatible with the version they are using (or have to use).

sometimes jenkins-jobs fail, due to infrastructure issues (the
build-cluster is sometimes not that stable...).
(we could improve our config in view of e.g. ports, but that's just one
issue... -> we will face such issues in any case.)

before every release i check if there are new (failing) tests and in those
cases i run those profiles locally (usually it's working locally, if it
isn't a known issue).

regards,
gerhard



2015-12-21 11:56 GMT+01:00 Harald Wellmann <hw...@gmail.com>:

> Do we have any defined policy regarding Jenkins jobs?
>
> Looking at https://builds.apache.org/view/A-D/view/DeltaSpike/. it seems
> a number of jobs are disabled or have been failing for months.
>
> It is not quite clear if a given job is supported, obsolete or
> experimental.
>
> What about a simple rule like "don't release until all balls are blue",
> except for a few jobs to be marked as experimental, like WildFly 10 and
> TomEE 7?
>
> Some jobs fail deterministically due to bugs in older CDI or app server
> versions. If there's a good reason to test on these old versions, then we
> should at least disable the tests known to fail on a given version with
> excludes in a version-specific Maven profile.
>
> Other jobs fail sporadically due to port conflicts. This could be
> addressed with the build-helper-maven-plugin (reserve-network-ports).
>
> Regarding build triggers, I can't look at the job configurations, but it
> seems that some jobs are directly triggered by a commit, while others are
> triggered by completion of the DeltaSpike Deploy job, which results in some
> jobs being run twice.
>
> Wouldn't it be better to have a single gatekeeper job triggered by commits
> (e.g. DeltaSpike Deploy) and treat all other jobs as downstream
> dependencies that are only triggered by a _stable_ run of the gatekeeper
> job?
>
> Regards,
> Harald
>
>
>
>
>