You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mxnet.apache.org by Marco de Abreu <ma...@googlemail.com.INVALID> on 2018/11/20 16:04:54 UTC
Splitting Jenkins pipelines - stop changes to Jenkinsfiles!
Hello,
we ran into issues around the maximum filesize of the Jenkinsfile a few
times already. In order to resolve this issue, I'd like to combine this
with some refactors I have planned for quite some time.
The idea is basically to move away from one big Jenkinsfile and instead
split it into separate jobs that run in parallel and report their status
individually. Besides avoiding the size restriction, this will greatly
speed up the PR validation process by reducing the critical path. Instead
of having to wait for every single step within a stage to finish before the
next stage (e.g. tests) is getting executed, these pipelines would now be
able to move forward individually. I'm still in the process of refactoring
and can't provide any numbers or documentation at this time, but I would
like to announce this early on to avoid conflicts:
Since I will remove the original Jenkinsfile, this might cause conflicts
with ongoing efforts that try to change the Jenkinsfile. This poses the
risk that I might forget to port a change. Thus, I'd like to ask all
contributors to wait with changes of Jenkinsfile and would like to request
fellow-committers to wait with merging any Jenkinsfile-related PRs until
further notice.
I expect to finish this refactor until the end of the week. Please don't
hesitate to ask if you've got further questions.
Please excuse any caused inconveniences.
Best regards,
Marco
Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!
Posted by Marco de Abreu <ma...@googlemail.com.INVALID>.
Hi Anirudh,
I have just deployed a fix for this. The status reports are now back to
where they were:
- "continuous-integration/jenkins/pr-merge" is the pipeline everybody is
used and backed by the main Jenkinsfile -> This is the one you have to look
at
- "ci/jenkins/mxnet-validation/XXX" this is the new pipeline. It's 100%
equivalent with the existing pipeline. It's completely working, but you can
ignore it for now.
- "continuous-integration/travis-ci/pr" this is the travis pipeline which
is currently in beta, you can ignore it for now
This means that the status report with the "required" flag is really
required and you can ignore everything else.
I will respond the CI status question in a separate thread.
Best regards,
Marco
On Thu, Nov 22, 2018 at 12:25 AM Anirudh <an...@gmail.com> wrote:
> Hi Marco,
>
> Can you point out specifically which checks we have to make sure pass
> before merging PRs. Currently apart from the required one there are six
> steps added. Also, is the CI down currently :
>
> http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-13324/17/pipeline
>
>
> Anirudh
>
> On Wed, Nov 21, 2018 at 9:31 AM Marco de Abreu
> <ma...@googlemail.com.invalid> wrote:
>
> > Please notice that the "continuous-integration/jenkins/pr-merge"
> currently
> > is overlapping with the new pipelines. Please make sure all checks pass
> > (also the non-required ones) before merging the PRs. I will work on a fix
> > for this overlap.
> >
> > -Marco
> >
> > On Wed, Nov 21, 2018 at 5:42 PM Anton Chernov <me...@gmail.com>
> wrote:
> >
> > > The ability to retrigger the pipelines separately is an amazing step
> > > forward. Great job Marco!
> > >
> > > ср, 21 нояб. 2018 г. в 15:03, Marco de Abreu
> > > <ma...@googlemail.com.invalid>:
> > >
> > > > Hello,
> > > >
> > > > the PR has been merged and I've created the new pipelines at [1]. You
> > can
> > > > see the new reports if you have a look at this example PR at [2].
> > > >
> > > > The new status messages will be the ones starting with
> > > > "ci/jenkins/mxnet-validation/".
> > > >
> > > > This now allows you to retrigger specific pipelines if they fail. For
> > > > example, if you're interested in the website pipeline, you can now go
> > to
> > > > [3] and just retrigger that instead of running the entire suite.
> > Whenever
> > > > there's a new commit, all pipelines will still be scheduled as before
> > > (the
> > > > overall behaviour or coverage of our pipeline did not change, I just
> > > > decoupled them and increased the usability).
> > > >
> > > > The next step will be the deprecation of the main Jenkinsfile (the
> one
> > > > which reports the status as
> "continuous-integration/jenkins/pr-merge")
> > > and
> > > > requesting these new statuses to be marked as required (protected
> > master
> > > > branch). Since we have to change some reporting tools to point to the
> > new
> > > > jobs and I'd like to observe the stability for some time, this will
> > take
> > > > some times.
> > > >
> > > > You can now resume changes in the Jenkinsfiles. But please do not
> > modify
> > > > the Jenkinsfile in the root directory but instead the ones at [4].
> The
> > > > nightly Jenkinsfiles (or basically all Jenkinsfiles that are not part
> > of
> > > > the main pipeline) have not been migrated yet and I will do that at a
> > > later
> > > > point in time.
> > > >
> > > > Best regards,
> > > > Marco
> > > >
> > > > [1]: http://jenkins.mxnet-ci.amazon-ml.com/job/mxnet-validation/
> > > > [2]: https://github.com/apache/incubator-mxnet/pull/13352
> > > > [3]:
> > > >
> > > >
> > >
> >
> http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwebsite/detail/PR-13352/1/pipeline
> > > > [4]:
> https://github.com/apache/incubator-mxnet/tree/master/ci/jenkins
> > > >
> > > > On Tue, Nov 20, 2018 at 9:33 PM Marco de Abreu <
> > > > marco.g.abreu@googlemail.com>
> > > > wrote:
> > > >
> > > > > I have just submitted my PR at
> > > > > https://github.com/apache/incubator-mxnet/pull/13344. Test jobs
> are
> > > > > available at
> > > > > http://jenkins.mxnet-ci-dev.amazon-ml.com/view/test-marco-mxnet/.
> > > > >
> > > > > As soon as I'm done with my tests, I will mark it as ready for
> > review.
> > > > >
> > > > > Best regards,
> > > > > Marco
> > > > >
> > > > > On Tue, Nov 20, 2018 at 9:09 PM Marco de Abreu <
> > > > > marco.g.abreu@googlemail.com> wrote:
> > > > >
> > > > >> Thanks, Pedro!
> > > > >>
> > > > >> I have also been looking into that issue, but it seems like this
> > would
> > > > >> require changes in the groovy interpreter of Jenkins. From what I
> > can
> > > > tell,
> > > > >> a refactor will give us multiple benefits (clarity and speed)
> aside
> > > from
> > > > >> resolving this issue.
> > > > >>
> > > > >> Best regards,
> > > > >> Marco
> > > > >>
> > > > >> Am Di., 20. Nov. 2018, 19:54 hat Pedro Larroy <
> > > > >> pedro.larroy.lists@gmail.com> geschrieben:
> > > > >>
> > > > >>> I think this is a big problem, which has blocked us before. I
> want
> > to
> > > > >>> point out that you are doing a great thing by avoiding everyone
> > > > >>> getting blocked by refactoring the pipelines.
> > > > >>>
> > > > >>> My concern is that we are kicking the can down the road and not
> > > > >>> addressing the root cause of the problem with is known
> > > > >>> https://issues.jenkins-ci.org/browse/JENKINS-37984
> > > > >>>
> > > > >>> Pedro.
> > > > >>>
> > > > >>>
> > > > >>> On Tue, Nov 20, 2018 at 6:08 PM Marco de Abreu
> > > > >>> <ma...@googlemail.com.invalid> wrote:
> > > > >>> >
> > > > >>> > Hello Steffen,
> > > > >>> >
> > > > >>> > no, there won't be any impact on the PR process or nightly
> > > > regressions.
> > > > >>> > Only the reporting will have to be updated with the new job
> > links,
> > > > but
> > > > >>> that
> > > > >>> > should be a minor issue. To avoid any outage, I have been
> > thinking
> > > > >>> about
> > > > >>> > running both versions in parallel.
> > > > >>> >
> > > > >>> > Best regards,
> > > > >>> > Marco
> > > > >>> >
> > > > >>> >
> > > > >>> >
> > > > >>> > On Tue, Nov 20, 2018 at 5:53 PM Steffen Rochel <
> > > > >>> steffenrochel@gmail.com>
> > > > >>> > wrote:
> > > > >>> >
> > > > >>> > > Hi Marco - is there any impact on reporting, the PR process
> or
> > > > >>> nightly
> > > > >>> > > regression beside reduction in TAT? If yes, please
> elaborate.
> > > > >>> > > Steffen
> > > > >>> > >
> > > > >>> > > On Tue, Nov 20, 2018 at 8:05 AM Marco de Abreu
> > > > >>> > > <ma...@googlemail.com.invalid> wrote:
> > > > >>> > >
> > > > >>> > > > Hello,
> > > > >>> > > >
> > > > >>> > > > we ran into issues around the maximum filesize of the
> > > Jenkinsfile
> > > > >>> a few
> > > > >>> > > > times already. In order to resolve this issue, I'd like to
> > > > combine
> > > > >>> this
> > > > >>> > > > with some refactors I have planned for quite some time.
> > > > >>> > > >
> > > > >>> > > > The idea is basically to move away from one big Jenkinsfile
> > and
> > > > >>> instead
> > > > >>> > > > split it into separate jobs that run in parallel and report
> > > their
> > > > >>> status
> > > > >>> > > > individually. Besides avoiding the size restriction, this
> > will
> > > > >>> greatly
> > > > >>> > > > speed up the PR validation process by reducing the critical
> > > path.
> > > > >>> Instead
> > > > >>> > > > of having to wait for every single step within a stage to
> > > finish
> > > > >>> before
> > > > >>> > > the
> > > > >>> > > > next stage (e.g. tests) is getting executed, these
> pipelines
> > > > would
> > > > >>> now be
> > > > >>> > > > able to move forward individually. I'm still in the process
> > of
> > > > >>> > > refactoring
> > > > >>> > > > and can't provide any numbers or documentation at this
> time,
> > > but
> > > > I
> > > > >>> would
> > > > >>> > > > like to announce this early on to avoid conflicts:
> > > > >>> > > >
> > > > >>> > > > Since I will remove the original Jenkinsfile, this might
> > cause
> > > > >>> conflicts
> > > > >>> > > > with ongoing efforts that try to change the Jenkinsfile.
> This
> > > > >>> poses the
> > > > >>> > > > risk that I might forget to port a change. Thus, I'd like
> to
> > > ask
> > > > >>> all
> > > > >>> > > > contributors to wait with changes of Jenkinsfile and would
> > like
> > > > to
> > > > >>> > > request
> > > > >>> > > > fellow-committers to wait with merging any
> > Jenkinsfile-related
> > > > PRs
> > > > >>> until
> > > > >>> > > > further notice.
> > > > >>> > > >
> > > > >>> > > > I expect to finish this refactor until the end of the week.
> > > > Please
> > > > >>> don't
> > > > >>> > > > hesitate to ask if you've got further questions.
> > > > >>> > > >
> > > > >>> > > > Please excuse any caused inconveniences.
> > > > >>> > > >
> > > > >>> > > > Best regards,
> > > > >>> > > > Marco
> > > > >>> > > >
> > > > >>> > >
> > > > >>>
> > > > >>
> > > >
> > >
> >
>
Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!
Posted by Anirudh <an...@gmail.com>.
Hi Marco,
Can you point out specifically which checks we have to make sure pass
before merging PRs. Currently apart from the required one there are six
steps added. Also, is the CI down currently :
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-13324/17/pipeline
Anirudh
On Wed, Nov 21, 2018 at 9:31 AM Marco de Abreu
<ma...@googlemail.com.invalid> wrote:
> Please notice that the "continuous-integration/jenkins/pr-merge" currently
> is overlapping with the new pipelines. Please make sure all checks pass
> (also the non-required ones) before merging the PRs. I will work on a fix
> for this overlap.
>
> -Marco
>
> On Wed, Nov 21, 2018 at 5:42 PM Anton Chernov <me...@gmail.com> wrote:
>
> > The ability to retrigger the pipelines separately is an amazing step
> > forward. Great job Marco!
> >
> > ср, 21 нояб. 2018 г. в 15:03, Marco de Abreu
> > <ma...@googlemail.com.invalid>:
> >
> > > Hello,
> > >
> > > the PR has been merged and I've created the new pipelines at [1]. You
> can
> > > see the new reports if you have a look at this example PR at [2].
> > >
> > > The new status messages will be the ones starting with
> > > "ci/jenkins/mxnet-validation/".
> > >
> > > This now allows you to retrigger specific pipelines if they fail. For
> > > example, if you're interested in the website pipeline, you can now go
> to
> > > [3] and just retrigger that instead of running the entire suite.
> Whenever
> > > there's a new commit, all pipelines will still be scheduled as before
> > (the
> > > overall behaviour or coverage of our pipeline did not change, I just
> > > decoupled them and increased the usability).
> > >
> > > The next step will be the deprecation of the main Jenkinsfile (the one
> > > which reports the status as "continuous-integration/jenkins/pr-merge")
> > and
> > > requesting these new statuses to be marked as required (protected
> master
> > > branch). Since we have to change some reporting tools to point to the
> new
> > > jobs and I'd like to observe the stability for some time, this will
> take
> > > some times.
> > >
> > > You can now resume changes in the Jenkinsfiles. But please do not
> modify
> > > the Jenkinsfile in the root directory but instead the ones at [4]. The
> > > nightly Jenkinsfiles (or basically all Jenkinsfiles that are not part
> of
> > > the main pipeline) have not been migrated yet and I will do that at a
> > later
> > > point in time.
> > >
> > > Best regards,
> > > Marco
> > >
> > > [1]: http://jenkins.mxnet-ci.amazon-ml.com/job/mxnet-validation/
> > > [2]: https://github.com/apache/incubator-mxnet/pull/13352
> > > [3]:
> > >
> > >
> >
> http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwebsite/detail/PR-13352/1/pipeline
> > > [4]: https://github.com/apache/incubator-mxnet/tree/master/ci/jenkins
> > >
> > > On Tue, Nov 20, 2018 at 9:33 PM Marco de Abreu <
> > > marco.g.abreu@googlemail.com>
> > > wrote:
> > >
> > > > I have just submitted my PR at
> > > > https://github.com/apache/incubator-mxnet/pull/13344. Test jobs are
> > > > available at
> > > > http://jenkins.mxnet-ci-dev.amazon-ml.com/view/test-marco-mxnet/.
> > > >
> > > > As soon as I'm done with my tests, I will mark it as ready for
> review.
> > > >
> > > > Best regards,
> > > > Marco
> > > >
> > > > On Tue, Nov 20, 2018 at 9:09 PM Marco de Abreu <
> > > > marco.g.abreu@googlemail.com> wrote:
> > > >
> > > >> Thanks, Pedro!
> > > >>
> > > >> I have also been looking into that issue, but it seems like this
> would
> > > >> require changes in the groovy interpreter of Jenkins. From what I
> can
> > > tell,
> > > >> a refactor will give us multiple benefits (clarity and speed) aside
> > from
> > > >> resolving this issue.
> > > >>
> > > >> Best regards,
> > > >> Marco
> > > >>
> > > >> Am Di., 20. Nov. 2018, 19:54 hat Pedro Larroy <
> > > >> pedro.larroy.lists@gmail.com> geschrieben:
> > > >>
> > > >>> I think this is a big problem, which has blocked us before. I want
> to
> > > >>> point out that you are doing a great thing by avoiding everyone
> > > >>> getting blocked by refactoring the pipelines.
> > > >>>
> > > >>> My concern is that we are kicking the can down the road and not
> > > >>> addressing the root cause of the problem with is known
> > > >>> https://issues.jenkins-ci.org/browse/JENKINS-37984
> > > >>>
> > > >>> Pedro.
> > > >>>
> > > >>>
> > > >>> On Tue, Nov 20, 2018 at 6:08 PM Marco de Abreu
> > > >>> <ma...@googlemail.com.invalid> wrote:
> > > >>> >
> > > >>> > Hello Steffen,
> > > >>> >
> > > >>> > no, there won't be any impact on the PR process or nightly
> > > regressions.
> > > >>> > Only the reporting will have to be updated with the new job
> links,
> > > but
> > > >>> that
> > > >>> > should be a minor issue. To avoid any outage, I have been
> thinking
> > > >>> about
> > > >>> > running both versions in parallel.
> > > >>> >
> > > >>> > Best regards,
> > > >>> > Marco
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > On Tue, Nov 20, 2018 at 5:53 PM Steffen Rochel <
> > > >>> steffenrochel@gmail.com>
> > > >>> > wrote:
> > > >>> >
> > > >>> > > Hi Marco - is there any impact on reporting, the PR process or
> > > >>> nightly
> > > >>> > > regression beside reduction in TAT? If yes, please elaborate.
> > > >>> > > Steffen
> > > >>> > >
> > > >>> > > On Tue, Nov 20, 2018 at 8:05 AM Marco de Abreu
> > > >>> > > <ma...@googlemail.com.invalid> wrote:
> > > >>> > >
> > > >>> > > > Hello,
> > > >>> > > >
> > > >>> > > > we ran into issues around the maximum filesize of the
> > Jenkinsfile
> > > >>> a few
> > > >>> > > > times already. In order to resolve this issue, I'd like to
> > > combine
> > > >>> this
> > > >>> > > > with some refactors I have planned for quite some time.
> > > >>> > > >
> > > >>> > > > The idea is basically to move away from one big Jenkinsfile
> and
> > > >>> instead
> > > >>> > > > split it into separate jobs that run in parallel and report
> > their
> > > >>> status
> > > >>> > > > individually. Besides avoiding the size restriction, this
> will
> > > >>> greatly
> > > >>> > > > speed up the PR validation process by reducing the critical
> > path.
> > > >>> Instead
> > > >>> > > > of having to wait for every single step within a stage to
> > finish
> > > >>> before
> > > >>> > > the
> > > >>> > > > next stage (e.g. tests) is getting executed, these pipelines
> > > would
> > > >>> now be
> > > >>> > > > able to move forward individually. I'm still in the process
> of
> > > >>> > > refactoring
> > > >>> > > > and can't provide any numbers or documentation at this time,
> > but
> > > I
> > > >>> would
> > > >>> > > > like to announce this early on to avoid conflicts:
> > > >>> > > >
> > > >>> > > > Since I will remove the original Jenkinsfile, this might
> cause
> > > >>> conflicts
> > > >>> > > > with ongoing efforts that try to change the Jenkinsfile. This
> > > >>> poses the
> > > >>> > > > risk that I might forget to port a change. Thus, I'd like to
> > ask
> > > >>> all
> > > >>> > > > contributors to wait with changes of Jenkinsfile and would
> like
> > > to
> > > >>> > > request
> > > >>> > > > fellow-committers to wait with merging any
> Jenkinsfile-related
> > > PRs
> > > >>> until
> > > >>> > > > further notice.
> > > >>> > > >
> > > >>> > > > I expect to finish this refactor until the end of the week.
> > > Please
> > > >>> don't
> > > >>> > > > hesitate to ask if you've got further questions.
> > > >>> > > >
> > > >>> > > > Please excuse any caused inconveniences.
> > > >>> > > >
> > > >>> > > > Best regards,
> > > >>> > > > Marco
> > > >>> > > >
> > > >>> > >
> > > >>>
> > > >>
> > >
> >
>
Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!
Posted by Marco de Abreu <ma...@googlemail.com.INVALID>.
Please notice that the "continuous-integration/jenkins/pr-merge" currently
is overlapping with the new pipelines. Please make sure all checks pass
(also the non-required ones) before merging the PRs. I will work on a fix
for this overlap.
-Marco
On Wed, Nov 21, 2018 at 5:42 PM Anton Chernov <me...@gmail.com> wrote:
> The ability to retrigger the pipelines separately is an amazing step
> forward. Great job Marco!
>
> ср, 21 нояб. 2018 г. в 15:03, Marco de Abreu
> <ma...@googlemail.com.invalid>:
>
> > Hello,
> >
> > the PR has been merged and I've created the new pipelines at [1]. You can
> > see the new reports if you have a look at this example PR at [2].
> >
> > The new status messages will be the ones starting with
> > "ci/jenkins/mxnet-validation/".
> >
> > This now allows you to retrigger specific pipelines if they fail. For
> > example, if you're interested in the website pipeline, you can now go to
> > [3] and just retrigger that instead of running the entire suite. Whenever
> > there's a new commit, all pipelines will still be scheduled as before
> (the
> > overall behaviour or coverage of our pipeline did not change, I just
> > decoupled them and increased the usability).
> >
> > The next step will be the deprecation of the main Jenkinsfile (the one
> > which reports the status as "continuous-integration/jenkins/pr-merge")
> and
> > requesting these new statuses to be marked as required (protected master
> > branch). Since we have to change some reporting tools to point to the new
> > jobs and I'd like to observe the stability for some time, this will take
> > some times.
> >
> > You can now resume changes in the Jenkinsfiles. But please do not modify
> > the Jenkinsfile in the root directory but instead the ones at [4]. The
> > nightly Jenkinsfiles (or basically all Jenkinsfiles that are not part of
> > the main pipeline) have not been migrated yet and I will do that at a
> later
> > point in time.
> >
> > Best regards,
> > Marco
> >
> > [1]: http://jenkins.mxnet-ci.amazon-ml.com/job/mxnet-validation/
> > [2]: https://github.com/apache/incubator-mxnet/pull/13352
> > [3]:
> >
> >
> http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwebsite/detail/PR-13352/1/pipeline
> > [4]: https://github.com/apache/incubator-mxnet/tree/master/ci/jenkins
> >
> > On Tue, Nov 20, 2018 at 9:33 PM Marco de Abreu <
> > marco.g.abreu@googlemail.com>
> > wrote:
> >
> > > I have just submitted my PR at
> > > https://github.com/apache/incubator-mxnet/pull/13344. Test jobs are
> > > available at
> > > http://jenkins.mxnet-ci-dev.amazon-ml.com/view/test-marco-mxnet/.
> > >
> > > As soon as I'm done with my tests, I will mark it as ready for review.
> > >
> > > Best regards,
> > > Marco
> > >
> > > On Tue, Nov 20, 2018 at 9:09 PM Marco de Abreu <
> > > marco.g.abreu@googlemail.com> wrote:
> > >
> > >> Thanks, Pedro!
> > >>
> > >> I have also been looking into that issue, but it seems like this would
> > >> require changes in the groovy interpreter of Jenkins. From what I can
> > tell,
> > >> a refactor will give us multiple benefits (clarity and speed) aside
> from
> > >> resolving this issue.
> > >>
> > >> Best regards,
> > >> Marco
> > >>
> > >> Am Di., 20. Nov. 2018, 19:54 hat Pedro Larroy <
> > >> pedro.larroy.lists@gmail.com> geschrieben:
> > >>
> > >>> I think this is a big problem, which has blocked us before. I want to
> > >>> point out that you are doing a great thing by avoiding everyone
> > >>> getting blocked by refactoring the pipelines.
> > >>>
> > >>> My concern is that we are kicking the can down the road and not
> > >>> addressing the root cause of the problem with is known
> > >>> https://issues.jenkins-ci.org/browse/JENKINS-37984
> > >>>
> > >>> Pedro.
> > >>>
> > >>>
> > >>> On Tue, Nov 20, 2018 at 6:08 PM Marco de Abreu
> > >>> <ma...@googlemail.com.invalid> wrote:
> > >>> >
> > >>> > Hello Steffen,
> > >>> >
> > >>> > no, there won't be any impact on the PR process or nightly
> > regressions.
> > >>> > Only the reporting will have to be updated with the new job links,
> > but
> > >>> that
> > >>> > should be a minor issue. To avoid any outage, I have been thinking
> > >>> about
> > >>> > running both versions in parallel.
> > >>> >
> > >>> > Best regards,
> > >>> > Marco
> > >>> >
> > >>> >
> > >>> >
> > >>> > On Tue, Nov 20, 2018 at 5:53 PM Steffen Rochel <
> > >>> steffenrochel@gmail.com>
> > >>> > wrote:
> > >>> >
> > >>> > > Hi Marco - is there any impact on reporting, the PR process or
> > >>> nightly
> > >>> > > regression beside reduction in TAT? If yes, please elaborate.
> > >>> > > Steffen
> > >>> > >
> > >>> > > On Tue, Nov 20, 2018 at 8:05 AM Marco de Abreu
> > >>> > > <ma...@googlemail.com.invalid> wrote:
> > >>> > >
> > >>> > > > Hello,
> > >>> > > >
> > >>> > > > we ran into issues around the maximum filesize of the
> Jenkinsfile
> > >>> a few
> > >>> > > > times already. In order to resolve this issue, I'd like to
> > combine
> > >>> this
> > >>> > > > with some refactors I have planned for quite some time.
> > >>> > > >
> > >>> > > > The idea is basically to move away from one big Jenkinsfile and
> > >>> instead
> > >>> > > > split it into separate jobs that run in parallel and report
> their
> > >>> status
> > >>> > > > individually. Besides avoiding the size restriction, this will
> > >>> greatly
> > >>> > > > speed up the PR validation process by reducing the critical
> path.
> > >>> Instead
> > >>> > > > of having to wait for every single step within a stage to
> finish
> > >>> before
> > >>> > > the
> > >>> > > > next stage (e.g. tests) is getting executed, these pipelines
> > would
> > >>> now be
> > >>> > > > able to move forward individually. I'm still in the process of
> > >>> > > refactoring
> > >>> > > > and can't provide any numbers or documentation at this time,
> but
> > I
> > >>> would
> > >>> > > > like to announce this early on to avoid conflicts:
> > >>> > > >
> > >>> > > > Since I will remove the original Jenkinsfile, this might cause
> > >>> conflicts
> > >>> > > > with ongoing efforts that try to change the Jenkinsfile. This
> > >>> poses the
> > >>> > > > risk that I might forget to port a change. Thus, I'd like to
> ask
> > >>> all
> > >>> > > > contributors to wait with changes of Jenkinsfile and would like
> > to
> > >>> > > request
> > >>> > > > fellow-committers to wait with merging any Jenkinsfile-related
> > PRs
> > >>> until
> > >>> > > > further notice.
> > >>> > > >
> > >>> > > > I expect to finish this refactor until the end of the week.
> > Please
> > >>> don't
> > >>> > > > hesitate to ask if you've got further questions.
> > >>> > > >
> > >>> > > > Please excuse any caused inconveniences.
> > >>> > > >
> > >>> > > > Best regards,
> > >>> > > > Marco
> > >>> > > >
> > >>> > >
> > >>>
> > >>
> >
>
Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!
Posted by Anton Chernov <me...@gmail.com>.
The ability to retrigger the pipelines separately is an amazing step
forward. Great job Marco!
ср, 21 нояб. 2018 г. в 15:03, Marco de Abreu
<ma...@googlemail.com.invalid>:
> Hello,
>
> the PR has been merged and I've created the new pipelines at [1]. You can
> see the new reports if you have a look at this example PR at [2].
>
> The new status messages will be the ones starting with
> "ci/jenkins/mxnet-validation/".
>
> This now allows you to retrigger specific pipelines if they fail. For
> example, if you're interested in the website pipeline, you can now go to
> [3] and just retrigger that instead of running the entire suite. Whenever
> there's a new commit, all pipelines will still be scheduled as before (the
> overall behaviour or coverage of our pipeline did not change, I just
> decoupled them and increased the usability).
>
> The next step will be the deprecation of the main Jenkinsfile (the one
> which reports the status as "continuous-integration/jenkins/pr-merge") and
> requesting these new statuses to be marked as required (protected master
> branch). Since we have to change some reporting tools to point to the new
> jobs and I'd like to observe the stability for some time, this will take
> some times.
>
> You can now resume changes in the Jenkinsfiles. But please do not modify
> the Jenkinsfile in the root directory but instead the ones at [4]. The
> nightly Jenkinsfiles (or basically all Jenkinsfiles that are not part of
> the main pipeline) have not been migrated yet and I will do that at a later
> point in time.
>
> Best regards,
> Marco
>
> [1]: http://jenkins.mxnet-ci.amazon-ml.com/job/mxnet-validation/
> [2]: https://github.com/apache/incubator-mxnet/pull/13352
> [3]:
>
> http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwebsite/detail/PR-13352/1/pipeline
> [4]: https://github.com/apache/incubator-mxnet/tree/master/ci/jenkins
>
> On Tue, Nov 20, 2018 at 9:33 PM Marco de Abreu <
> marco.g.abreu@googlemail.com>
> wrote:
>
> > I have just submitted my PR at
> > https://github.com/apache/incubator-mxnet/pull/13344. Test jobs are
> > available at
> > http://jenkins.mxnet-ci-dev.amazon-ml.com/view/test-marco-mxnet/.
> >
> > As soon as I'm done with my tests, I will mark it as ready for review.
> >
> > Best regards,
> > Marco
> >
> > On Tue, Nov 20, 2018 at 9:09 PM Marco de Abreu <
> > marco.g.abreu@googlemail.com> wrote:
> >
> >> Thanks, Pedro!
> >>
> >> I have also been looking into that issue, but it seems like this would
> >> require changes in the groovy interpreter of Jenkins. From what I can
> tell,
> >> a refactor will give us multiple benefits (clarity and speed) aside from
> >> resolving this issue.
> >>
> >> Best regards,
> >> Marco
> >>
> >> Am Di., 20. Nov. 2018, 19:54 hat Pedro Larroy <
> >> pedro.larroy.lists@gmail.com> geschrieben:
> >>
> >>> I think this is a big problem, which has blocked us before. I want to
> >>> point out that you are doing a great thing by avoiding everyone
> >>> getting blocked by refactoring the pipelines.
> >>>
> >>> My concern is that we are kicking the can down the road and not
> >>> addressing the root cause of the problem with is known
> >>> https://issues.jenkins-ci.org/browse/JENKINS-37984
> >>>
> >>> Pedro.
> >>>
> >>>
> >>> On Tue, Nov 20, 2018 at 6:08 PM Marco de Abreu
> >>> <ma...@googlemail.com.invalid> wrote:
> >>> >
> >>> > Hello Steffen,
> >>> >
> >>> > no, there won't be any impact on the PR process or nightly
> regressions.
> >>> > Only the reporting will have to be updated with the new job links,
> but
> >>> that
> >>> > should be a minor issue. To avoid any outage, I have been thinking
> >>> about
> >>> > running both versions in parallel.
> >>> >
> >>> > Best regards,
> >>> > Marco
> >>> >
> >>> >
> >>> >
> >>> > On Tue, Nov 20, 2018 at 5:53 PM Steffen Rochel <
> >>> steffenrochel@gmail.com>
> >>> > wrote:
> >>> >
> >>> > > Hi Marco - is there any impact on reporting, the PR process or
> >>> nightly
> >>> > > regression beside reduction in TAT? If yes, please elaborate.
> >>> > > Steffen
> >>> > >
> >>> > > On Tue, Nov 20, 2018 at 8:05 AM Marco de Abreu
> >>> > > <ma...@googlemail.com.invalid> wrote:
> >>> > >
> >>> > > > Hello,
> >>> > > >
> >>> > > > we ran into issues around the maximum filesize of the Jenkinsfile
> >>> a few
> >>> > > > times already. In order to resolve this issue, I'd like to
> combine
> >>> this
> >>> > > > with some refactors I have planned for quite some time.
> >>> > > >
> >>> > > > The idea is basically to move away from one big Jenkinsfile and
> >>> instead
> >>> > > > split it into separate jobs that run in parallel and report their
> >>> status
> >>> > > > individually. Besides avoiding the size restriction, this will
> >>> greatly
> >>> > > > speed up the PR validation process by reducing the critical path.
> >>> Instead
> >>> > > > of having to wait for every single step within a stage to finish
> >>> before
> >>> > > the
> >>> > > > next stage (e.g. tests) is getting executed, these pipelines
> would
> >>> now be
> >>> > > > able to move forward individually. I'm still in the process of
> >>> > > refactoring
> >>> > > > and can't provide any numbers or documentation at this time, but
> I
> >>> would
> >>> > > > like to announce this early on to avoid conflicts:
> >>> > > >
> >>> > > > Since I will remove the original Jenkinsfile, this might cause
> >>> conflicts
> >>> > > > with ongoing efforts that try to change the Jenkinsfile. This
> >>> poses the
> >>> > > > risk that I might forget to port a change. Thus, I'd like to ask
> >>> all
> >>> > > > contributors to wait with changes of Jenkinsfile and would like
> to
> >>> > > request
> >>> > > > fellow-committers to wait with merging any Jenkinsfile-related
> PRs
> >>> until
> >>> > > > further notice.
> >>> > > >
> >>> > > > I expect to finish this refactor until the end of the week.
> Please
> >>> don't
> >>> > > > hesitate to ask if you've got further questions.
> >>> > > >
> >>> > > > Please excuse any caused inconveniences.
> >>> > > >
> >>> > > > Best regards,
> >>> > > > Marco
> >>> > > >
> >>> > >
> >>>
> >>
>
Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!
Posted by Marco de Abreu <ma...@googlemail.com.INVALID>.
Hello,
the PR has been merged and I've created the new pipelines at [1]. You can
see the new reports if you have a look at this example PR at [2].
The new status messages will be the ones starting with
"ci/jenkins/mxnet-validation/".
This now allows you to retrigger specific pipelines if they fail. For
example, if you're interested in the website pipeline, you can now go to
[3] and just retrigger that instead of running the entire suite. Whenever
there's a new commit, all pipelines will still be scheduled as before (the
overall behaviour or coverage of our pipeline did not change, I just
decoupled them and increased the usability).
The next step will be the deprecation of the main Jenkinsfile (the one
which reports the status as "continuous-integration/jenkins/pr-merge") and
requesting these new statuses to be marked as required (protected master
branch). Since we have to change some reporting tools to point to the new
jobs and I'd like to observe the stability for some time, this will take
some times.
You can now resume changes in the Jenkinsfiles. But please do not modify
the Jenkinsfile in the root directory but instead the ones at [4]. The
nightly Jenkinsfiles (or basically all Jenkinsfiles that are not part of
the main pipeline) have not been migrated yet and I will do that at a later
point in time.
Best regards,
Marco
[1]: http://jenkins.mxnet-ci.amazon-ml.com/job/mxnet-validation/
[2]: https://github.com/apache/incubator-mxnet/pull/13352
[3]:
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwebsite/detail/PR-13352/1/pipeline
[4]: https://github.com/apache/incubator-mxnet/tree/master/ci/jenkins
On Tue, Nov 20, 2018 at 9:33 PM Marco de Abreu <ma...@googlemail.com>
wrote:
> I have just submitted my PR at
> https://github.com/apache/incubator-mxnet/pull/13344. Test jobs are
> available at
> http://jenkins.mxnet-ci-dev.amazon-ml.com/view/test-marco-mxnet/.
>
> As soon as I'm done with my tests, I will mark it as ready for review.
>
> Best regards,
> Marco
>
> On Tue, Nov 20, 2018 at 9:09 PM Marco de Abreu <
> marco.g.abreu@googlemail.com> wrote:
>
>> Thanks, Pedro!
>>
>> I have also been looking into that issue, but it seems like this would
>> require changes in the groovy interpreter of Jenkins. From what I can tell,
>> a refactor will give us multiple benefits (clarity and speed) aside from
>> resolving this issue.
>>
>> Best regards,
>> Marco
>>
>> Am Di., 20. Nov. 2018, 19:54 hat Pedro Larroy <
>> pedro.larroy.lists@gmail.com> geschrieben:
>>
>>> I think this is a big problem, which has blocked us before. I want to
>>> point out that you are doing a great thing by avoiding everyone
>>> getting blocked by refactoring the pipelines.
>>>
>>> My concern is that we are kicking the can down the road and not
>>> addressing the root cause of the problem with is known
>>> https://issues.jenkins-ci.org/browse/JENKINS-37984
>>>
>>> Pedro.
>>>
>>>
>>> On Tue, Nov 20, 2018 at 6:08 PM Marco de Abreu
>>> <ma...@googlemail.com.invalid> wrote:
>>> >
>>> > Hello Steffen,
>>> >
>>> > no, there won't be any impact on the PR process or nightly regressions.
>>> > Only the reporting will have to be updated with the new job links, but
>>> that
>>> > should be a minor issue. To avoid any outage, I have been thinking
>>> about
>>> > running both versions in parallel.
>>> >
>>> > Best regards,
>>> > Marco
>>> >
>>> >
>>> >
>>> > On Tue, Nov 20, 2018 at 5:53 PM Steffen Rochel <
>>> steffenrochel@gmail.com>
>>> > wrote:
>>> >
>>> > > Hi Marco - is there any impact on reporting, the PR process or
>>> nightly
>>> > > regression beside reduction in TAT? If yes, please elaborate.
>>> > > Steffen
>>> > >
>>> > > On Tue, Nov 20, 2018 at 8:05 AM Marco de Abreu
>>> > > <ma...@googlemail.com.invalid> wrote:
>>> > >
>>> > > > Hello,
>>> > > >
>>> > > > we ran into issues around the maximum filesize of the Jenkinsfile
>>> a few
>>> > > > times already. In order to resolve this issue, I'd like to combine
>>> this
>>> > > > with some refactors I have planned for quite some time.
>>> > > >
>>> > > > The idea is basically to move away from one big Jenkinsfile and
>>> instead
>>> > > > split it into separate jobs that run in parallel and report their
>>> status
>>> > > > individually. Besides avoiding the size restriction, this will
>>> greatly
>>> > > > speed up the PR validation process by reducing the critical path.
>>> Instead
>>> > > > of having to wait for every single step within a stage to finish
>>> before
>>> > > the
>>> > > > next stage (e.g. tests) is getting executed, these pipelines would
>>> now be
>>> > > > able to move forward individually. I'm still in the process of
>>> > > refactoring
>>> > > > and can't provide any numbers or documentation at this time, but I
>>> would
>>> > > > like to announce this early on to avoid conflicts:
>>> > > >
>>> > > > Since I will remove the original Jenkinsfile, this might cause
>>> conflicts
>>> > > > with ongoing efforts that try to change the Jenkinsfile. This
>>> poses the
>>> > > > risk that I might forget to port a change. Thus, I'd like to ask
>>> all
>>> > > > contributors to wait with changes of Jenkinsfile and would like to
>>> > > request
>>> > > > fellow-committers to wait with merging any Jenkinsfile-related PRs
>>> until
>>> > > > further notice.
>>> > > >
>>> > > > I expect to finish this refactor until the end of the week. Please
>>> don't
>>> > > > hesitate to ask if you've got further questions.
>>> > > >
>>> > > > Please excuse any caused inconveniences.
>>> > > >
>>> > > > Best regards,
>>> > > > Marco
>>> > > >
>>> > >
>>>
>>
Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!
Posted by Marco de Abreu <ma...@googlemail.com.INVALID>.
I have just submitted my PR at
https://github.com/apache/incubator-mxnet/pull/13344. Test jobs are
available at
http://jenkins.mxnet-ci-dev.amazon-ml.com/view/test-marco-mxnet/.
As soon as I'm done with my tests, I will mark it as ready for review.
Best regards,
Marco
On Tue, Nov 20, 2018 at 9:09 PM Marco de Abreu <ma...@googlemail.com>
wrote:
> Thanks, Pedro!
>
> I have also been looking into that issue, but it seems like this would
> require changes in the groovy interpreter of Jenkins. From what I can tell,
> a refactor will give us multiple benefits (clarity and speed) aside from
> resolving this issue.
>
> Best regards,
> Marco
>
> Am Di., 20. Nov. 2018, 19:54 hat Pedro Larroy <
> pedro.larroy.lists@gmail.com> geschrieben:
>
>> I think this is a big problem, which has blocked us before. I want to
>> point out that you are doing a great thing by avoiding everyone
>> getting blocked by refactoring the pipelines.
>>
>> My concern is that we are kicking the can down the road and not
>> addressing the root cause of the problem with is known
>> https://issues.jenkins-ci.org/browse/JENKINS-37984
>>
>> Pedro.
>>
>>
>> On Tue, Nov 20, 2018 at 6:08 PM Marco de Abreu
>> <ma...@googlemail.com.invalid> wrote:
>> >
>> > Hello Steffen,
>> >
>> > no, there won't be any impact on the PR process or nightly regressions.
>> > Only the reporting will have to be updated with the new job links, but
>> that
>> > should be a minor issue. To avoid any outage, I have been thinking about
>> > running both versions in parallel.
>> >
>> > Best regards,
>> > Marco
>> >
>> >
>> >
>> > On Tue, Nov 20, 2018 at 5:53 PM Steffen Rochel <steffenrochel@gmail.com
>> >
>> > wrote:
>> >
>> > > Hi Marco - is there any impact on reporting, the PR process or nightly
>> > > regression beside reduction in TAT? If yes, please elaborate.
>> > > Steffen
>> > >
>> > > On Tue, Nov 20, 2018 at 8:05 AM Marco de Abreu
>> > > <ma...@googlemail.com.invalid> wrote:
>> > >
>> > > > Hello,
>> > > >
>> > > > we ran into issues around the maximum filesize of the Jenkinsfile a
>> few
>> > > > times already. In order to resolve this issue, I'd like to combine
>> this
>> > > > with some refactors I have planned for quite some time.
>> > > >
>> > > > The idea is basically to move away from one big Jenkinsfile and
>> instead
>> > > > split it into separate jobs that run in parallel and report their
>> status
>> > > > individually. Besides avoiding the size restriction, this will
>> greatly
>> > > > speed up the PR validation process by reducing the critical path.
>> Instead
>> > > > of having to wait for every single step within a stage to finish
>> before
>> > > the
>> > > > next stage (e.g. tests) is getting executed, these pipelines would
>> now be
>> > > > able to move forward individually. I'm still in the process of
>> > > refactoring
>> > > > and can't provide any numbers or documentation at this time, but I
>> would
>> > > > like to announce this early on to avoid conflicts:
>> > > >
>> > > > Since I will remove the original Jenkinsfile, this might cause
>> conflicts
>> > > > with ongoing efforts that try to change the Jenkinsfile. This poses
>> the
>> > > > risk that I might forget to port a change. Thus, I'd like to ask all
>> > > > contributors to wait with changes of Jenkinsfile and would like to
>> > > request
>> > > > fellow-committers to wait with merging any Jenkinsfile-related PRs
>> until
>> > > > further notice.
>> > > >
>> > > > I expect to finish this refactor until the end of the week. Please
>> don't
>> > > > hesitate to ask if you've got further questions.
>> > > >
>> > > > Please excuse any caused inconveniences.
>> > > >
>> > > > Best regards,
>> > > > Marco
>> > > >
>> > >
>>
>
Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!
Posted by Marco de Abreu <ma...@googlemail.com.INVALID>.
Thanks, Pedro!
I have also been looking into that issue, but it seems like this would
require changes in the groovy interpreter of Jenkins. From what I can tell,
a refactor will give us multiple benefits (clarity and speed) aside from
resolving this issue.
Best regards,
Marco
Am Di., 20. Nov. 2018, 19:54 hat Pedro Larroy <pe...@gmail.com>
geschrieben:
> I think this is a big problem, which has blocked us before. I want to
> point out that you are doing a great thing by avoiding everyone
> getting blocked by refactoring the pipelines.
>
> My concern is that we are kicking the can down the road and not
> addressing the root cause of the problem with is known
> https://issues.jenkins-ci.org/browse/JENKINS-37984
>
> Pedro.
>
>
> On Tue, Nov 20, 2018 at 6:08 PM Marco de Abreu
> <ma...@googlemail.com.invalid> wrote:
> >
> > Hello Steffen,
> >
> > no, there won't be any impact on the PR process or nightly regressions.
> > Only the reporting will have to be updated with the new job links, but
> that
> > should be a minor issue. To avoid any outage, I have been thinking about
> > running both versions in parallel.
> >
> > Best regards,
> > Marco
> >
> >
> >
> > On Tue, Nov 20, 2018 at 5:53 PM Steffen Rochel <st...@gmail.com>
> > wrote:
> >
> > > Hi Marco - is there any impact on reporting, the PR process or nightly
> > > regression beside reduction in TAT? If yes, please elaborate.
> > > Steffen
> > >
> > > On Tue, Nov 20, 2018 at 8:05 AM Marco de Abreu
> > > <ma...@googlemail.com.invalid> wrote:
> > >
> > > > Hello,
> > > >
> > > > we ran into issues around the maximum filesize of the Jenkinsfile a
> few
> > > > times already. In order to resolve this issue, I'd like to combine
> this
> > > > with some refactors I have planned for quite some time.
> > > >
> > > > The idea is basically to move away from one big Jenkinsfile and
> instead
> > > > split it into separate jobs that run in parallel and report their
> status
> > > > individually. Besides avoiding the size restriction, this will
> greatly
> > > > speed up the PR validation process by reducing the critical path.
> Instead
> > > > of having to wait for every single step within a stage to finish
> before
> > > the
> > > > next stage (e.g. tests) is getting executed, these pipelines would
> now be
> > > > able to move forward individually. I'm still in the process of
> > > refactoring
> > > > and can't provide any numbers or documentation at this time, but I
> would
> > > > like to announce this early on to avoid conflicts:
> > > >
> > > > Since I will remove the original Jenkinsfile, this might cause
> conflicts
> > > > with ongoing efforts that try to change the Jenkinsfile. This poses
> the
> > > > risk that I might forget to port a change. Thus, I'd like to ask all
> > > > contributors to wait with changes of Jenkinsfile and would like to
> > > request
> > > > fellow-committers to wait with merging any Jenkinsfile-related PRs
> until
> > > > further notice.
> > > >
> > > > I expect to finish this refactor until the end of the week. Please
> don't
> > > > hesitate to ask if you've got further questions.
> > > >
> > > > Please excuse any caused inconveniences.
> > > >
> > > > Best regards,
> > > > Marco
> > > >
> > >
>
Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!
Posted by Pedro Larroy <pe...@gmail.com>.
I think this is a big problem, which has blocked us before. I want to
point out that you are doing a great thing by avoiding everyone
getting blocked by refactoring the pipelines.
My concern is that we are kicking the can down the road and not
addressing the root cause of the problem with is known
https://issues.jenkins-ci.org/browse/JENKINS-37984
Pedro.
On Tue, Nov 20, 2018 at 6:08 PM Marco de Abreu
<ma...@googlemail.com.invalid> wrote:
>
> Hello Steffen,
>
> no, there won't be any impact on the PR process or nightly regressions.
> Only the reporting will have to be updated with the new job links, but that
> should be a minor issue. To avoid any outage, I have been thinking about
> running both versions in parallel.
>
> Best regards,
> Marco
>
>
>
> On Tue, Nov 20, 2018 at 5:53 PM Steffen Rochel <st...@gmail.com>
> wrote:
>
> > Hi Marco - is there any impact on reporting, the PR process or nightly
> > regression beside reduction in TAT? If yes, please elaborate.
> > Steffen
> >
> > On Tue, Nov 20, 2018 at 8:05 AM Marco de Abreu
> > <ma...@googlemail.com.invalid> wrote:
> >
> > > Hello,
> > >
> > > we ran into issues around the maximum filesize of the Jenkinsfile a few
> > > times already. In order to resolve this issue, I'd like to combine this
> > > with some refactors I have planned for quite some time.
> > >
> > > The idea is basically to move away from one big Jenkinsfile and instead
> > > split it into separate jobs that run in parallel and report their status
> > > individually. Besides avoiding the size restriction, this will greatly
> > > speed up the PR validation process by reducing the critical path. Instead
> > > of having to wait for every single step within a stage to finish before
> > the
> > > next stage (e.g. tests) is getting executed, these pipelines would now be
> > > able to move forward individually. I'm still in the process of
> > refactoring
> > > and can't provide any numbers or documentation at this time, but I would
> > > like to announce this early on to avoid conflicts:
> > >
> > > Since I will remove the original Jenkinsfile, this might cause conflicts
> > > with ongoing efforts that try to change the Jenkinsfile. This poses the
> > > risk that I might forget to port a change. Thus, I'd like to ask all
> > > contributors to wait with changes of Jenkinsfile and would like to
> > request
> > > fellow-committers to wait with merging any Jenkinsfile-related PRs until
> > > further notice.
> > >
> > > I expect to finish this refactor until the end of the week. Please don't
> > > hesitate to ask if you've got further questions.
> > >
> > > Please excuse any caused inconveniences.
> > >
> > > Best regards,
> > > Marco
> > >
> >
Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!
Posted by Marco de Abreu <ma...@googlemail.com.INVALID>.
Hello Steffen,
no, there won't be any impact on the PR process or nightly regressions.
Only the reporting will have to be updated with the new job links, but that
should be a minor issue. To avoid any outage, I have been thinking about
running both versions in parallel.
Best regards,
Marco
On Tue, Nov 20, 2018 at 5:53 PM Steffen Rochel <st...@gmail.com>
wrote:
> Hi Marco - is there any impact on reporting, the PR process or nightly
> regression beside reduction in TAT? If yes, please elaborate.
> Steffen
>
> On Tue, Nov 20, 2018 at 8:05 AM Marco de Abreu
> <ma...@googlemail.com.invalid> wrote:
>
> > Hello,
> >
> > we ran into issues around the maximum filesize of the Jenkinsfile a few
> > times already. In order to resolve this issue, I'd like to combine this
> > with some refactors I have planned for quite some time.
> >
> > The idea is basically to move away from one big Jenkinsfile and instead
> > split it into separate jobs that run in parallel and report their status
> > individually. Besides avoiding the size restriction, this will greatly
> > speed up the PR validation process by reducing the critical path. Instead
> > of having to wait for every single step within a stage to finish before
> the
> > next stage (e.g. tests) is getting executed, these pipelines would now be
> > able to move forward individually. I'm still in the process of
> refactoring
> > and can't provide any numbers or documentation at this time, but I would
> > like to announce this early on to avoid conflicts:
> >
> > Since I will remove the original Jenkinsfile, this might cause conflicts
> > with ongoing efforts that try to change the Jenkinsfile. This poses the
> > risk that I might forget to port a change. Thus, I'd like to ask all
> > contributors to wait with changes of Jenkinsfile and would like to
> request
> > fellow-committers to wait with merging any Jenkinsfile-related PRs until
> > further notice.
> >
> > I expect to finish this refactor until the end of the week. Please don't
> > hesitate to ask if you've got further questions.
> >
> > Please excuse any caused inconveniences.
> >
> > Best regards,
> > Marco
> >
>
Re: Splitting Jenkins pipelines - stop changes to Jenkinsfiles!
Posted by Steffen Rochel <st...@gmail.com>.
Hi Marco - is there any impact on reporting, the PR process or nightly
regression beside reduction in TAT? If yes, please elaborate.
Steffen
On Tue, Nov 20, 2018 at 8:05 AM Marco de Abreu
<ma...@googlemail.com.invalid> wrote:
> Hello,
>
> we ran into issues around the maximum filesize of the Jenkinsfile a few
> times already. In order to resolve this issue, I'd like to combine this
> with some refactors I have planned for quite some time.
>
> The idea is basically to move away from one big Jenkinsfile and instead
> split it into separate jobs that run in parallel and report their status
> individually. Besides avoiding the size restriction, this will greatly
> speed up the PR validation process by reducing the critical path. Instead
> of having to wait for every single step within a stage to finish before the
> next stage (e.g. tests) is getting executed, these pipelines would now be
> able to move forward individually. I'm still in the process of refactoring
> and can't provide any numbers or documentation at this time, but I would
> like to announce this early on to avoid conflicts:
>
> Since I will remove the original Jenkinsfile, this might cause conflicts
> with ongoing efforts that try to change the Jenkinsfile. This poses the
> risk that I might forget to port a change. Thus, I'd like to ask all
> contributors to wait with changes of Jenkinsfile and would like to request
> fellow-committers to wait with merging any Jenkinsfile-related PRs until
> further notice.
>
> I expect to finish this refactor until the end of the week. Please don't
> hesitate to ask if you've got further questions.
>
> Please excuse any caused inconveniences.
>
> Best regards,
> Marco
>