You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mxnet.apache.org by "Larroy, Pedro" <pl...@amazon.de> on 2017/09/14 10:03:33 UTC
Testing and continuous integration of MXNet on devices / Android etc.
Hi there
As a part of my daily job in Amazon which involves working with MXNet. Together with my team we have setup a dockerized multi-platform build for arm architectures including Android.
It has anonymous read access so anyone can check the build status on the multiple platforms:
( http://ci.mxnet.amazon-ml.com/ )
I think it would be useful to have this build feedback on PRs and commits on the Apache MXNet repository, and we are happy to contribute with this effort to the community.
What I would like to do is to be able to set the build status on PRs and commits on Github so contributors sending PRs can see if their changes break the builds on devices, for that I need write access to the MXNet repository with a dedicated account that will be used exclusively for this and not by humans to prevent any potential errors. https://github.com/MXNetEdgeBot/
You can see the build files in the following PR: https://github.com/apache/incubator-mxnet/pull/7792 we will extend this further with extensive testing on the different platforms, performance regressions and more.
Because of the potential load of emulating and testing on this many architectures and flavors and for moving fast, for the moment I think it would be better to iterate on our own infrastructure and then if there’s the wish integrate with the testing infra in Apache do it.
Any feedback or comments on this? Do you like the idea?
Constructive feedback is welcome.
Thank you very much.
Pedro.
Amazon Development Center Germany GmbH
Berlin - Dresden - Aachen
main office: Krausenstr. 38, 10117 Berlin
Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger
Ust-ID: DE289237879
Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
Re: Testing and continuous integration of MXNet on devices / Android etc.
Posted by Naveen Swamy <mn...@gmail.com>.
Hi Pedro,
I think we should integrate your build steps into the Apache MXNet CI
System. Your PR is creating a set of ARM Docker files, I don't see the
build steps. Can you please create your job using JenkinsFile like
https://github.com/apache/incubator-mxnet/blob/master/Jenkinsfile.
I have a few questions:
1. How much time does your build job take?
2. I see a few ARM architectures in the PR? I think its best to build on a
common subset(preferably 1) of the architecture in the PR build. Do you
think the complete build can be run every night instead?
3. What kind of infrastructure does your build require? Can they be run on
AWS? The current set of slaves are run on AWS.
4. Please ensure there is enough documentation for continued
maintainability?
Thanks, Naveen
On Thu, Sep 14, 2017 at 10:54 PM, Steffen Rochel <st...@gmail.com>
wrote:
> Hi Pedro - would it not be simpler to add your tests like another stage in
> the existing Jenkins pipeline, e.g.
> https://builds.apache.org/blue/organizations/jenkins/
> incubator-mxnet/detail/PR-7902/1/pipeline
> instead of having yet another jenkins master for MXNet.
> Yes, we would need Apache infra help to setup another Jenkins slave.
> However, the slave itself can then be managed by the community.
>
> We should consider Naveen's proposal when to invoke the the whole pipeline
> for a "completed" PR vs. sanity checks on PR's still in discussion and
> review to minimize system load.
>
> Steffen
>
> On Thu, Sep 14, 2017 at 9:39 AM Henri Yandell <ba...@apache.org> wrote:
>
> > I've a feeling that you don't need write access but a token or oauth app
> > that is setup to edit the build status? I know that's what TravisCI asks
> > for.
> >
> > Similar issue of needing an account to access (and infra say no bot
> > accounts), but I think this can be done with lesser permissions to that
> > account than write.
> >
> > On Thu, Sep 14, 2017 at 03:03 Larroy, Pedro <pl...@amazon.de> wrote:
> >
> > > Hi there
> > >
> > > As a part of my daily job in Amazon which involves working with MXNet.
> > > Together with my team we have setup a dockerized multi-platform build
> for
> > > arm architectures including Android.
> > >
> > > It has anonymous read access so anyone can check the build status on
> the
> > > multiple platforms:
> > >
> > > ( http://ci.mxnet.amazon-ml.com/ )
> > >
> > > I think it would be useful to have this build feedback on PRs and
> commits
> > > on the Apache MXNet repository, and we are happy to contribute with
> this
> > > effort to the community.
> > >
> > > What I would like to do is to be able to set the build status on PRs
> and
> > > commits on Github so contributors sending PRs can see if their changes
> > > break the builds on devices, for that I need write access to the MXNet
> > > repository with a dedicated account that will be used exclusively for
> > this
> > > and not by humans to prevent any potential errors.
> > > https://github.com/MXNetEdgeBot/
> > >
> > > You can see the build files in the following PR:
> > > https://github.com/apache/incubator-mxnet/pull/7792 we will extend
> this
> > > further with extensive testing on the different platforms, performance
> > > regressions and more.
> > >
> > > Because of the potential load of emulating and testing on this many
> > > architectures and flavors and for moving fast, for the moment I think
> it
> > > would be better to iterate on our own infrastructure and then if
> there’s
> > > the wish integrate with the testing infra in Apache do it.
> > >
> > > Any feedback or comments on this? Do you like the idea?
> > > Constructive feedback is welcome.
> > >
> > > Thank you very much.
> > >
> > > Pedro.
> > >
> > > Amazon Development Center Germany Gm
> > <https://maps.google.com/?q=ter+Germany+Gm&entry=gmail&source=g>bH
> > > Berlin - Dresden - Aachen
> > > main office: Krausenstr. 38
> > > <https://maps.google.com/?q=Krausenstr.+38&entry=gmail&source=g>,
> 10117
> > > Berlin
> > > Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger
> > > Ust-ID: DE289237879
> > > Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
> > >
> >
>
Re: Testing and continuous integration of MXNet on devices / Android etc.
Posted by Steffen Rochel <st...@gmail.com>.
Hi Pedro - would it not be simpler to add your tests like another stage in
the existing Jenkins pipeline, e.g.
https://builds.apache.org/blue/organizations/jenkins/incubator-mxnet/detail/PR-7902/1/pipeline
instead of having yet another jenkins master for MXNet.
Yes, we would need Apache infra help to setup another Jenkins slave.
However, the slave itself can then be managed by the community.
We should consider Naveen's proposal when to invoke the the whole pipeline
for a "completed" PR vs. sanity checks on PR's still in discussion and
review to minimize system load.
Steffen
On Thu, Sep 14, 2017 at 9:39 AM Henri Yandell <ba...@apache.org> wrote:
> I've a feeling that you don't need write access but a token or oauth app
> that is setup to edit the build status? I know that's what TravisCI asks
> for.
>
> Similar issue of needing an account to access (and infra say no bot
> accounts), but I think this can be done with lesser permissions to that
> account than write.
>
> On Thu, Sep 14, 2017 at 03:03 Larroy, Pedro <pl...@amazon.de> wrote:
>
> > Hi there
> >
> > As a part of my daily job in Amazon which involves working with MXNet.
> > Together with my team we have setup a dockerized multi-platform build for
> > arm architectures including Android.
> >
> > It has anonymous read access so anyone can check the build status on the
> > multiple platforms:
> >
> > ( http://ci.mxnet.amazon-ml.com/ )
> >
> > I think it would be useful to have this build feedback on PRs and commits
> > on the Apache MXNet repository, and we are happy to contribute with this
> > effort to the community.
> >
> > What I would like to do is to be able to set the build status on PRs and
> > commits on Github so contributors sending PRs can see if their changes
> > break the builds on devices, for that I need write access to the MXNet
> > repository with a dedicated account that will be used exclusively for
> this
> > and not by humans to prevent any potential errors.
> > https://github.com/MXNetEdgeBot/
> >
> > You can see the build files in the following PR:
> > https://github.com/apache/incubator-mxnet/pull/7792 we will extend this
> > further with extensive testing on the different platforms, performance
> > regressions and more.
> >
> > Because of the potential load of emulating and testing on this many
> > architectures and flavors and for moving fast, for the moment I think it
> > would be better to iterate on our own infrastructure and then if there’s
> > the wish integrate with the testing infra in Apache do it.
> >
> > Any feedback or comments on this? Do you like the idea?
> > Constructive feedback is welcome.
> >
> > Thank you very much.
> >
> > Pedro.
> >
> > Amazon Development Center Germany Gm
> <https://maps.google.com/?q=ter+Germany+Gm&entry=gmail&source=g>bH
> > Berlin - Dresden - Aachen
> > main office: Krausenstr. 38
> > <https://maps.google.com/?q=Krausenstr.+38&entry=gmail&source=g>, 10117
> > Berlin
> > Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger
> > Ust-ID: DE289237879
> > Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
> >
>
Re: Testing and continuous integration of MXNet on devices / Android etc.
Posted by Henri Yandell <ba...@apache.org>.
I've a feeling that you don't need write access but a token or oauth app
that is setup to edit the build status? I know that's what TravisCI asks
for.
Similar issue of needing an account to access (and infra say no bot
accounts), but I think this can be done with lesser permissions to that
account than write.
On Thu, Sep 14, 2017 at 03:03 Larroy, Pedro <pl...@amazon.de> wrote:
> Hi there
>
> As a part of my daily job in Amazon which involves working with MXNet.
> Together with my team we have setup a dockerized multi-platform build for
> arm architectures including Android.
>
> It has anonymous read access so anyone can check the build status on the
> multiple platforms:
>
> ( http://ci.mxnet.amazon-ml.com/ )
>
> I think it would be useful to have this build feedback on PRs and commits
> on the Apache MXNet repository, and we are happy to contribute with this
> effort to the community.
>
> What I would like to do is to be able to set the build status on PRs and
> commits on Github so contributors sending PRs can see if their changes
> break the builds on devices, for that I need write access to the MXNet
> repository with a dedicated account that will be used exclusively for this
> and not by humans to prevent any potential errors.
> https://github.com/MXNetEdgeBot/
>
> You can see the build files in the following PR:
> https://github.com/apache/incubator-mxnet/pull/7792 we will extend this
> further with extensive testing on the different platforms, performance
> regressions and more.
>
> Because of the potential load of emulating and testing on this many
> architectures and flavors and for moving fast, for the moment I think it
> would be better to iterate on our own infrastructure and then if there’s
> the wish integrate with the testing infra in Apache do it.
>
> Any feedback or comments on this? Do you like the idea?
> Constructive feedback is welcome.
>
> Thank you very much.
>
> Pedro.
>
> Amazon Development Center Germany GmbH
> Berlin - Dresden - Aachen
> main office: Krausenstr. 38
> <https://maps.google.com/?q=Krausenstr.+38&entry=gmail&source=g>, 10117
> Berlin
> Geschaeftsfuehrer: Dr. Ralf Herbrich, Christian Schlaeger
> Ust-ID: DE289237879
> Eingetragen am Amtsgericht Charlottenburg HRB 149173 B
>