You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mxnet.apache.org by Pedro Larroy <pe...@gmail.com> on 2018/11/02 19:02:06 UTC
[Announce] Virtualized testing on ARM with Qemu and Docker
Hi MXNet community
AI on MCUs can enable cheaper, lower power, better privacy and lower
latency applications. There’s an estimate of more than 20 billion connected
devices to be deployed in 2020 and a part of them will do some amount of AI
/ ML tasks. Testing in embedded devices is very challenging and expensive
due to logistics, tooling and resource constraints. Here I would like to
announce a contribution I have done using the free and open-source emulator
QEMU and Docker to perform hardware virtualization and test MXNet on edge
devices, specifically to test the MXNet artifacts on ARM such as Pip wheels
and run unit tests.
There's small instructions to run a virtualized environment on the bottom
of the README.md in the ci folder:
https://github.com/apache/incubator-mxnet/tree/master/ci#testing-with-qemu
I would encourage you to give it a try and report any comments or feedback.
The plan is to integrate it into nightly testing. We would need to narrow
down a bit the scope of testing since still the full suite is just too big
and resource intensive to finish in a reasonable time.
My idea would be to split the unit tests into different suites such as core
/ gluon / extended. Do you have any suggestions for this split?
As a cool thing to try, you can execute the following command which will
give you a shell in an ARM VM (also sshable via ssh -p2222 qemu@localhost)
so you can use and debug MXNet in ARM:
ci/build.py -p test.arm_qemu -b && docker run -p2222:2222 -ti
mxnetci/build.test.arm_qemu
How cool is that? If you are curious or want to hack on it, have a look
at the qemu folders under ci.
Pedro.
Re: [Announce] Virtualized testing on ARM with Qemu and Docker
Posted by Pedro Larroy <pe...@gmail.com>.
Thanks, good idea.
Pedro.
On Mon, Dec 3, 2018 at 2:15 AM Steffen Rochel <st...@gmail.com> wrote:
>
> Thanks Pedro, nice work. Please consider adding instructions also to
> https://cwiki.apache.org/confluence/display/MXNET/MXNet+Development+Guide
>
> On Fri, Nov 2, 2018 at 2:35 PM Marco de Abreu
> <ma...@googlemail.com.invalid> wrote:
>
> > Great work, Pedro! This will help to have a consistent quality on all ARM
> > devices.
> >
> > -Marco
> >
> > Am Fr., 2. Nov. 2018, 20:02 hat Pedro Larroy <pedro.larroy.lists@gmail.com
> > >
> > geschrieben:
> >
> > > Hi MXNet community
> > >
> > > AI on MCUs can enable cheaper, lower power, better privacy and lower
> > > latency applications. There’s an estimate of more than 20 billion
> > connected
> > > devices to be deployed in 2020 and a part of them will do some amount of
> > AI
> > > / ML tasks. Testing in embedded devices is very challenging and expensive
> > > due to logistics, tooling and resource constraints. Here I would like to
> > > announce a contribution I have done using the free and open-source
> > emulator
> > > QEMU and Docker to perform hardware virtualization and test MXNet on edge
> > > devices, specifically to test the MXNet artifacts on ARM such as Pip
> > wheels
> > > and run unit tests.
> > >
> > > There's small instructions to run a virtualized environment on the bottom
> > > of the README.md in the ci folder:
> > >
> > >
> > https://github.com/apache/incubator-mxnet/tree/master/ci#testing-with-qemu
> > >
> > > I would encourage you to give it a try and report any comments or
> > feedback.
> > > The plan is to integrate it into nightly testing. We would need to narrow
> > > down a bit the scope of testing since still the full suite is just too
> > big
> > > and resource intensive to finish in a reasonable time.
> > >
> > > My idea would be to split the unit tests into different suites such as
> > core
> > > / gluon / extended. Do you have any suggestions for this split?
> > >
> > > As a cool thing to try, you can execute the following command which will
> > > give you a shell in an ARM VM (also sshable via ssh -p2222 qemu@localhost
> > )
> > > so you can use and debug MXNet in ARM:
> > >
> > > ci/build.py -p test.arm_qemu -b && docker run -p2222:2222 -ti
> > > mxnetci/build.test.arm_qemu
> > >
> > >
> > > How cool is that? If you are curious or want to hack on it, have a look
> > > at the qemu folders under ci.
> > >
> > > Pedro.
> > >
> >
Re: [Announce] Virtualized testing on ARM with Qemu and Docker
Posted by Steffen Rochel <st...@gmail.com>.
Thanks Pedro, nice work. Please consider adding instructions also to
https://cwiki.apache.org/confluence/display/MXNET/MXNet+Development+Guide
On Fri, Nov 2, 2018 at 2:35 PM Marco de Abreu
<ma...@googlemail.com.invalid> wrote:
> Great work, Pedro! This will help to have a consistent quality on all ARM
> devices.
>
> -Marco
>
> Am Fr., 2. Nov. 2018, 20:02 hat Pedro Larroy <pedro.larroy.lists@gmail.com
> >
> geschrieben:
>
> > Hi MXNet community
> >
> > AI on MCUs can enable cheaper, lower power, better privacy and lower
> > latency applications. There’s an estimate of more than 20 billion
> connected
> > devices to be deployed in 2020 and a part of them will do some amount of
> AI
> > / ML tasks. Testing in embedded devices is very challenging and expensive
> > due to logistics, tooling and resource constraints. Here I would like to
> > announce a contribution I have done using the free and open-source
> emulator
> > QEMU and Docker to perform hardware virtualization and test MXNet on edge
> > devices, specifically to test the MXNet artifacts on ARM such as Pip
> wheels
> > and run unit tests.
> >
> > There's small instructions to run a virtualized environment on the bottom
> > of the README.md in the ci folder:
> >
> >
> https://github.com/apache/incubator-mxnet/tree/master/ci#testing-with-qemu
> >
> > I would encourage you to give it a try and report any comments or
> feedback.
> > The plan is to integrate it into nightly testing. We would need to narrow
> > down a bit the scope of testing since still the full suite is just too
> big
> > and resource intensive to finish in a reasonable time.
> >
> > My idea would be to split the unit tests into different suites such as
> core
> > / gluon / extended. Do you have any suggestions for this split?
> >
> > As a cool thing to try, you can execute the following command which will
> > give you a shell in an ARM VM (also sshable via ssh -p2222 qemu@localhost
> )
> > so you can use and debug MXNet in ARM:
> >
> > ci/build.py -p test.arm_qemu -b && docker run -p2222:2222 -ti
> > mxnetci/build.test.arm_qemu
> >
> >
> > How cool is that? If you are curious or want to hack on it, have a look
> > at the qemu folders under ci.
> >
> > Pedro.
> >
>
Re: [Announce] Virtualized testing on ARM with Qemu and Docker
Posted by Marco de Abreu <ma...@googlemail.com.INVALID>.
Great work, Pedro! This will help to have a consistent quality on all ARM
devices.
-Marco
Am Fr., 2. Nov. 2018, 20:02 hat Pedro Larroy <pe...@gmail.com>
geschrieben:
> Hi MXNet community
>
> AI on MCUs can enable cheaper, lower power, better privacy and lower
> latency applications. There’s an estimate of more than 20 billion connected
> devices to be deployed in 2020 and a part of them will do some amount of AI
> / ML tasks. Testing in embedded devices is very challenging and expensive
> due to logistics, tooling and resource constraints. Here I would like to
> announce a contribution I have done using the free and open-source emulator
> QEMU and Docker to perform hardware virtualization and test MXNet on edge
> devices, specifically to test the MXNet artifacts on ARM such as Pip wheels
> and run unit tests.
>
> There's small instructions to run a virtualized environment on the bottom
> of the README.md in the ci folder:
>
> https://github.com/apache/incubator-mxnet/tree/master/ci#testing-with-qemu
>
> I would encourage you to give it a try and report any comments or feedback.
> The plan is to integrate it into nightly testing. We would need to narrow
> down a bit the scope of testing since still the full suite is just too big
> and resource intensive to finish in a reasonable time.
>
> My idea would be to split the unit tests into different suites such as core
> / gluon / extended. Do you have any suggestions for this split?
>
> As a cool thing to try, you can execute the following command which will
> give you a shell in an ARM VM (also sshable via ssh -p2222 qemu@localhost)
> so you can use and debug MXNet in ARM:
>
> ci/build.py -p test.arm_qemu -b && docker run -p2222:2222 -ti
> mxnetci/build.test.arm_qemu
>
>
> How cool is that? If you are curious or want to hack on it, have a look
> at the qemu folders under ci.
>
> Pedro.
>