[DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Hello Incubator,

I'd like to propose a new incubator Apache MXNet podling.

The existing MXNet project (http://mxnet.io - 1.5 years old, 15 committers,
200 contributors) is very interested in joining Apache. MXNet is an
open-source deep learning framework that allows you to define, train, and
deploy deep neural networks on a wide array of devices, from cloud
infrastructure to mobile devices.

The wiki proposal page is located here:

  https://wiki.apache.org/incubator/MXNetProposal

I've included the text below in case anyone wants to focus on parts of it
in a reply.

Looking forward to your thoughts, and for lots of interested Apache members
to volunteer to mentor the project in addition to Sebastian and myself.

Currently the list of committers is based on the current active coders, so
we're also very interested in hearing from anyone else who is interested in
working on the project, be they current or future contributor!

Thanks,

Hen
On behalf of the MXNet project

---------

= MXNet: Apache Incubator Proposal =

== Abstract ==

MXNet is a Flexible and Efficient Library for Deep Learning

== Proposal ==

MXNet is an open-source deep learning framework that allows you to define,
train, and deploy deep neural networks on a wide array of devices, from
cloud infrastructure to mobile devices. It is highly scalable, allowing for
fast model training, and supports a flexible programming model and multiple
languages. MXNet allows you to mix symbolic and imperative programming
flavors to maximize both efficiency and productivity. MXNet is built on a
dynamic dependency scheduler that automatically parallelizes both symbolic
and imperative operations on the fly. A graph optimization layer on top of
that makes symbolic execution fast and memory efficient. The MXNet library
is portable and lightweight, and it scales to multiple GPUs and multiple
machines.

== Background ==

Deep learning is a subset of Machine learning and refers to a class of
algorithms that use a hierarchical approach with non-linearities to
discover and learn representations within data. Deep Learning has recently
become very popular due to its applicability and advancement of domains
such as Computer Vision, Speech Recognition, Natural Language Understanding
and Recommender Systems. With pervasive and cost effective cloud computing,
large labeled datasets and continued algorithmic innovation, Deep Learning
has become the one of the most popular classes of algorithms for machine
learning practitioners in recent years.

== Rational ==

The adoption of deep learning is quickly expanding from initial deep domain
experts rooted in academia to data scientists and developers working to
deploy intelligent services and products. Deep learning however has many
challenges.  These include model training time (which can take days to
weeks), programmability (not everyone writes Python or C++ and like
symbolic programming) and balancing production readiness (support for
things like failover) with development flexibility (ability to program
different ways, support for new operators and model types) and speed of
execution (fast and scalable model training).  Other frameworks excel on
some but not all of these aspects.


== Initial Goals ==

MXNet is a fairly established project on GitHub with its first code
contribution in April 2015 and roughly 200 contributors. It is used by
several large companies and some of the top research institutions on the
planet. Initial goals would be the following:

 1. Move the existing codebase(s) to Apache
 1. Integrate with the Apache development process/sign CLAs
 1. Ensure all dependencies are compliant with Apache License version 2.0
 1. Incremental development and releases per Apache guidelines
 1. Establish engineering discipline and a predictable release cadence of
high quality releases
 1. Expand the community beyond the current base of expert level users
 1. Improve usability and the overall developer/user experience
 1. Add additional functionality to address newer problem types and
algorithms


== Current Status ==

=== Meritocracy ===

The MXNet project already operates on meritocratic principles. Today, MXNet
has developers worldwide and has accepted multiple major patches from a
diverse set of contributors within both industry and academia. We would
like to follow ASF meritocratic principles to encourage more developers to
contribute in this project. We know that only active and committed
developers from a diverse set of backgrounds can make MXNet a successful
project.  We are also improving the documentation and code to help new
developers get started quickly.

=== Community ===

Acceptance into the Apache foundation would bolster the growing user and
developer community around MXNet. That community includes around 200
contributors from academia and industry. The core developers of our project
are listed in our contributors below and are also represented by logos on
the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
University of Washington and Wolfram.

=== Core Developers ===

(with GitHub logins)

 * Tianqi Chen (@tqchen)
 * Mu Li (@mli)
 * Junyuan Xie (@piiswrong)
 * Bing Xu (@antinucleon)
 * Chiyuan Zhang (@pluskid)
 * Minjie Wang (@jermainewang)
 * Naiyan Wang (@winstywang)
 * Yizhi Liu (@javelinjs)
 * Tong He (@hetong007)
 * Qiang Kou (@thirdwing)
 * Xingjian Shi (@sxjscience)

=== Alignment ===

ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
and Mahout, each of which targets a different application domain. MXNet,
being a distributed platform for large-scale deep learning, focuses on
another important domain for which there still lacks a scalable,
programmable, flexible and super fast open-source platform. The recent
success of deep learning models especially for vision and speech
recognition tasks has generated interests in both applying existing deep
learning models and in developing new ones. Thus, an open-source platform
for deep learning backed by some of the top industry and academic players
will be able to attract a large community of users and developers. MXNet is
a complex system needing many iterations of design, implementation and
testing. Apache's collaboration framework which encourages active
contribution from developers will inevitably help improve the quality of
the system, as shown in the success of Hadoop, Spark, etc. Equally
important is the community of users which helps identify real-life
applications of deep learning, and helps to evaluate the system's
performance and ease-of-use. We hope to leverage ASF for coordinating and
promoting both communities, and in return benefit the communities with
another useful tool.

== Known Risks ==

=== Orphaned products ===

Given the current level of investment in MXNet and the stakeholders using
it - the risk of the project being abandoned is minimal. Amazon, for
example, is in active development to use MXNet in many of its services and
many large corporations use it in their production applications.

=== Inexperience with Open Source ===

MXNet has existed as a healthy open source project for more than a year.
During that time, the project has attracted 200+ contributors.

=== Homogenous Developers ===

The initial list of committers and contributors includes developers from
several institutions and industry participants (see above).

=== Reliance on Salaried Developers ===

Like most open source projects, MXNet receives a substantial support from
salaried developers. A large fraction of MXNet development is supported by
graduate students at various universities in the course of research degrees
- this is more a “volunteer” relationship, since in most cases students
contribute vastly more than is necessary to immediately support research.
In addition, those working from within corporations are devoting
significant time and effort in the project - and these come from several
organizations.

=== A Excessive Fascination with the Apache Brand ===

We choose Apache not for publicity. We have two purposes. First, we hope
that Apache's known best-practices for managing a mature open source
project can help guide us.  For example, we are feeling the growing pains
of a successful open source project as we attempt a major refactor of the
internals while customers are using the system in production. We seek
guidance in communicating breaking API changes and version revisions.
Also, as our involvement from major corporations increases, we want to
assure our users that MXNet will stay open and not favor any particular
platform or environment. These are some examples of the know-how and
discipline we're hoping Apache can bring to our project.

Second, we want to leverage Apache's reputation to recruit more developers
to create a diverse community.

=== Relationship with Other Apache Products ===

Apache Mahout and Apache Spark's MLlib are general machine learning
systems. Deep learning algorithms can thus be implemented on these two
platforms as well. However, in practice, the overlap will be minimal.  Deep
learning is so computationally intensive that it often requires specialized
GPU hardware to accomplish tasks of meaningful size.  Making efficient use
of GPU hardware is complex because the hardware is so fast that the
supporting systems around it must be carefully optimized to keep the GPU
cores busy.  Extending this capability to distributed multi-GPU and
multi-host environments requires great care.  This is a critical
differentiator between MXNet and existing Apache machine learning systems.

Mahout and Spark ML-LIB follow models where their nodes run synchronously.
This is the fundamental difference to MXNet who follows the parameter
server framework. MXNet can run synchronously or asynchronously. In
addition, MXNet has optimizations for training a wide range of deep
learning models using a variety of approaches (e.g., model parallelism and
data parallelism) which makes MXNet much more efficient (near-linear
speedup on state of the art models). MXNet also supports both imperative
and symbolic approaches providing ease of programming for deep learning
algorithms.

Other Apache projects that are potentially complimentary:

Apache Arrow - read data in Apache Arrow‘s internal format from MXNet, that
would allow users to run ETL/preprocessing in Spark, save the results in
Arrow’s format and then run DL algorithms on it.

Apache Singa - MXNet and Singa are both deep learning projects, and can
benefit from a larger deep learning community at Apache.

== Documentation ==

Documentation has recently migrated to http://mxnet.io.  We continue to
refine and improve the documentation.

== Initial Source ==

We currently use Github to maintain our source code,
https://github.com/MXNet

== Source and Intellectual Property Submission Plan ==

MXNet Code is available under Apache License, Version 2.0. We will work
with the committers to get CLAs signed and review previous contributions.

== External Dependencies ==

 * required by the core code base: GCC or CLOM, Clang, any BLAS library
(ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
lib-zeromq), TBB
 * required for GPU usage: cudnn, cuda
 * required for python usage: Python 2/3
 * required for R module: R, Rcpp (GPLv2 licensing)
 * optional for image preparation and preprocessing: opencv
 * optional dependencies for additional features: torch7, numba, cython (in
NNVM branch)

Rcpt and lib-zeromq are expected to be licensing discussions.

== Cryptography ==

Not Applicable

== Required Resources ==

=== Mailing Lists ===

There is currently no mailing list.

=== Issue Tracking ===

Currently uses GitHub to track issues. Would like to continue to do so.

== Committers and Affiliations ==

 * Tianqi Chen (UW)
 * Mu Li (AWS)
 * Junyuan Xie (AWS)
 * Bing Xu (Apple)
 * Chiyuan Zhang (MIT)
 * Minjie Wang (UYU)
 * Naiyan Wang (Tusimple)
 * Yizhi Liu (Mediav)
 * Tong He (Simon Fraser University)
 * Qiang Kou (Indiana U)
 * Xingjian Shi (HKUST)

== Sponsors ==

=== Champion ===

Henri Yandell (bayard at apache.org)

=== Nominated Mentors ===

Sebastian Schelter (ssc@apache.org)


=== Sponsoring Entity ===

We are requesting the Incubator to sponsor this project.

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Joe Spisak <sp...@gmail.com>]

Please sign me up as a committer - I've been working with Mu at work on MXNet (Amazon) and would love to get more involved in the project.

Github ID: jspisak

On 2017-01-05 21:12 (-0800), Henri Yandell <ba...@apache.org> wrote: 
> Hello Incubator,
> 
> I'd like to propose a new incubator Apache MXNet podling.
> 
> The existing MXNet project (http://mxnet.io - 1.5 years old, 15 committers,
> 200 contributors) is very interested in joining Apache. MXNet is an
> open-source deep learning framework that allows you to define, train, and
> deploy deep neural networks on a wide array of devices, from cloud
> infrastructure to mobile devices.
> 
> The wiki proposal page is located here:
> 
>   https://wiki.apache.org/incubator/MXNetProposal
> 
> I've included the text below in case anyone wants to focus on parts of it
> in a reply.
> 
> Looking forward to your thoughts, and for lots of interested Apache members
> to volunteer to mentor the project in addition to Sebastian and myself.
> 
> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!
> 
> Thanks,
> 
> Hen
> On behalf of the MXNet project
> 
> ---------
> 
> = MXNet: Apache Incubator Proposal =
> 
> == Abstract ==
> 
> MXNet is a Flexible and Efficient Library for Deep Learning
> 
> == Proposal ==
> 
> MXNet is an open-source deep learning framework that allows you to define,
> train, and deploy deep neural networks on a wide array of devices, from
> cloud infrastructure to mobile devices. It is highly scalable, allowing for
> fast model training, and supports a flexible programming model and multiple
> languages. MXNet allows you to mix symbolic and imperative programming
> flavors to maximize both efficiency and productivity. MXNet is built on a
> dynamic dependency scheduler that automatically parallelizes both symbolic
> and imperative operations on the fly. A graph optimization layer on top of
> that makes symbolic execution fast and memory efficient. The MXNet library
> is portable and lightweight, and it scales to multiple GPUs and multiple
> machines.
> 
> == Background ==
> 
> Deep learning is a subset of Machine learning and refers to a class of
> algorithms that use a hierarchical approach with non-linearities to
> discover and learn representations within data. Deep Learning has recently
> become very popular due to its applicability and advancement of domains
> such as Computer Vision, Speech Recognition, Natural Language Understanding
> and Recommender Systems. With pervasive and cost effective cloud computing,
> large labeled datasets and continued algorithmic innovation, Deep Learning
> has become the one of the most popular classes of algorithms for machine
> learning practitioners in recent years.
> 
> == Rational ==
> 
> The adoption of deep learning is quickly expanding from initial deep domain
> experts rooted in academia to data scientists and developers working to
> deploy intelligent services and products. Deep learning however has many
> challenges.  These include model training time (which can take days to
> weeks), programmability (not everyone writes Python or C++ and like
> symbolic programming) and balancing production readiness (support for
> things like failover) with development flexibility (ability to program
> different ways, support for new operators and model types) and speed of
> execution (fast and scalable model training).  Other frameworks excel on
> some but not all of these aspects.
> 
> 
> == Initial Goals ==
> 
> MXNet is a fairly established project on GitHub with its first code
> contribution in April 2015 and roughly 200 contributors. It is used by
> several large companies and some of the top research institutions on the
> planet. Initial goals would be the following:
> 
>  1. Move the existing codebase(s) to Apache
>  1. Integrate with the Apache development process/sign CLAs
>  1. Ensure all dependencies are compliant with Apache License version 2.0
>  1. Incremental development and releases per Apache guidelines
>  1. Establish engineering discipline and a predictable release cadence of
> high quality releases
>  1. Expand the community beyond the current base of expert level users
>  1. Improve usability and the overall developer/user experience
>  1. Add additional functionality to address newer problem types and
> algorithms
> 
> 
> == Current Status ==
> 
> === Meritocracy ===
> 
> The MXNet project already operates on meritocratic principles. Today, MXNet
> has developers worldwide and has accepted multiple major patches from a
> diverse set of contributors within both industry and academia. We would
> like to follow ASF meritocratic principles to encourage more developers to
> contribute in this project. We know that only active and committed
> developers from a diverse set of backgrounds can make MXNet a successful
> project.  We are also improving the documentation and code to help new
> developers get started quickly.
> 
> === Community ===
> 
> Acceptance into the Apache foundation would bolster the growing user and
> developer community around MXNet. That community includes around 200
> contributors from academia and industry. The core developers of our project
> are listed in our contributors below and are also represented by logos on
> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
> University of Washington and Wolfram.
> 
> === Core Developers ===
> 
> (with GitHub logins)
> 
>  * Tianqi Chen (@tqchen)
>  * Mu Li (@mli)
>  * Junyuan Xie (@piiswrong)
>  * Bing Xu (@antinucleon)
>  * Chiyuan Zhang (@pluskid)
>  * Minjie Wang (@jermainewang)
>  * Naiyan Wang (@winstywang)
>  * Yizhi Liu (@javelinjs)
>  * Tong He (@hetong007)
>  * Qiang Kou (@thirdwing)
>  * Xingjian Shi (@sxjscience)
> 
> === Alignment ===
> 
> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
> and Mahout, each of which targets a different application domain. MXNet,
> being a distributed platform for large-scale deep learning, focuses on
> another important domain for which there still lacks a scalable,
> programmable, flexible and super fast open-source platform. The recent
> success of deep learning models especially for vision and speech
> recognition tasks has generated interests in both applying existing deep
> learning models and in developing new ones. Thus, an open-source platform
> for deep learning backed by some of the top industry and academic players
> will be able to attract a large community of users and developers. MXNet is
> a complex system needing many iterations of design, implementation and
> testing. Apache's collaboration framework which encourages active
> contribution from developers will inevitably help improve the quality of
> the system, as shown in the success of Hadoop, Spark, etc. Equally
> important is the community of users which helps identify real-life
> applications of deep learning, and helps to evaluate the system's
> performance and ease-of-use. We hope to leverage ASF for coordinating and
> promoting both communities, and in return benefit the communities with
> another useful tool.
> 
> == Known Risks ==
> 
> === Orphaned products ===
> 
> Given the current level of investment in MXNet and the stakeholders using
> it - the risk of the project being abandoned is minimal. Amazon, for
> example, is in active development to use MXNet in many of its services and
> many large corporations use it in their production applications.
> 
> === Inexperience with Open Source ===
> 
> MXNet has existed as a healthy open source project for more than a year.
> During that time, the project has attracted 200+ contributors.
> 
> === Homogenous Developers ===
> 
> The initial list of committers and contributors includes developers from
> several institutions and industry participants (see above).
> 
> === Reliance on Salaried Developers ===
> 
> Like most open source projects, MXNet receives a substantial support from
> salaried developers. A large fraction of MXNet development is supported by
> graduate students at various universities in the course of research degrees
> - this is more a \u201cvolunteer\u201d relationship, since in most cases students
> contribute vastly more than is necessary to immediately support research.
> In addition, those working from within corporations are devoting
> significant time and effort in the project - and these come from several
> organizations.
> 
> === A Excessive Fascination with the Apache Brand ===
> 
> We choose Apache not for publicity. We have two purposes. First, we hope
> that Apache's known best-practices for managing a mature open source
> project can help guide us.  For example, we are feeling the growing pains
> of a successful open source project as we attempt a major refactor of the
> internals while customers are using the system in production. We seek
> guidance in communicating breaking API changes and version revisions.
> Also, as our involvement from major corporations increases, we want to
> assure our users that MXNet will stay open and not favor any particular
> platform or environment. These are some examples of the know-how and
> discipline we're hoping Apache can bring to our project.
> 
> Second, we want to leverage Apache's reputation to recruit more developers
> to create a diverse community.
> 
> === Relationship with Other Apache Products ===
> 
> Apache Mahout and Apache Spark's MLlib are general machine learning
> systems. Deep learning algorithms can thus be implemented on these two
> platforms as well. However, in practice, the overlap will be minimal.  Deep
> learning is so computationally intensive that it often requires specialized
> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
> of GPU hardware is complex because the hardware is so fast that the
> supporting systems around it must be carefully optimized to keep the GPU
> cores busy.  Extending this capability to distributed multi-GPU and
> multi-host environments requires great care.  This is a critical
> differentiator between MXNet and existing Apache machine learning systems.
> 
> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
> This is the fundamental difference to MXNet who follows the parameter
> server framework. MXNet can run synchronously or asynchronously. In
> addition, MXNet has optimizations for training a wide range of deep
> learning models using a variety of approaches (e.g., model parallelism and
> data parallelism) which makes MXNet much more efficient (near-linear
> speedup on state of the art models). MXNet also supports both imperative
> and symbolic approaches providing ease of programming for deep learning
> algorithms.
> 
> Other Apache projects that are potentially complimentary:
> 
> Apache Arrow - read data in Apache Arrow\u2018s internal format from MXNet, that
> would allow users to run ETL/preprocessing in Spark, save the results in
> Arrow\u2019s format and then run DL algorithms on it.
> 
> Apache Singa - MXNet and Singa are both deep learning projects, and can
> benefit from a larger deep learning community at Apache.
> 
> == Documentation ==
> 
> Documentation has recently migrated to http://mxnet.io.  We continue to
> refine and improve the documentation.
> 
> == Initial Source ==
> 
> We currently use Github to maintain our source code,
> https://github.com/MXNet
> 
> == Source and Intellectual Property Submission Plan ==
> 
> MXNet Code is available under Apache License, Version 2.0. We will work
> with the committers to get CLAs signed and review previous contributions.
> 
> == External Dependencies ==
> 
>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> lib-zeromq), TBB
>  * required for GPU usage: cudnn, cuda
>  * required for python usage: Python 2/3
>  * required for R module: R, Rcpp (GPLv2 licensing)
>  * optional for image preparation and preprocessing: opencv
>  * optional dependencies for additional features: torch7, numba, cython (in
> NNVM branch)
> 
> Rcpt and lib-zeromq are expected to be licensing discussions.
> 
> == Cryptography ==
> 
> Not Applicable
> 
> == Required Resources ==
> 
> === Mailing Lists ===
> 
> There is currently no mailing list.
> 
> === Issue Tracking ===
> 
> Currently uses GitHub to track issues. Would like to continue to do so.
> 
> == Committers and Affiliations ==
> 
>  * Tianqi Chen (UW)
>  * Mu Li (AWS)
>  * Junyuan Xie (AWS)
>  * Bing Xu (Apple)
>  * Chiyuan Zhang (MIT)
>  * Minjie Wang (UYU)
>  * Naiyan Wang (Tusimple)
>  * Yizhi Liu (Mediav)
>  * Tong He (Simon Fraser University)
>  * Qiang Kou (Indiana U)
>  * Xingjian Shi (HKUST)
> 
> == Sponsors ==
> 
> === Champion ===
> 
> Henri Yandell (bayard at apache.org)
> 
> === Nominated Mentors ===
> 
> Sebastian Schelter (ssc@apache.org)
> 
> 
> === Sponsoring Entity ===
> 
> We are requesting the Incubator to sponsor this project.
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Brilliant - thank you Suneel :)

I see you're on the Incubator PMC and have added you to the wiki page.

On Thu, Jan 5, 2017 at 9:25 PM, Suneel Marthi <sm...@apache.org> wrote:

> I would like to sign up as mentor for MxNet.
>
> On Fri, Jan 6, 2017 at 12:12 AM, Henri Yandell <ba...@apache.org> wrote:
>
> > Hello Incubator,
> >
> > I'd like to propose a new incubator Apache MXNet podling.
> >
> > The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> > committers,
> > 200 contributors) is very interested in joining Apache. MXNet is an
> > open-source deep learning framework that allows you to define, train, and
> > deploy deep neural networks on a wide array of devices, from cloud
> > infrastructure to mobile devices.
> >
> > The wiki proposal page is located here:
> >
> >   https://wiki.apache.org/incubator/MXNetProposal
> >
> > I've included the text below in case anyone wants to focus on parts of it
> > in a reply.
> >
> > Looking forward to your thoughts, and for lots of interested Apache
> members
> > to volunteer to mentor the project in addition to Sebastian and myself.
> >
> > Currently the list of committers is based on the current active coders,
> so
> > we're also very interested in hearing from anyone else who is interested
> in
> > working on the project, be they current or future contributor!
> >
> > Thanks,
> >
> > Hen
> > On behalf of the MXNet project
> >
> > ---------
> >
> > = MXNet: Apache Incubator Proposal =
> >
> > == Abstract ==
> >
> > MXNet is a Flexible and Efficient Library for Deep Learning
> >
> > == Proposal ==
> >
> > MXNet is an open-source deep learning framework that allows you to
> define,
> > train, and deploy deep neural networks on a wide array of devices, from
> > cloud infrastructure to mobile devices. It is highly scalable, allowing
> for
> > fast model training, and supports a flexible programming model and
> multiple
> > languages. MXNet allows you to mix symbolic and imperative programming
> > flavors to maximize both efficiency and productivity. MXNet is built on a
> > dynamic dependency scheduler that automatically parallelizes both
> symbolic
> > and imperative operations on the fly. A graph optimization layer on top
> of
> > that makes symbolic execution fast and memory efficient. The MXNet
> library
> > is portable and lightweight, and it scales to multiple GPUs and multiple
> > machines.
> >
> > == Background ==
> >
> > Deep learning is a subset of Machine learning and refers to a class of
> > algorithms that use a hierarchical approach with non-linearities to
> > discover and learn representations within data. Deep Learning has
> recently
> > become very popular due to its applicability and advancement of domains
> > such as Computer Vision, Speech Recognition, Natural Language
> Understanding
> > and Recommender Systems. With pervasive and cost effective cloud
> computing,
> > large labeled datasets and continued algorithmic innovation, Deep
> Learning
> > has become the one of the most popular classes of algorithms for machine
> > learning practitioners in recent years.
> >
> > == Rational ==
> >
> > The adoption of deep learning is quickly expanding from initial deep
> domain
> > experts rooted in academia to data scientists and developers working to
> > deploy intelligent services and products. Deep learning however has many
> > challenges.  These include model training time (which can take days to
> > weeks), programmability (not everyone writes Python or C++ and like
> > symbolic programming) and balancing production readiness (support for
> > things like failover) with development flexibility (ability to program
> > different ways, support for new operators and model types) and speed of
> > execution (fast and scalable model training).  Other frameworks excel on
> > some but not all of these aspects.
> >
> >
> > == Initial Goals ==
> >
> > MXNet is a fairly established project on GitHub with its first code
> > contribution in April 2015 and roughly 200 contributors. It is used by
> > several large companies and some of the top research institutions on the
> > planet. Initial goals would be the following:
> >
> >  1. Move the existing codebase(s) to Apache
> >  1. Integrate with the Apache development process/sign CLAs
> >  1. Ensure all dependencies are compliant with Apache License version 2.0
> >  1. Incremental development and releases per Apache guidelines
> >  1. Establish engineering discipline and a predictable release cadence of
> > high quality releases
> >  1. Expand the community beyond the current base of expert level users
> >  1. Improve usability and the overall developer/user experience
> >  1. Add additional functionality to address newer problem types and
> > algorithms
> >
> >
> > == Current Status ==
> >
> > === Meritocracy ===
> >
> > The MXNet project already operates on meritocratic principles. Today,
> MXNet
> > has developers worldwide and has accepted multiple major patches from a
> > diverse set of contributors within both industry and academia. We would
> > like to follow ASF meritocratic principles to encourage more developers
> to
> > contribute in this project. We know that only active and committed
> > developers from a diverse set of backgrounds can make MXNet a successful
> > project.  We are also improving the documentation and code to help new
> > developers get started quickly.
> >
> > === Community ===
> >
> > Acceptance into the Apache foundation would bolster the growing user and
> > developer community around MXNet. That community includes around 200
> > contributors from academia and industry. The core developers of our
> project
> > are listed in our contributors below and are also represented by logos on
> > the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> > Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of
> Alberta,
> > University of Washington and Wolfram.
> >
> > === Core Developers ===
> >
> > (with GitHub logins)
> >
> >  * Tianqi Chen (@tqchen)
> >  * Mu Li (@mli)
> >  * Junyuan Xie (@piiswrong)
> >  * Bing Xu (@antinucleon)
> >  * Chiyuan Zhang (@pluskid)
> >  * Minjie Wang (@jermainewang)
> >  * Naiyan Wang (@winstywang)
> >  * Yizhi Liu (@javelinjs)
> >  * Tong He (@hetong007)
> >  * Qiang Kou (@thirdwing)
> >  * Xingjian Shi (@sxjscience)
> >
> > === Alignment ===
> >
> > ASF is already the home of many distributed platforms, e.g., Hadoop,
> Spark
> > and Mahout, each of which targets a different application domain. MXNet,
> > being a distributed platform for large-scale deep learning, focuses on
> > another important domain for which there still lacks a scalable,
> > programmable, flexible and super fast open-source platform. The recent
> > success of deep learning models especially for vision and speech
> > recognition tasks has generated interests in both applying existing deep
> > learning models and in developing new ones. Thus, an open-source platform
> > for deep learning backed by some of the top industry and academic players
> > will be able to attract a large community of users and developers. MXNet
> is
> > a complex system needing many iterations of design, implementation and
> > testing. Apache's collaboration framework which encourages active
> > contribution from developers will inevitably help improve the quality of
> > the system, as shown in the success of Hadoop, Spark, etc. Equally
> > important is the community of users which helps identify real-life
> > applications of deep learning, and helps to evaluate the system's
> > performance and ease-of-use. We hope to leverage ASF for coordinating and
> > promoting both communities, and in return benefit the communities with
> > another useful tool.
> >
> > == Known Risks ==
> >
> > === Orphaned products ===
> >
> > Given the current level of investment in MXNet and the stakeholders using
> > it - the risk of the project being abandoned is minimal. Amazon, for
> > example, is in active development to use MXNet in many of its services
> and
> > many large corporations use it in their production applications.
> >
> > === Inexperience with Open Source ===
> >
> > MXNet has existed as a healthy open source project for more than a year.
> > During that time, the project has attracted 200+ contributors.
> >
> > === Homogenous Developers ===
> >
> > The initial list of committers and contributors includes developers from
> > several institutions and industry participants (see above).
> >
> > === Reliance on Salaried Developers ===
> >
> > Like most open source projects, MXNet receives a substantial support from
> > salaried developers. A large fraction of MXNet development is supported
> by
> > graduate students at various universities in the course of research
> degrees
> > - this is more a “volunteer” relationship, since in most cases students
> > contribute vastly more than is necessary to immediately support research.
> > In addition, those working from within corporations are devoting
> > significant time and effort in the project - and these come from several
> > organizations.
> >
> > === A Excessive Fascination with the Apache Brand ===
> >
> > We choose Apache not for publicity. We have two purposes. First, we hope
> > that Apache's known best-practices for managing a mature open source
> > project can help guide us.  For example, we are feeling the growing pains
> > of a successful open source project as we attempt a major refactor of the
> > internals while customers are using the system in production. We seek
> > guidance in communicating breaking API changes and version revisions.
> > Also, as our involvement from major corporations increases, we want to
> > assure our users that MXNet will stay open and not favor any particular
> > platform or environment. These are some examples of the know-how and
> > discipline we're hoping Apache can bring to our project.
> >
> > Second, we want to leverage Apache's reputation to recruit more
> developers
> > to create a diverse community.
> >
> > === Relationship with Other Apache Products ===
> >
> > Apache Mahout and Apache Spark's MLlib are general machine learning
> > systems. Deep learning algorithms can thus be implemented on these two
> > platforms as well. However, in practice, the overlap will be minimal.
> Deep
> > learning is so computationally intensive that it often requires
> specialized
> > GPU hardware to accomplish tasks of meaningful size.  Making efficient
> use
> > of GPU hardware is complex because the hardware is so fast that the
> > supporting systems around it must be carefully optimized to keep the GPU
> > cores busy.  Extending this capability to distributed multi-GPU and
> > multi-host environments requires great care.  This is a critical
> > differentiator between MXNet and existing Apache machine learning
> systems.
> >
> > Mahout and Spark ML-LIB follow models where their nodes run
> synchronously.
> > This is the fundamental difference to MXNet who follows the parameter
> > server framework. MXNet can run synchronously or asynchronously. In
> > addition, MXNet has optimizations for training a wide range of deep
> > learning models using a variety of approaches (e.g., model parallelism
> and
> > data parallelism) which makes MXNet much more efficient (near-linear
> > speedup on state of the art models). MXNet also supports both imperative
> > and symbolic approaches providing ease of programming for deep learning
> > algorithms.
> >
> > Other Apache projects that are potentially complimentary:
> >
> > Apache Arrow - read data in Apache Arrow‘s internal format from MXNet,
> that
> > would allow users to run ETL/preprocessing in Spark, save the results in
> > Arrow’s format and then run DL algorithms on it.
> >
> > Apache Singa - MXNet and Singa are both deep learning projects, and can
> > benefit from a larger deep learning community at Apache.
> >
> > == Documentation ==
> >
> > Documentation has recently migrated to http://mxnet.io.  We continue to
> > refine and improve the documentation.
> >
> > == Initial Source ==
> >
> > We currently use Github to maintain our source code,
> > https://github.com/MXNet
> >
> > == Source and Intellectual Property Submission Plan ==
> >
> > MXNet Code is available under Apache License, Version 2.0. We will work
> > with the committers to get CLAs signed and review previous contributions.
> >
> > == External Dependencies ==
> >
> >  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> > (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> > lib-zeromq), TBB
> >  * required for GPU usage: cudnn, cuda
> >  * required for python usage: Python 2/3
> >  * required for R module: R, Rcpp (GPLv2 licensing)
> >  * optional for image preparation and preprocessing: opencv
> >  * optional dependencies for additional features: torch7, numba, cython
> (in
> > NNVM branch)
> >
> > Rcpt and lib-zeromq are expected to be licensing discussions.
> >
> > == Cryptography ==
> >
> > Not Applicable
> >
> > == Required Resources ==
> >
> > === Mailing Lists ===
> >
> > There is currently no mailing list.
> >
> > === Issue Tracking ===
> >
> > Currently uses GitHub to track issues. Would like to continue to do so.
> >
> > == Committers and Affiliations ==
> >
> >  * Tianqi Chen (UW)
> >  * Mu Li (AWS)
> >  * Junyuan Xie (AWS)
> >  * Bing Xu (Apple)
> >  * Chiyuan Zhang (MIT)
> >  * Minjie Wang (UYU)
> >  * Naiyan Wang (Tusimple)
> >  * Yizhi Liu (Mediav)
> >  * Tong He (Simon Fraser University)
> >  * Qiang Kou (Indiana U)
> >  * Xingjian Shi (HKUST)
> >
> > == Sponsors ==
> >
> > === Champion ===
> >
> > Henri Yandell (bayard at apache.org)
> >
> > === Nominated Mentors ===
> >
> > Sebastian Schelter (ssc@apache.org)
> >
> >
> > === Sponsoring Entity ===
> >
> > We are requesting the Incubator to sponsor this project.
> >
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Suneel Marthi <sm...@apache.org>]

I would like to sign up as mentor for MxNet.

On Fri, Jan 6, 2017 at 12:12 AM, Henri Yandell <ba...@apache.org> wrote:

> Hello Incubator,
>
> I'd like to propose a new incubator Apache MXNet podling.
>
> The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> committers,
> 200 contributors) is very interested in joining Apache. MXNet is an
> open-source deep learning framework that allows you to define, train, and
> deploy deep neural networks on a wide array of devices, from cloud
> infrastructure to mobile devices.
>
> The wiki proposal page is located here:
>
>   https://wiki.apache.org/incubator/MXNetProposal
>
> I've included the text below in case anyone wants to focus on parts of it
> in a reply.
>
> Looking forward to your thoughts, and for lots of interested Apache members
> to volunteer to mentor the project in addition to Sebastian and myself.
>
> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!
>
> Thanks,
>
> Hen
> On behalf of the MXNet project
>
> ---------
>
> = MXNet: Apache Incubator Proposal =
>
> == Abstract ==
>
> MXNet is a Flexible and Efficient Library for Deep Learning
>
> == Proposal ==
>
> MXNet is an open-source deep learning framework that allows you to define,
> train, and deploy deep neural networks on a wide array of devices, from
> cloud infrastructure to mobile devices. It is highly scalable, allowing for
> fast model training, and supports a flexible programming model and multiple
> languages. MXNet allows you to mix symbolic and imperative programming
> flavors to maximize both efficiency and productivity. MXNet is built on a
> dynamic dependency scheduler that automatically parallelizes both symbolic
> and imperative operations on the fly. A graph optimization layer on top of
> that makes symbolic execution fast and memory efficient. The MXNet library
> is portable and lightweight, and it scales to multiple GPUs and multiple
> machines.
>
> == Background ==
>
> Deep learning is a subset of Machine learning and refers to a class of
> algorithms that use a hierarchical approach with non-linearities to
> discover and learn representations within data. Deep Learning has recently
> become very popular due to its applicability and advancement of domains
> such as Computer Vision, Speech Recognition, Natural Language Understanding
> and Recommender Systems. With pervasive and cost effective cloud computing,
> large labeled datasets and continued algorithmic innovation, Deep Learning
> has become the one of the most popular classes of algorithms for machine
> learning practitioners in recent years.
>
> == Rational ==
>
> The adoption of deep learning is quickly expanding from initial deep domain
> experts rooted in academia to data scientists and developers working to
> deploy intelligent services and products. Deep learning however has many
> challenges.  These include model training time (which can take days to
> weeks), programmability (not everyone writes Python or C++ and like
> symbolic programming) and balancing production readiness (support for
> things like failover) with development flexibility (ability to program
> different ways, support for new operators and model types) and speed of
> execution (fast and scalable model training).  Other frameworks excel on
> some but not all of these aspects.
>
>
> == Initial Goals ==
>
> MXNet is a fairly established project on GitHub with its first code
> contribution in April 2015 and roughly 200 contributors. It is used by
> several large companies and some of the top research institutions on the
> planet. Initial goals would be the following:
>
>  1. Move the existing codebase(s) to Apache
>  1. Integrate with the Apache development process/sign CLAs
>  1. Ensure all dependencies are compliant with Apache License version 2.0
>  1. Incremental development and releases per Apache guidelines
>  1. Establish engineering discipline and a predictable release cadence of
> high quality releases
>  1. Expand the community beyond the current base of expert level users
>  1. Improve usability and the overall developer/user experience
>  1. Add additional functionality to address newer problem types and
> algorithms
>
>
> == Current Status ==
>
> === Meritocracy ===
>
> The MXNet project already operates on meritocratic principles. Today, MXNet
> has developers worldwide and has accepted multiple major patches from a
> diverse set of contributors within both industry and academia. We would
> like to follow ASF meritocratic principles to encourage more developers to
> contribute in this project. We know that only active and committed
> developers from a diverse set of backgrounds can make MXNet a successful
> project.  We are also improving the documentation and code to help new
> developers get started quickly.
>
> === Community ===
>
> Acceptance into the Apache foundation would bolster the growing user and
> developer community around MXNet. That community includes around 200
> contributors from academia and industry. The core developers of our project
> are listed in our contributors below and are also represented by logos on
> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
> University of Washington and Wolfram.
>
> === Core Developers ===
>
> (with GitHub logins)
>
>  * Tianqi Chen (@tqchen)
>  * Mu Li (@mli)
>  * Junyuan Xie (@piiswrong)
>  * Bing Xu (@antinucleon)
>  * Chiyuan Zhang (@pluskid)
>  * Minjie Wang (@jermainewang)
>  * Naiyan Wang (@winstywang)
>  * Yizhi Liu (@javelinjs)
>  * Tong He (@hetong007)
>  * Qiang Kou (@thirdwing)
>  * Xingjian Shi (@sxjscience)
>
> === Alignment ===
>
> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
> and Mahout, each of which targets a different application domain. MXNet,
> being a distributed platform for large-scale deep learning, focuses on
> another important domain for which there still lacks a scalable,
> programmable, flexible and super fast open-source platform. The recent
> success of deep learning models especially for vision and speech
> recognition tasks has generated interests in both applying existing deep
> learning models and in developing new ones. Thus, an open-source platform
> for deep learning backed by some of the top industry and academic players
> will be able to attract a large community of users and developers. MXNet is
> a complex system needing many iterations of design, implementation and
> testing. Apache's collaboration framework which encourages active
> contribution from developers will inevitably help improve the quality of
> the system, as shown in the success of Hadoop, Spark, etc. Equally
> important is the community of users which helps identify real-life
> applications of deep learning, and helps to evaluate the system's
> performance and ease-of-use. We hope to leverage ASF for coordinating and
> promoting both communities, and in return benefit the communities with
> another useful tool.
>
> == Known Risks ==
>
> === Orphaned products ===
>
> Given the current level of investment in MXNet and the stakeholders using
> it - the risk of the project being abandoned is minimal. Amazon, for
> example, is in active development to use MXNet in many of its services and
> many large corporations use it in their production applications.
>
> === Inexperience with Open Source ===
>
> MXNet has existed as a healthy open source project for more than a year.
> During that time, the project has attracted 200+ contributors.
>
> === Homogenous Developers ===
>
> The initial list of committers and contributors includes developers from
> several institutions and industry participants (see above).
>
> === Reliance on Salaried Developers ===
>
> Like most open source projects, MXNet receives a substantial support from
> salaried developers. A large fraction of MXNet development is supported by
> graduate students at various universities in the course of research degrees
> - this is more a “volunteer” relationship, since in most cases students
> contribute vastly more than is necessary to immediately support research.
> In addition, those working from within corporations are devoting
> significant time and effort in the project - and these come from several
> organizations.
>
> === A Excessive Fascination with the Apache Brand ===
>
> We choose Apache not for publicity. We have two purposes. First, we hope
> that Apache's known best-practices for managing a mature open source
> project can help guide us.  For example, we are feeling the growing pains
> of a successful open source project as we attempt a major refactor of the
> internals while customers are using the system in production. We seek
> guidance in communicating breaking API changes and version revisions.
> Also, as our involvement from major corporations increases, we want to
> assure our users that MXNet will stay open and not favor any particular
> platform or environment. These are some examples of the know-how and
> discipline we're hoping Apache can bring to our project.
>
> Second, we want to leverage Apache's reputation to recruit more developers
> to create a diverse community.
>
> === Relationship with Other Apache Products ===
>
> Apache Mahout and Apache Spark's MLlib are general machine learning
> systems. Deep learning algorithms can thus be implemented on these two
> platforms as well. However, in practice, the overlap will be minimal.  Deep
> learning is so computationally intensive that it often requires specialized
> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
> of GPU hardware is complex because the hardware is so fast that the
> supporting systems around it must be carefully optimized to keep the GPU
> cores busy.  Extending this capability to distributed multi-GPU and
> multi-host environments requires great care.  This is a critical
> differentiator between MXNet and existing Apache machine learning systems.
>
> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
> This is the fundamental difference to MXNet who follows the parameter
> server framework. MXNet can run synchronously or asynchronously. In
> addition, MXNet has optimizations for training a wide range of deep
> learning models using a variety of approaches (e.g., model parallelism and
> data parallelism) which makes MXNet much more efficient (near-linear
> speedup on state of the art models). MXNet also supports both imperative
> and symbolic approaches providing ease of programming for deep learning
> algorithms.
>
> Other Apache projects that are potentially complimentary:
>
> Apache Arrow - read data in Apache Arrow‘s internal format from MXNet, that
> would allow users to run ETL/preprocessing in Spark, save the results in
> Arrow’s format and then run DL algorithms on it.
>
> Apache Singa - MXNet and Singa are both deep learning projects, and can
> benefit from a larger deep learning community at Apache.
>
> == Documentation ==
>
> Documentation has recently migrated to http://mxnet.io.  We continue to
> refine and improve the documentation.
>
> == Initial Source ==
>
> We currently use Github to maintain our source code,
> https://github.com/MXNet
>
> == Source and Intellectual Property Submission Plan ==
>
> MXNet Code is available under Apache License, Version 2.0. We will work
> with the committers to get CLAs signed and review previous contributions.
>
> == External Dependencies ==
>
>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> lib-zeromq), TBB
>  * required for GPU usage: cudnn, cuda
>  * required for python usage: Python 2/3
>  * required for R module: R, Rcpp (GPLv2 licensing)
>  * optional for image preparation and preprocessing: opencv
>  * optional dependencies for additional features: torch7, numba, cython (in
> NNVM branch)
>
> Rcpt and lib-zeromq are expected to be licensing discussions.
>
> == Cryptography ==
>
> Not Applicable
>
> == Required Resources ==
>
> === Mailing Lists ===
>
> There is currently no mailing list.
>
> === Issue Tracking ===
>
> Currently uses GitHub to track issues. Would like to continue to do so.
>
> == Committers and Affiliations ==
>
>  * Tianqi Chen (UW)
>  * Mu Li (AWS)
>  * Junyuan Xie (AWS)
>  * Bing Xu (Apple)
>  * Chiyuan Zhang (MIT)
>  * Minjie Wang (UYU)
>  * Naiyan Wang (Tusimple)
>  * Yizhi Liu (Mediav)
>  * Tong He (Simon Fraser University)
>  * Qiang Kou (Indiana U)
>  * Xingjian Shi (HKUST)
>
> == Sponsors ==
>
> === Champion ===
>
> Henri Yandell (bayard at apache.org)
>
> === Nominated Mentors ===
>
> Sebastian Schelter (ssc@apache.org)
>
>
> === Sponsoring Entity ===
>
> We are requesting the Incubator to sponsor this project.
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Terry Chen <ja...@gmail.com>]

Hello, 

I would like to be committer of MxNet

Github ID:   terrychenism
Email:  terrychen2012@live.com
Affiliations:  Novumind


Best,
Terry


On 2017-01-05 21:12 (-0800), Henri Yandell <ba...@apache.org> wrote: 
> Hello Incubator,
> 
> I'd like to propose a new incubator Apache MXNet podling.
> 
> The existing MXNet project (http://mxnet.io - 1.5 years old, 15 committers,
> 200 contributors) is very interested in joining Apache. MXNet is an
> open-source deep learning framework that allows you to define, train, and
> deploy deep neural networks on a wide array of devices, from cloud
> infrastructure to mobile devices.
> 
> The wiki proposal page is located here:
> 
>   https://wiki.apache.org/incubator/MXNetProposal
> 
> I've included the text below in case anyone wants to focus on parts of it
> in a reply.
> 
> Looking forward to your thoughts, and for lots of interested Apache members
> to volunteer to mentor the project in addition to Sebastian and myself.
> 
> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!
> 
> Thanks,
> 
> Hen
> On behalf of the MXNet project
> 
> ---------
> 
> = MXNet: Apache Incubator Proposal =
> 
> == Abstract ==
> 
> MXNet is a Flexible and Efficient Library for Deep Learning
> 
> == Proposal ==
> 
> MXNet is an open-source deep learning framework that allows you to define,
> train, and deploy deep neural networks on a wide array of devices, from
> cloud infrastructure to mobile devices. It is highly scalable, allowing for
> fast model training, and supports a flexible programming model and multiple
> languages. MXNet allows you to mix symbolic and imperative programming
> flavors to maximize both efficiency and productivity. MXNet is built on a
> dynamic dependency scheduler that automatically parallelizes both symbolic
> and imperative operations on the fly. A graph optimization layer on top of
> that makes symbolic execution fast and memory efficient. The MXNet library
> is portable and lightweight, and it scales to multiple GPUs and multiple
> machines.
> 
> == Background ==
> 
> Deep learning is a subset of Machine learning and refers to a class of
> algorithms that use a hierarchical approach with non-linearities to
> discover and learn representations within data. Deep Learning has recently
> become very popular due to its applicability and advancement of domains
> such as Computer Vision, Speech Recognition, Natural Language Understanding
> and Recommender Systems. With pervasive and cost effective cloud computing,
> large labeled datasets and continued algorithmic innovation, Deep Learning
> has become the one of the most popular classes of algorithms for machine
> learning practitioners in recent years.
> 
> == Rational ==
> 
> The adoption of deep learning is quickly expanding from initial deep domain
> experts rooted in academia to data scientists and developers working to
> deploy intelligent services and products. Deep learning however has many
> challenges.  These include model training time (which can take days to
> weeks), programmability (not everyone writes Python or C++ and like
> symbolic programming) and balancing production readiness (support for
> things like failover) with development flexibility (ability to program
> different ways, support for new operators and model types) and speed of
> execution (fast and scalable model training).  Other frameworks excel on
> some but not all of these aspects.
> 
> 
> == Initial Goals ==
> 
> MXNet is a fairly established project on GitHub with its first code
> contribution in April 2015 and roughly 200 contributors. It is used by
> several large companies and some of the top research institutions on the
> planet. Initial goals would be the following:
> 
>  1. Move the existing codebase(s) to Apache
>  1. Integrate with the Apache development process/sign CLAs
>  1. Ensure all dependencies are compliant with Apache License version 2.0
>  1. Incremental development and releases per Apache guidelines
>  1. Establish engineering discipline and a predictable release cadence of
> high quality releases
>  1. Expand the community beyond the current base of expert level users
>  1. Improve usability and the overall developer/user experience
>  1. Add additional functionality to address newer problem types and
> algorithms
> 
> 
> == Current Status ==
> 
> === Meritocracy ===
> 
> The MXNet project already operates on meritocratic principles. Today, MXNet
> has developers worldwide and has accepted multiple major patches from a
> diverse set of contributors within both industry and academia. We would
> like to follow ASF meritocratic principles to encourage more developers to
> contribute in this project. We know that only active and committed
> developers from a diverse set of backgrounds can make MXNet a successful
> project.  We are also improving the documentation and code to help new
> developers get started quickly.
> 
> === Community ===
> 
> Acceptance into the Apache foundation would bolster the growing user and
> developer community around MXNet. That community includes around 200
> contributors from academia and industry. The core developers of our project
> are listed in our contributors below and are also represented by logos on
> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
> University of Washington and Wolfram.
> 
> === Core Developers ===
> 
> (with GitHub logins)
> 
>  * Tianqi Chen (@tqchen)
>  * Mu Li (@mli)
>  * Junyuan Xie (@piiswrong)
>  * Bing Xu (@antinucleon)
>  * Chiyuan Zhang (@pluskid)
>  * Minjie Wang (@jermainewang)
>  * Naiyan Wang (@winstywang)
>  * Yizhi Liu (@javelinjs)
>  * Tong He (@hetong007)
>  * Qiang Kou (@thirdwing)
>  * Xingjian Shi (@sxjscience)
> 
> === Alignment ===
> 
> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
> and Mahout, each of which targets a different application domain. MXNet,
> being a distributed platform for large-scale deep learning, focuses on
> another important domain for which there still lacks a scalable,
> programmable, flexible and super fast open-source platform. The recent
> success of deep learning models especially for vision and speech
> recognition tasks has generated interests in both applying existing deep
> learning models and in developing new ones. Thus, an open-source platform
> for deep learning backed by some of the top industry and academic players
> will be able to attract a large community of users and developers. MXNet is
> a complex system needing many iterations of design, implementation and
> testing. Apache's collaboration framework which encourages active
> contribution from developers will inevitably help improve the quality of
> the system, as shown in the success of Hadoop, Spark, etc. Equally
> important is the community of users which helps identify real-life
> applications of deep learning, and helps to evaluate the system's
> performance and ease-of-use. We hope to leverage ASF for coordinating and
> promoting both communities, and in return benefit the communities with
> another useful tool.
> 
> == Known Risks ==
> 
> === Orphaned products ===
> 
> Given the current level of investment in MXNet and the stakeholders using
> it - the risk of the project being abandoned is minimal. Amazon, for
> example, is in active development to use MXNet in many of its services and
> many large corporations use it in their production applications.
> 
> === Inexperience with Open Source ===
> 
> MXNet has existed as a healthy open source project for more than a year.
> During that time, the project has attracted 200+ contributors.
> 
> === Homogenous Developers ===
> 
> The initial list of committers and contributors includes developers from
> several institutions and industry participants (see above).
> 
> === Reliance on Salaried Developers ===
> 
> Like most open source projects, MXNet receives a substantial support from
> salaried developers. A large fraction of MXNet development is supported by
> graduate students at various universities in the course of research degrees
> - this is more a \u201cvolunteer\u201d relationship, since in most cases students
> contribute vastly more than is necessary to immediately support research.
> In addition, those working from within corporations are devoting
> significant time and effort in the project - and these come from several
> organizations.
> 
> === A Excessive Fascination with the Apache Brand ===
> 
> We choose Apache not for publicity. We have two purposes. First, we hope
> that Apache's known best-practices for managing a mature open source
> project can help guide us.  For example, we are feeling the growing pains
> of a successful open source project as we attempt a major refactor of the
> internals while customers are using the system in production. We seek
> guidance in communicating breaking API changes and version revisions.
> Also, as our involvement from major corporations increases, we want to
> assure our users that MXNet will stay open and not favor any particular
> platform or environment. These are some examples of the know-how and
> discipline we're hoping Apache can bring to our project.
> 
> Second, we want to leverage Apache's reputation to recruit more developers
> to create a diverse community.
> 
> === Relationship with Other Apache Products ===
> 
> Apache Mahout and Apache Spark's MLlib are general machine learning
> systems. Deep learning algorithms can thus be implemented on these two
> platforms as well. However, in practice, the overlap will be minimal.  Deep
> learning is so computationally intensive that it often requires specialized
> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
> of GPU hardware is complex because the hardware is so fast that the
> supporting systems around it must be carefully optimized to keep the GPU
> cores busy.  Extending this capability to distributed multi-GPU and
> multi-host environments requires great care.  This is a critical
> differentiator between MXNet and existing Apache machine learning systems.
> 
> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
> This is the fundamental difference to MXNet who follows the parameter
> server framework. MXNet can run synchronously or asynchronously. In
> addition, MXNet has optimizations for training a wide range of deep
> learning models using a variety of approaches (e.g., model parallelism and
> data parallelism) which makes MXNet much more efficient (near-linear
> speedup on state of the art models). MXNet also supports both imperative
> and symbolic approaches providing ease of programming for deep learning
> algorithms.
> 
> Other Apache projects that are potentially complimentary:
> 
> Apache Arrow - read data in Apache Arrow\u2018s internal format from MXNet, that
> would allow users to run ETL/preprocessing in Spark, save the results in
> Arrow\u2019s format and then run DL algorithms on it.
> 
> Apache Singa - MXNet and Singa are both deep learning projects, and can
> benefit from a larger deep learning community at Apache.
> 
> == Documentation ==
> 
> Documentation has recently migrated to http://mxnet.io.  We continue to
> refine and improve the documentation.
> 
> == Initial Source ==
> 
> We currently use Github to maintain our source code,
> https://github.com/MXNet
> 
> == Source and Intellectual Property Submission Plan ==
> 
> MXNet Code is available under Apache License, Version 2.0. We will work
> with the committers to get CLAs signed and review previous contributions.
> 
> == External Dependencies ==
> 
>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> lib-zeromq), TBB
>  * required for GPU usage: cudnn, cuda
>  * required for python usage: Python 2/3
>  * required for R module: R, Rcpp (GPLv2 licensing)
>  * optional for image preparation and preprocessing: opencv
>  * optional dependencies for additional features: torch7, numba, cython (in
> NNVM branch)
> 
> Rcpt and lib-zeromq are expected to be licensing discussions.
> 
> == Cryptography ==
> 
> Not Applicable
> 
> == Required Resources ==
> 
> === Mailing Lists ===
> 
> There is currently no mailing list.
> 
> === Issue Tracking ===
> 
> Currently uses GitHub to track issues. Would like to continue to do so.
> 
> == Committers and Affiliations ==
> 
>  * Tianqi Chen (UW)
>  * Mu Li (AWS)
>  * Junyuan Xie (AWS)
>  * Bing Xu (Apple)
>  * Chiyuan Zhang (MIT)
>  * Minjie Wang (UYU)
>  * Naiyan Wang (Tusimple)
>  * Yizhi Liu (Mediav)
>  * Tong He (Simon Fraser University)
>  * Qiang Kou (Indiana U)
>  * Xingjian Shi (HKUST)
> 
> == Sponsors ==
> 
> === Champion ===
> 
> Henri Yandell (bayard at apache.org)
> 
> === Nominated Mentors ===
> 
> Sebastian Schelter (ssc@apache.org)
> 
> 
> === Sponsoring Entity ===
> 
> We are requesting the Incubator to sponsor this project.
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Added :)

On Fri, Jan 13, 2017 at 3:48 AM, Zhang Jian <zh...@gmail.com>
wrote:

> Hi,
> Please sign me up as a committer of MxNet
>
> Github ID: jzhang-zju
> Email: zhang.jian.zju@gmail.com
> Affiliations: Horizon Robotics
>
> Best,
> Jian
>
> On 2017-01-06 13:12 (+0800), Henri Yandell <ba...@apache.org> wrote:
> > Hello Incubator,
> >
> > I'd like to propose a new incubator Apache MXNet podling.
> >
> > The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> committers,
> > 200 contributors) is very interested in joining Apache. MXNet is an
> > open-source deep learning framework that allows you to define, train, and
> > deploy deep neural networks on a wide array of devices, from cloud
> > infrastructure to mobile devices.
> >
> > The wiki proposal page is located here:
> >
> >   https://wiki.apache.org/incubator/MXNetProposal
> >
> > I've included the text below in case anyone wants to focus on parts of it
> > in a reply.
> >
> > Looking forward to your thoughts, and for lots of interested Apache
> members
> > to volunteer to mentor the project in addition to Sebastian and myself.
> >
> > Currently the list of committers is based on the current active coders,
> so
> > we're also very interested in hearing from anyone else who is interested
> in
> > working on the project, be they current or future contributor!
> >
> > Thanks,
> >
> > Hen
> > On behalf of the MXNet project
> >
> > ---------
> >
> > = MXNet: Apache Incubator Proposal =
> >
> > == Abstract ==
> >
> > MXNet is a Flexible and Efficient Library for Deep Learning
> >
> > == Proposal ==
> >
> > MXNet is an open-source deep learning framework that allows you to
> define,
> > train, and deploy deep neural networks on a wide array of devices, from
> > cloud infrastructure to mobile devices. It is highly scalable, allowing
> for
> > fast model training, and supports a flexible programming model and
> multiple
> > languages. MXNet allows you to mix symbolic and imperative programming
> > flavors to maximize both efficiency and productivity. MXNet is built on a
> > dynamic dependency scheduler that automatically parallelizes both
> symbolic
> > and imperative operations on the fly. A graph optimization layer on top
> of
> > that makes symbolic execution fast and memory efficient. The MXNet
> library
> > is portable and lightweight, and it scales to multiple GPUs and multiple
> > machines.
> >
> > == Background ==
> >
> > Deep learning is a subset of Machine learning and refers to a class of
> > algorithms that use a hierarchical approach with non-linearities to
> > discover and learn representations within data. Deep Learning has
> recently
> > become very popular due to its applicability and advancement of domains
> > such as Computer Vision, Speech Recognition, Natural Language
> Understanding
> > and Recommender Systems. With pervasive and cost effective cloud
> computing,
> > large labeled datasets and continued algorithmic innovation, Deep
> Learning
> > has become the one of the most popular classes of algorithms for machine
> > learning practitioners in recent years.
> >
> > == Rational ==
> >
> > The adoption of deep learning is quickly expanding from initial deep
> domain
> > experts rooted in academia to data scientists and developers working to
> > deploy intelligent services and products. Deep learning however has many
> > challenges.  These include model training time (which can take days to
> > weeks), programmability (not everyone writes Python or C++ and like
> > symbolic programming) and balancing production readiness (support for
> > things like failover) with development flexibility (ability to program
> > different ways, support for new operators and model types) and speed of
> > execution (fast and scalable model training).  Other frameworks excel on
> > some but not all of these aspects.
> >
> >
> > == Initial Goals ==
> >
> > MXNet is a fairly established project on GitHub with its first code
> > contribution in April 2015 and roughly 200 contributors. It is used by
> > several large companies and some of the top research institutions on the
> > planet. Initial goals would be the following:
> >
> >  1. Move the existing codebase(s) to Apache
> >  1. Integrate with the Apache development process/sign CLAs
> >  1. Ensure all dependencies are compliant with Apache License version 2.0
> >  1. Incremental development and releases per Apache guidelines
> >  1. Establish engineering discipline and a predictable release cadence of
> > high quality releases
> >  1. Expand the community beyond the current base of expert level users
> >  1. Improve usability and the overall developer/user experience
> >  1. Add additional functionality to address newer problem types and
> > algorithms
> >
> >
> > == Current Status ==
> >
> > === Meritocracy ===
> >
> > The MXNet project already operates on meritocratic principles. Today,
> MXNet
> > has developers worldwide and has accepted multiple major patches from a
> > diverse set of contributors within both industry and academia. We would
> > like to follow ASF meritocratic principles to encourage more developers
> to
> > contribute in this project. We know that only active and committed
> > developers from a diverse set of backgrounds can make MXNet a successful
> > project.  We are also improving the documentation and code to help new
> > developers get started quickly.
> >
> > === Community ===
> >
> > Acceptance into the Apache foundation would bolster the growing user and
> > developer community around MXNet. That community includes around 200
> > contributors from academia and industry. The core developers of our
> project
> > are listed in our contributors below and are also represented by logos on
> > the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> > Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of
> Alberta,
> > University of Washington and Wolfram.
> >
> > === Core Developers ===
> >
> > (with GitHub logins)
> >
> >  * Tianqi Chen (@tqchen)
> >  * Mu Li (@mli)
> >  * Junyuan Xie (@piiswrong)
> >  * Bing Xu (@antinucleon)
> >  * Chiyuan Zhang (@pluskid)
> >  * Minjie Wang (@jermainewang)
> >  * Naiyan Wang (@winstywang)
> >  * Yizhi Liu (@javelinjs)
> >  * Tong He (@hetong007)
> >  * Qiang Kou (@thirdwing)
> >  * Xingjian Shi (@sxjscience)
> >
> > === Alignment ===
> >
> > ASF is already the home of many distributed platforms, e.g., Hadoop,
> Spark
> > and Mahout, each of which targets a different application domain. MXNet,
> > being a distributed platform for large-scale deep learning, focuses on
> > another important domain for which there still lacks a scalable,
> > programmable, flexible and super fast open-source platform. The recent
> > success of deep learning models especially for vision and speech
> > recognition tasks has generated interests in both applying existing deep
> > learning models and in developing new ones. Thus, an open-source platform
> > for deep learning backed by some of the top industry and academic players
> > will be able to attract a large community of users and developers. MXNet
> is
> > a complex system needing many iterations of design, implementation and
> > testing. Apache's collaboration framework which encourages active
> > contribution from developers will inevitably help improve the quality of
> > the system, as shown in the success of Hadoop, Spark, etc. Equally
> > important is the community of users which helps identify real-life
> > applications of deep learning, and helps to evaluate the system's
> > performance and ease-of-use. We hope to leverage ASF for coordinating and
> > promoting both communities, and in return benefit the communities with
> > another useful tool.
> >
> > == Known Risks ==
> >
> > === Orphaned products ===
> >
> > Given the current level of investment in MXNet and the stakeholders using
> > it - the risk of the project being abandoned is minimal. Amazon, for
> > example, is in active development to use MXNet in many of its services
> and
> > many large corporations use it in their production applications.
> >
> > === Inexperience with Open Source ===
> >
> > MXNet has existed as a healthy open source project for more than a year.
> > During that time, the project has attracted 200+ contributors.
> >
> > === Homogenous Developers ===
> >
> > The initial list of committers and contributors includes developers from
> > several institutions and industry participants (see above).
> >
> > === Reliance on Salaried Developers ===
> >
> > Like most open source projects, MXNet receives a substantial support from
> > salaried developers. A large fraction of MXNet development is supported
> by
> > graduate students at various universities in the course of research
> degrees
> > - this is more a “volunteer” relationship, since in most cases students
> > contribute vastly more than is necessary to immediately support research.
> > In addition, those working from within corporations are devoting
> > significant time and effort in the project - and these come from several
> > organizations.
> >
> > === A Excessive Fascination with the Apache Brand ===
> >
> > We choose Apache not for publicity. We have two purposes. First, we hope
> > that Apache's known best-practices for managing a mature open source
> > project can help guide us.  For example, we are feeling the growing pains
> > of a successful open source project as we attempt a major refactor of the
> > internals while customers are using the system in production. We seek
> > guidance in communicating breaking API changes and version revisions.
> > Also, as our involvement from major corporations increases, we want to
> > assure our users that MXNet will stay open and not favor any particular
> > platform or environment. These are some examples of the know-how and
> > discipline we're hoping Apache can bring to our project.
> >
> > Second, we want to leverage Apache's reputation to recruit more
> developers
> > to create a diverse community.
> >
> > === Relationship with Other Apache Products ===
> >
> > Apache Mahout and Apache Spark's MLlib are general machine learning
> > systems. Deep learning algorithms can thus be implemented on these two
> > platforms as well. However, in practice, the overlap will be minimal.
> Deep
> > learning is so computationally intensive that it often requires
> specialized
> > GPU hardware to accomplish tasks of meaningful size.  Making efficient
> use
> > of GPU hardware is complex because the hardware is so fast that the
> > supporting systems around it must be carefully optimized to keep the GPU
> > cores busy.  Extending this capability to distributed multi-GPU and
> > multi-host environments requires great care.  This is a critical
> > differentiator between MXNet and existing Apache machine learning
> systems.
> >
> > Mahout and Spark ML-LIB follow models where their nodes run
> synchronously.
> > This is the fundamental difference to MXNet who follows the parameter
> > server framework. MXNet can run synchronously or asynchronously. In
> > addition, MXNet has optimizations for training a wide range of deep
> > learning models using a variety of approaches (e.g., model parallelism
> and
> > data parallelism) which makes MXNet much more efficient (near-linear
> > speedup on state of the art models). MXNet also supports both imperative
> > and symbolic approaches providing ease of programming for deep learning
> > algorithms.
> >
> > Other Apache projects that are potentially complimentary:
> >
> > Apache Arrow - read data in Apache Arrow‘s internal format from MXNet,
> that
> > would allow users to run ETL/preprocessing in Spark, save the results in
> > Arrow’s format and then run DL algorithms on it.
> >
> > Apache Singa - MXNet and Singa are both deep learning projects, and can
> > benefit from a larger deep learning community at Apache.
> >
> > == Documentation ==
> >
> > Documentation has recently migrated to http://mxnet.io.  We continue to
> > refine and improve the documentation.
> >
> > == Initial Source ==
> >
> > We currently use Github to maintain our source code,
> > https://github.com/MXNet
> >
> > == Source and Intellectual Property Submission Plan ==
> >
> > MXNet Code is available under Apache License, Version 2.0. We will work
> > with the committers to get CLAs signed and review previous contributions.
> >
> > == External Dependencies ==
> >
> >  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> > (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> > lib-zeromq), TBB
> >  * required for GPU usage: cudnn, cuda
> >  * required for python usage: Python 2/3
> >  * required for R module: R, Rcpp (GPLv2 licensing)
> >  * optional for image preparation and preprocessing: opencv
> >  * optional dependencies for additional features: torch7, numba, cython
> (in
> > NNVM branch)
> >
> > Rcpt and lib-zeromq are expected to be licensing discussions.
> >
> > == Cryptography ==
> >
> > Not Applicable
> >
> > == Required Resources ==
> >
> > === Mailing Lists ===
> >
> > There is currently no mailing list.
> >
> > === Issue Tracking ===
> >
> > Currently uses GitHub to track issues. Would like to continue to do so.
> >
> > == Committers and Affiliations ==
> >
> >  * Tianqi Chen (UW)
> >  * Mu Li (AWS)
> >  * Junyuan Xie (AWS)
> >  * Bing Xu (Apple)
> >  * Chiyuan Zhang (MIT)
> >  * Minjie Wang (UYU)
> >  * Naiyan Wang (Tusimple)
> >  * Yizhi Liu (Mediav)
> >  * Tong He (Simon Fraser University)
> >  * Qiang Kou (Indiana U)
> >  * Xingjian Shi (HKUST)
> >
> > == Sponsors ==
> >
> > === Champion ===
> >
> > Henri Yandell (bayard at apache.org)
> >
> > === Nominated Mentors ===
> >
> > Sebastian Schelter (ssc@apache.org)
> >
> >
> > === Sponsoring Entity ===
> >
> > We are requesting the Incubator to sponsor this project.
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Zhang Jian <zh...@gmail.com>]

Hi,  
Please sign me up as a committer of MxNet  

Github ID: jzhang-zju
Email: zhang.jian.zju@gmail.com
Affiliations: Horizon Robotics

Best, 
Jian

On 2017-01-06 13:12 (+0800), Henri Yandell <ba...@apache.org> wrote: 
> Hello Incubator,
> 
> I'd like to propose a new incubator Apache MXNet podling.
> 
> The existing MXNet project (http://mxnet.io - 1.5 years old, 15 committers,
> 200 contributors) is very interested in joining Apache. MXNet is an
> open-source deep learning framework that allows you to define, train, and
> deploy deep neural networks on a wide array of devices, from cloud
> infrastructure to mobile devices.
> 
> The wiki proposal page is located here:
> 
>   https://wiki.apache.org/incubator/MXNetProposal
> 
> I've included the text below in case anyone wants to focus on parts of it
> in a reply.
> 
> Looking forward to your thoughts, and for lots of interested Apache members
> to volunteer to mentor the project in addition to Sebastian and myself.
> 
> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!
> 
> Thanks,
> 
> Hen
> On behalf of the MXNet project
> 
> ---------
> 
> = MXNet: Apache Incubator Proposal =
> 
> == Abstract ==
> 
> MXNet is a Flexible and Efficient Library for Deep Learning
> 
> == Proposal ==
> 
> MXNet is an open-source deep learning framework that allows you to define,
> train, and deploy deep neural networks on a wide array of devices, from
> cloud infrastructure to mobile devices. It is highly scalable, allowing for
> fast model training, and supports a flexible programming model and multiple
> languages. MXNet allows you to mix symbolic and imperative programming
> flavors to maximize both efficiency and productivity. MXNet is built on a
> dynamic dependency scheduler that automatically parallelizes both symbolic
> and imperative operations on the fly. A graph optimization layer on top of
> that makes symbolic execution fast and memory efficient. The MXNet library
> is portable and lightweight, and it scales to multiple GPUs and multiple
> machines.
> 
> == Background ==
> 
> Deep learning is a subset of Machine learning and refers to a class of
> algorithms that use a hierarchical approach with non-linearities to
> discover and learn representations within data. Deep Learning has recently
> become very popular due to its applicability and advancement of domains
> such as Computer Vision, Speech Recognition, Natural Language Understanding
> and Recommender Systems. With pervasive and cost effective cloud computing,
> large labeled datasets and continued algorithmic innovation, Deep Learning
> has become the one of the most popular classes of algorithms for machine
> learning practitioners in recent years.
> 
> == Rational ==
> 
> The adoption of deep learning is quickly expanding from initial deep domain
> experts rooted in academia to data scientists and developers working to
> deploy intelligent services and products. Deep learning however has many
> challenges.  These include model training time (which can take days to
> weeks), programmability (not everyone writes Python or C++ and like
> symbolic programming) and balancing production readiness (support for
> things like failover) with development flexibility (ability to program
> different ways, support for new operators and model types) and speed of
> execution (fast and scalable model training).  Other frameworks excel on
> some but not all of these aspects.
> 
> 
> == Initial Goals ==
> 
> MXNet is a fairly established project on GitHub with its first code
> contribution in April 2015 and roughly 200 contributors. It is used by
> several large companies and some of the top research institutions on the
> planet. Initial goals would be the following:
> 
>  1. Move the existing codebase(s) to Apache
>  1. Integrate with the Apache development process/sign CLAs
>  1. Ensure all dependencies are compliant with Apache License version 2.0
>  1. Incremental development and releases per Apache guidelines
>  1. Establish engineering discipline and a predictable release cadence of
> high quality releases
>  1. Expand the community beyond the current base of expert level users
>  1. Improve usability and the overall developer/user experience
>  1. Add additional functionality to address newer problem types and
> algorithms
> 
> 
> == Current Status ==
> 
> === Meritocracy ===
> 
> The MXNet project already operates on meritocratic principles. Today, MXNet
> has developers worldwide and has accepted multiple major patches from a
> diverse set of contributors within both industry and academia. We would
> like to follow ASF meritocratic principles to encourage more developers to
> contribute in this project. We know that only active and committed
> developers from a diverse set of backgrounds can make MXNet a successful
> project.  We are also improving the documentation and code to help new
> developers get started quickly.
> 
> === Community ===
> 
> Acceptance into the Apache foundation would bolster the growing user and
> developer community around MXNet. That community includes around 200
> contributors from academia and industry. The core developers of our project
> are listed in our contributors below and are also represented by logos on
> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
> University of Washington and Wolfram.
> 
> === Core Developers ===
> 
> (with GitHub logins)
> 
>  * Tianqi Chen (@tqchen)
>  * Mu Li (@mli)
>  * Junyuan Xie (@piiswrong)
>  * Bing Xu (@antinucleon)
>  * Chiyuan Zhang (@pluskid)
>  * Minjie Wang (@jermainewang)
>  * Naiyan Wang (@winstywang)
>  * Yizhi Liu (@javelinjs)
>  * Tong He (@hetong007)
>  * Qiang Kou (@thirdwing)
>  * Xingjian Shi (@sxjscience)
> 
> === Alignment ===
> 
> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
> and Mahout, each of which targets a different application domain. MXNet,
> being a distributed platform for large-scale deep learning, focuses on
> another important domain for which there still lacks a scalable,
> programmable, flexible and super fast open-source platform. The recent
> success of deep learning models especially for vision and speech
> recognition tasks has generated interests in both applying existing deep
> learning models and in developing new ones. Thus, an open-source platform
> for deep learning backed by some of the top industry and academic players
> will be able to attract a large community of users and developers. MXNet is
> a complex system needing many iterations of design, implementation and
> testing. Apache's collaboration framework which encourages active
> contribution from developers will inevitably help improve the quality of
> the system, as shown in the success of Hadoop, Spark, etc. Equally
> important is the community of users which helps identify real-life
> applications of deep learning, and helps to evaluate the system's
> performance and ease-of-use. We hope to leverage ASF for coordinating and
> promoting both communities, and in return benefit the communities with
> another useful tool.
> 
> == Known Risks ==
> 
> === Orphaned products ===
> 
> Given the current level of investment in MXNet and the stakeholders using
> it - the risk of the project being abandoned is minimal. Amazon, for
> example, is in active development to use MXNet in many of its services and
> many large corporations use it in their production applications.
> 
> === Inexperience with Open Source ===
> 
> MXNet has existed as a healthy open source project for more than a year.
> During that time, the project has attracted 200+ contributors.
> 
> === Homogenous Developers ===
> 
> The initial list of committers and contributors includes developers from
> several institutions and industry participants (see above).
> 
> === Reliance on Salaried Developers ===
> 
> Like most open source projects, MXNet receives a substantial support from
> salaried developers. A large fraction of MXNet development is supported by
> graduate students at various universities in the course of research degrees
> - this is more a \u201cvolunteer\u201d relationship, since in most cases students
> contribute vastly more than is necessary to immediately support research.
> In addition, those working from within corporations are devoting
> significant time and effort in the project - and these come from several
> organizations.
> 
> === A Excessive Fascination with the Apache Brand ===
> 
> We choose Apache not for publicity. We have two purposes. First, we hope
> that Apache's known best-practices for managing a mature open source
> project can help guide us.  For example, we are feeling the growing pains
> of a successful open source project as we attempt a major refactor of the
> internals while customers are using the system in production. We seek
> guidance in communicating breaking API changes and version revisions.
> Also, as our involvement from major corporations increases, we want to
> assure our users that MXNet will stay open and not favor any particular
> platform or environment. These are some examples of the know-how and
> discipline we're hoping Apache can bring to our project.
> 
> Second, we want to leverage Apache's reputation to recruit more developers
> to create a diverse community.
> 
> === Relationship with Other Apache Products ===
> 
> Apache Mahout and Apache Spark's MLlib are general machine learning
> systems. Deep learning algorithms can thus be implemented on these two
> platforms as well. However, in practice, the overlap will be minimal.  Deep
> learning is so computationally intensive that it often requires specialized
> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
> of GPU hardware is complex because the hardware is so fast that the
> supporting systems around it must be carefully optimized to keep the GPU
> cores busy.  Extending this capability to distributed multi-GPU and
> multi-host environments requires great care.  This is a critical
> differentiator between MXNet and existing Apache machine learning systems.
> 
> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
> This is the fundamental difference to MXNet who follows the parameter
> server framework. MXNet can run synchronously or asynchronously. In
> addition, MXNet has optimizations for training a wide range of deep
> learning models using a variety of approaches (e.g., model parallelism and
> data parallelism) which makes MXNet much more efficient (near-linear
> speedup on state of the art models). MXNet also supports both imperative
> and symbolic approaches providing ease of programming for deep learning
> algorithms.
> 
> Other Apache projects that are potentially complimentary:
> 
> Apache Arrow - read data in Apache Arrow\u2018s internal format from MXNet, that
> would allow users to run ETL/preprocessing in Spark, save the results in
> Arrow\u2019s format and then run DL algorithms on it.
> 
> Apache Singa - MXNet and Singa are both deep learning projects, and can
> benefit from a larger deep learning community at Apache.
> 
> == Documentation ==
> 
> Documentation has recently migrated to http://mxnet.io.  We continue to
> refine and improve the documentation.
> 
> == Initial Source ==
> 
> We currently use Github to maintain our source code,
> https://github.com/MXNet
> 
> == Source and Intellectual Property Submission Plan ==
> 
> MXNet Code is available under Apache License, Version 2.0. We will work
> with the committers to get CLAs signed and review previous contributions.
> 
> == External Dependencies ==
> 
>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> lib-zeromq), TBB
>  * required for GPU usage: cudnn, cuda
>  * required for python usage: Python 2/3
>  * required for R module: R, Rcpp (GPLv2 licensing)
>  * optional for image preparation and preprocessing: opencv
>  * optional dependencies for additional features: torch7, numba, cython (in
> NNVM branch)
> 
> Rcpt and lib-zeromq are expected to be licensing discussions.
> 
> == Cryptography ==
> 
> Not Applicable
> 
> == Required Resources ==
> 
> === Mailing Lists ===
> 
> There is currently no mailing list.
> 
> === Issue Tracking ===
> 
> Currently uses GitHub to track issues. Would like to continue to do so.
> 
> == Committers and Affiliations ==
> 
>  * Tianqi Chen (UW)
>  * Mu Li (AWS)
>  * Junyuan Xie (AWS)
>  * Bing Xu (Apple)
>  * Chiyuan Zhang (MIT)
>  * Minjie Wang (UYU)
>  * Naiyan Wang (Tusimple)
>  * Yizhi Liu (Mediav)
>  * Tong He (Simon Fraser University)
>  * Qiang Kou (Indiana U)
>  * Xingjian Shi (HKUST)
> 
> == Sponsors ==
> 
> === Champion ===
> 
> Henri Yandell (bayard at apache.org)
> 
> === Nominated Mentors ===
> 
> Sebastian Schelter (ssc@apache.org)
> 
> 
> === Sponsoring Entity ===
> 
> We are requesting the Incubator to sponsor this project.
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Added :)

On Fri, Jan 13, 2017 at 4:37 PM, sandeep krishnamurthy <
sandeep.krishna98@gmail.com> wrote:

> Hi,
> Please sign me up as a committer of MxNet
>
> Github ID: sandeep-krishnamurthy
> Email: sandeep.krishna98@gmail.com
> Affiliations: AWS
>
> Best,
> Sandeep Krishnamurthy
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[sandeep krishnamurthy <sa...@gmail.com>]

Hi, 
Please sign me up as a committer of MxNet 

Github ID: sandeep-krishnamurthy
Email: sandeep.krishna98@gmail.com
Affiliations: AWS

Best, 
Sandeep Krishnamurthy

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[项亮 <xl...@gmail.com>]

I would like to be committer of Mxnet

Github ID: xlvector
xlvector#gmail.com

Affiliations: Toutiao

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Added :)

On Thu, Jan 12, 2017 at 1:37 AM, Terry Chen <ja...@gmail.com> wrote:

>
> Hello,
>
> I would like to be committer of MxNet
>
> Github ID:   terrychenism
> Email:  terrychen2012@live.com
> Affiliations:  Novumind
>
>
> Best,
> Terry
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Terry Chen <ja...@gmail.com>]

Hello, 

I would like to be committer of MxNet

Github ID:   terrychenism
Email:  terrychen2012@live.com
Affiliations:  Novumind


Best,
Terry

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Terry Chen <ja...@gmail.com>]

Hello, 

I would like to be committer of MxNet

Github ID:   terrychenism
Email:  terrychen2012@live.com
Affiliations:  Novumind

Best,
Terry

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Thanks :) Updated.

On Thu, Jan 12, 2017 at 10:49 AM, Henry Saputra <he...@gmail.com>
wrote:

> Hi Henri,
>
> Could you update the proposal for === Mailing Lists === section to list
> existing and required mailing lists from Incubator?
>
> Thanks,
>
> Henry
>
> On Thu, Jan 5, 2017 at 9:12 PM, Henri Yandell <ba...@apache.org> wrote:
>
> > Hello Incubator,
> >
> > I'd like to propose a new incubator Apache MXNet podling.
> >
> > The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> > committers,
> > 200 contributors) is very interested in joining Apache. MXNet is an
> > open-source deep learning framework that allows you to define, train, and
> > deploy deep neural networks on a wide array of devices, from cloud
> > infrastructure to mobile devices.
> >
> > The wiki proposal page is located here:
> >
> >   https://wiki.apache.org/incubator/MXNetProposal
> >
> > I've included the text below in case anyone wants to focus on parts of it
> > in a reply.
> >
> > Looking forward to your thoughts, and for lots of interested Apache
> members
> > to volunteer to mentor the project in addition to Sebastian and myself.
> >
> > Currently the list of committers is based on the current active coders,
> so
> > we're also very interested in hearing from anyone else who is interested
> in
> > working on the project, be they current or future contributor!
> >
> > Thanks,
> >
> > Hen
> > On behalf of the MXNet project
> >
> > ---------
> >
> > = MXNet: Apache Incubator Proposal =
> >
> > == Abstract ==
> >
> > MXNet is a Flexible and Efficient Library for Deep Learning
> >
> > == Proposal ==
> >
> > MXNet is an open-source deep learning framework that allows you to
> define,
> > train, and deploy deep neural networks on a wide array of devices, from
> > cloud infrastructure to mobile devices. It is highly scalable, allowing
> for
> > fast model training, and supports a flexible programming model and
> multiple
> > languages. MXNet allows you to mix symbolic and imperative programming
> > flavors to maximize both efficiency and productivity. MXNet is built on a
> > dynamic dependency scheduler that automatically parallelizes both
> symbolic
> > and imperative operations on the fly. A graph optimization layer on top
> of
> > that makes symbolic execution fast and memory efficient. The MXNet
> library
> > is portable and lightweight, and it scales to multiple GPUs and multiple
> > machines.
> >
> > == Background ==
> >
> > Deep learning is a subset of Machine learning and refers to a class of
> > algorithms that use a hierarchical approach with non-linearities to
> > discover and learn representations within data. Deep Learning has
> recently
> > become very popular due to its applicability and advancement of domains
> > such as Computer Vision, Speech Recognition, Natural Language
> Understanding
> > and Recommender Systems. With pervasive and cost effective cloud
> computing,
> > large labeled datasets and continued algorithmic innovation, Deep
> Learning
> > has become the one of the most popular classes of algorithms for machine
> > learning practitioners in recent years.
> >
> > == Rational ==
> >
> > The adoption of deep learning is quickly expanding from initial deep
> domain
> > experts rooted in academia to data scientists and developers working to
> > deploy intelligent services and products. Deep learning however has many
> > challenges.  These include model training time (which can take days to
> > weeks), programmability (not everyone writes Python or C++ and like
> > symbolic programming) and balancing production readiness (support for
> > things like failover) with development flexibility (ability to program
> > different ways, support for new operators and model types) and speed of
> > execution (fast and scalable model training).  Other frameworks excel on
> > some but not all of these aspects.
> >
> >
> > == Initial Goals ==
> >
> > MXNet is a fairly established project on GitHub with its first code
> > contribution in April 2015 and roughly 200 contributors. It is used by
> > several large companies and some of the top research institutions on the
> > planet. Initial goals would be the following:
> >
> >  1. Move the existing codebase(s) to Apache
> >  1. Integrate with the Apache development process/sign CLAs
> >  1. Ensure all dependencies are compliant with Apache License version 2.0
> >  1. Incremental development and releases per Apache guidelines
> >  1. Establish engineering discipline and a predictable release cadence of
> > high quality releases
> >  1. Expand the community beyond the current base of expert level users
> >  1. Improve usability and the overall developer/user experience
> >  1. Add additional functionality to address newer problem types and
> > algorithms
> >
> >
> > == Current Status ==
> >
> > === Meritocracy ===
> >
> > The MXNet project already operates on meritocratic principles. Today,
> MXNet
> > has developers worldwide and has accepted multiple major patches from a
> > diverse set of contributors within both industry and academia. We would
> > like to follow ASF meritocratic principles to encourage more developers
> to
> > contribute in this project. We know that only active and committed
> > developers from a diverse set of backgrounds can make MXNet a successful
> > project.  We are also improving the documentation and code to help new
> > developers get started quickly.
> >
> > === Community ===
> >
> > Acceptance into the Apache foundation would bolster the growing user and
> > developer community around MXNet. That community includes around 200
> > contributors from academia and industry. The core developers of our
> project
> > are listed in our contributors below and are also represented by logos on
> > the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> > Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of
> Alberta,
> > University of Washington and Wolfram.
> >
> > === Core Developers ===
> >
> > (with GitHub logins)
> >
> >  * Tianqi Chen (@tqchen)
> >  * Mu Li (@mli)
> >  * Junyuan Xie (@piiswrong)
> >  * Bing Xu (@antinucleon)
> >  * Chiyuan Zhang (@pluskid)
> >  * Minjie Wang (@jermainewang)
> >  * Naiyan Wang (@winstywang)
> >  * Yizhi Liu (@javelinjs)
> >  * Tong He (@hetong007)
> >  * Qiang Kou (@thirdwing)
> >  * Xingjian Shi (@sxjscience)
> >
> > === Alignment ===
> >
> > ASF is already the home of many distributed platforms, e.g., Hadoop,
> Spark
> > and Mahout, each of which targets a different application domain. MXNet,
> > being a distributed platform for large-scale deep learning, focuses on
> > another important domain for which there still lacks a scalable,
> > programmable, flexible and super fast open-source platform. The recent
> > success of deep learning models especially for vision and speech
> > recognition tasks has generated interests in both applying existing deep
> > learning models and in developing new ones. Thus, an open-source platform
> > for deep learning backed by some of the top industry and academic players
> > will be able to attract a large community of users and developers. MXNet
> is
> > a complex system needing many iterations of design, implementation and
> > testing. Apache's collaboration framework which encourages active
> > contribution from developers will inevitably help improve the quality of
> > the system, as shown in the success of Hadoop, Spark, etc. Equally
> > important is the community of users which helps identify real-life
> > applications of deep learning, and helps to evaluate the system's
> > performance and ease-of-use. We hope to leverage ASF for coordinating and
> > promoting both communities, and in return benefit the communities with
> > another useful tool.
> >
> > == Known Risks ==
> >
> > === Orphaned products ===
> >
> > Given the current level of investment in MXNet and the stakeholders using
> > it - the risk of the project being abandoned is minimal. Amazon, for
> > example, is in active development to use MXNet in many of its services
> and
> > many large corporations use it in their production applications.
> >
> > === Inexperience with Open Source ===
> >
> > MXNet has existed as a healthy open source project for more than a year.
> > During that time, the project has attracted 200+ contributors.
> >
> > === Homogenous Developers ===
> >
> > The initial list of committers and contributors includes developers from
> > several institutions and industry participants (see above).
> >
> > === Reliance on Salaried Developers ===
> >
> > Like most open source projects, MXNet receives a substantial support from
> > salaried developers. A large fraction of MXNet development is supported
> by
> > graduate students at various universities in the course of research
> degrees
> > - this is more a “volunteer” relationship, since in most cases students
> > contribute vastly more than is necessary to immediately support research.
> > In addition, those working from within corporations are devoting
> > significant time and effort in the project - and these come from several
> > organizations.
> >
> > === A Excessive Fascination with the Apache Brand ===
> >
> > We choose Apache not for publicity. We have two purposes. First, we hope
> > that Apache's known best-practices for managing a mature open source
> > project can help guide us.  For example, we are feeling the growing pains
> > of a successful open source project as we attempt a major refactor of the
> > internals while customers are using the system in production. We seek
> > guidance in communicating breaking API changes and version revisions.
> > Also, as our involvement from major corporations increases, we want to
> > assure our users that MXNet will stay open and not favor any particular
> > platform or environment. These are some examples of the know-how and
> > discipline we're hoping Apache can bring to our project.
> >
> > Second, we want to leverage Apache's reputation to recruit more
> developers
> > to create a diverse community.
> >
> > === Relationship with Other Apache Products ===
> >
> > Apache Mahout and Apache Spark's MLlib are general machine learning
> > systems. Deep learning algorithms can thus be implemented on these two
> > platforms as well. However, in practice, the overlap will be minimal.
> Deep
> > learning is so computationally intensive that it often requires
> specialized
> > GPU hardware to accomplish tasks of meaningful size.  Making efficient
> use
> > of GPU hardware is complex because the hardware is so fast that the
> > supporting systems around it must be carefully optimized to keep the GPU
> > cores busy.  Extending this capability to distributed multi-GPU and
> > multi-host environments requires great care.  This is a critical
> > differentiator between MXNet and existing Apache machine learning
> systems.
> >
> > Mahout and Spark ML-LIB follow models where their nodes run
> synchronously.
> > This is the fundamental difference to MXNet who follows the parameter
> > server framework. MXNet can run synchronously or asynchronously. In
> > addition, MXNet has optimizations for training a wide range of deep
> > learning models using a variety of approaches (e.g., model parallelism
> and
> > data parallelism) which makes MXNet much more efficient (near-linear
> > speedup on state of the art models). MXNet also supports both imperative
> > and symbolic approaches providing ease of programming for deep learning
> > algorithms.
> >
> > Other Apache projects that are potentially complimentary:
> >
> > Apache Arrow - read data in Apache Arrow‘s internal format from MXNet,
> that
> > would allow users to run ETL/preprocessing in Spark, save the results in
> > Arrow’s format and then run DL algorithms on it.
> >
> > Apache Singa - MXNet and Singa are both deep learning projects, and can
> > benefit from a larger deep learning community at Apache.
> >
> > == Documentation ==
> >
> > Documentation has recently migrated to http://mxnet.io.  We continue to
> > refine and improve the documentation.
> >
> > == Initial Source ==
> >
> > We currently use Github to maintain our source code,
> > https://github.com/MXNet
> >
> > == Source and Intellectual Property Submission Plan ==
> >
> > MXNet Code is available under Apache License, Version 2.0. We will work
> > with the committers to get CLAs signed and review previous contributions.
> >
> > == External Dependencies ==
> >
> >  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> > (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> > lib-zeromq), TBB
> >  * required for GPU usage: cudnn, cuda
> >  * required for python usage: Python 2/3
> >  * required for R module: R, Rcpp (GPLv2 licensing)
> >  * optional for image preparation and preprocessing: opencv
> >  * optional dependencies for additional features: torch7, numba, cython
> (in
> > NNVM branch)
> >
> > Rcpt and lib-zeromq are expected to be licensing discussions.
> >
> > == Cryptography ==
> >
> > Not Applicable
> >
> > == Required Resources ==
> >
> > === Mailing Lists ===
> >
> > There is currently no mailing list.
> >
> > === Issue Tracking ===
> >
> > Currently uses GitHub to track issues. Would like to continue to do so.
> >
> > == Committers and Affiliations ==
> >
> >  * Tianqi Chen (UW)
> >  * Mu Li (AWS)
> >  * Junyuan Xie (AWS)
> >  * Bing Xu (Apple)
> >  * Chiyuan Zhang (MIT)
> >  * Minjie Wang (UYU)
> >  * Naiyan Wang (Tusimple)
> >  * Yizhi Liu (Mediav)
> >  * Tong He (Simon Fraser University)
> >  * Qiang Kou (Indiana U)
> >  * Xingjian Shi (HKUST)
> >
> > == Sponsors ==
> >
> > === Champion ===
> >
> > Henri Yandell (bayard at apache.org)
> >
> > === Nominated Mentors ===
> >
> > Sebastian Schelter (ssc@apache.org)
> >
> >
> > === Sponsoring Entity ===
> >
> > We are requesting the Incubator to sponsor this project.
> >
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henry Saputra <he...@gmail.com>]

Hi Henri,

Could you update the proposal for === Mailing Lists === section to list
existing and required mailing lists from Incubator?

Thanks,

Henry

On Thu, Jan 5, 2017 at 9:12 PM, Henri Yandell <ba...@apache.org> wrote:

> Hello Incubator,
>
> I'd like to propose a new incubator Apache MXNet podling.
>
> The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> committers,
> 200 contributors) is very interested in joining Apache. MXNet is an
> open-source deep learning framework that allows you to define, train, and
> deploy deep neural networks on a wide array of devices, from cloud
> infrastructure to mobile devices.
>
> The wiki proposal page is located here:
>
>   https://wiki.apache.org/incubator/MXNetProposal
>
> I've included the text below in case anyone wants to focus on parts of it
> in a reply.
>
> Looking forward to your thoughts, and for lots of interested Apache members
> to volunteer to mentor the project in addition to Sebastian and myself.
>
> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!
>
> Thanks,
>
> Hen
> On behalf of the MXNet project
>
> ---------
>
> = MXNet: Apache Incubator Proposal =
>
> == Abstract ==
>
> MXNet is a Flexible and Efficient Library for Deep Learning
>
> == Proposal ==
>
> MXNet is an open-source deep learning framework that allows you to define,
> train, and deploy deep neural networks on a wide array of devices, from
> cloud infrastructure to mobile devices. It is highly scalable, allowing for
> fast model training, and supports a flexible programming model and multiple
> languages. MXNet allows you to mix symbolic and imperative programming
> flavors to maximize both efficiency and productivity. MXNet is built on a
> dynamic dependency scheduler that automatically parallelizes both symbolic
> and imperative operations on the fly. A graph optimization layer on top of
> that makes symbolic execution fast and memory efficient. The MXNet library
> is portable and lightweight, and it scales to multiple GPUs and multiple
> machines.
>
> == Background ==
>
> Deep learning is a subset of Machine learning and refers to a class of
> algorithms that use a hierarchical approach with non-linearities to
> discover and learn representations within data. Deep Learning has recently
> become very popular due to its applicability and advancement of domains
> such as Computer Vision, Speech Recognition, Natural Language Understanding
> and Recommender Systems. With pervasive and cost effective cloud computing,
> large labeled datasets and continued algorithmic innovation, Deep Learning
> has become the one of the most popular classes of algorithms for machine
> learning practitioners in recent years.
>
> == Rational ==
>
> The adoption of deep learning is quickly expanding from initial deep domain
> experts rooted in academia to data scientists and developers working to
> deploy intelligent services and products. Deep learning however has many
> challenges.  These include model training time (which can take days to
> weeks), programmability (not everyone writes Python or C++ and like
> symbolic programming) and balancing production readiness (support for
> things like failover) with development flexibility (ability to program
> different ways, support for new operators and model types) and speed of
> execution (fast and scalable model training).  Other frameworks excel on
> some but not all of these aspects.
>
>
> == Initial Goals ==
>
> MXNet is a fairly established project on GitHub with its first code
> contribution in April 2015 and roughly 200 contributors. It is used by
> several large companies and some of the top research institutions on the
> planet. Initial goals would be the following:
>
>  1. Move the existing codebase(s) to Apache
>  1. Integrate with the Apache development process/sign CLAs
>  1. Ensure all dependencies are compliant with Apache License version 2.0
>  1. Incremental development and releases per Apache guidelines
>  1. Establish engineering discipline and a predictable release cadence of
> high quality releases
>  1. Expand the community beyond the current base of expert level users
>  1. Improve usability and the overall developer/user experience
>  1. Add additional functionality to address newer problem types and
> algorithms
>
>
> == Current Status ==
>
> === Meritocracy ===
>
> The MXNet project already operates on meritocratic principles. Today, MXNet
> has developers worldwide and has accepted multiple major patches from a
> diverse set of contributors within both industry and academia. We would
> like to follow ASF meritocratic principles to encourage more developers to
> contribute in this project. We know that only active and committed
> developers from a diverse set of backgrounds can make MXNet a successful
> project.  We are also improving the documentation and code to help new
> developers get started quickly.
>
> === Community ===
>
> Acceptance into the Apache foundation would bolster the growing user and
> developer community around MXNet. That community includes around 200
> contributors from academia and industry. The core developers of our project
> are listed in our contributors below and are also represented by logos on
> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
> University of Washington and Wolfram.
>
> === Core Developers ===
>
> (with GitHub logins)
>
>  * Tianqi Chen (@tqchen)
>  * Mu Li (@mli)
>  * Junyuan Xie (@piiswrong)
>  * Bing Xu (@antinucleon)
>  * Chiyuan Zhang (@pluskid)
>  * Minjie Wang (@jermainewang)
>  * Naiyan Wang (@winstywang)
>  * Yizhi Liu (@javelinjs)
>  * Tong He (@hetong007)
>  * Qiang Kou (@thirdwing)
>  * Xingjian Shi (@sxjscience)
>
> === Alignment ===
>
> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
> and Mahout, each of which targets a different application domain. MXNet,
> being a distributed platform for large-scale deep learning, focuses on
> another important domain for which there still lacks a scalable,
> programmable, flexible and super fast open-source platform. The recent
> success of deep learning models especially for vision and speech
> recognition tasks has generated interests in both applying existing deep
> learning models and in developing new ones. Thus, an open-source platform
> for deep learning backed by some of the top industry and academic players
> will be able to attract a large community of users and developers. MXNet is
> a complex system needing many iterations of design, implementation and
> testing. Apache's collaboration framework which encourages active
> contribution from developers will inevitably help improve the quality of
> the system, as shown in the success of Hadoop, Spark, etc. Equally
> important is the community of users which helps identify real-life
> applications of deep learning, and helps to evaluate the system's
> performance and ease-of-use. We hope to leverage ASF for coordinating and
> promoting both communities, and in return benefit the communities with
> another useful tool.
>
> == Known Risks ==
>
> === Orphaned products ===
>
> Given the current level of investment in MXNet and the stakeholders using
> it - the risk of the project being abandoned is minimal. Amazon, for
> example, is in active development to use MXNet in many of its services and
> many large corporations use it in their production applications.
>
> === Inexperience with Open Source ===
>
> MXNet has existed as a healthy open source project for more than a year.
> During that time, the project has attracted 200+ contributors.
>
> === Homogenous Developers ===
>
> The initial list of committers and contributors includes developers from
> several institutions and industry participants (see above).
>
> === Reliance on Salaried Developers ===
>
> Like most open source projects, MXNet receives a substantial support from
> salaried developers. A large fraction of MXNet development is supported by
> graduate students at various universities in the course of research degrees
> - this is more a “volunteer” relationship, since in most cases students
> contribute vastly more than is necessary to immediately support research.
> In addition, those working from within corporations are devoting
> significant time and effort in the project - and these come from several
> organizations.
>
> === A Excessive Fascination with the Apache Brand ===
>
> We choose Apache not for publicity. We have two purposes. First, we hope
> that Apache's known best-practices for managing a mature open source
> project can help guide us.  For example, we are feeling the growing pains
> of a successful open source project as we attempt a major refactor of the
> internals while customers are using the system in production. We seek
> guidance in communicating breaking API changes and version revisions.
> Also, as our involvement from major corporations increases, we want to
> assure our users that MXNet will stay open and not favor any particular
> platform or environment. These are some examples of the know-how and
> discipline we're hoping Apache can bring to our project.
>
> Second, we want to leverage Apache's reputation to recruit more developers
> to create a diverse community.
>
> === Relationship with Other Apache Products ===
>
> Apache Mahout and Apache Spark's MLlib are general machine learning
> systems. Deep learning algorithms can thus be implemented on these two
> platforms as well. However, in practice, the overlap will be minimal.  Deep
> learning is so computationally intensive that it often requires specialized
> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
> of GPU hardware is complex because the hardware is so fast that the
> supporting systems around it must be carefully optimized to keep the GPU
> cores busy.  Extending this capability to distributed multi-GPU and
> multi-host environments requires great care.  This is a critical
> differentiator between MXNet and existing Apache machine learning systems.
>
> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
> This is the fundamental difference to MXNet who follows the parameter
> server framework. MXNet can run synchronously or asynchronously. In
> addition, MXNet has optimizations for training a wide range of deep
> learning models using a variety of approaches (e.g., model parallelism and
> data parallelism) which makes MXNet much more efficient (near-linear
> speedup on state of the art models). MXNet also supports both imperative
> and symbolic approaches providing ease of programming for deep learning
> algorithms.
>
> Other Apache projects that are potentially complimentary:
>
> Apache Arrow - read data in Apache Arrow‘s internal format from MXNet, that
> would allow users to run ETL/preprocessing in Spark, save the results in
> Arrow’s format and then run DL algorithms on it.
>
> Apache Singa - MXNet and Singa are both deep learning projects, and can
> benefit from a larger deep learning community at Apache.
>
> == Documentation ==
>
> Documentation has recently migrated to http://mxnet.io.  We continue to
> refine and improve the documentation.
>
> == Initial Source ==
>
> We currently use Github to maintain our source code,
> https://github.com/MXNet
>
> == Source and Intellectual Property Submission Plan ==
>
> MXNet Code is available under Apache License, Version 2.0. We will work
> with the committers to get CLAs signed and review previous contributions.
>
> == External Dependencies ==
>
>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> lib-zeromq), TBB
>  * required for GPU usage: cudnn, cuda
>  * required for python usage: Python 2/3
>  * required for R module: R, Rcpp (GPLv2 licensing)
>  * optional for image preparation and preprocessing: opencv
>  * optional dependencies for additional features: torch7, numba, cython (in
> NNVM branch)
>
> Rcpt and lib-zeromq are expected to be licensing discussions.
>
> == Cryptography ==
>
> Not Applicable
>
> == Required Resources ==
>
> === Mailing Lists ===
>
> There is currently no mailing list.
>
> === Issue Tracking ===
>
> Currently uses GitHub to track issues. Would like to continue to do so.
>
> == Committers and Affiliations ==
>
>  * Tianqi Chen (UW)
>  * Mu Li (AWS)
>  * Junyuan Xie (AWS)
>  * Bing Xu (Apple)
>  * Chiyuan Zhang (MIT)
>  * Minjie Wang (UYU)
>  * Naiyan Wang (Tusimple)
>  * Yizhi Liu (Mediav)
>  * Tong He (Simon Fraser University)
>  * Qiang Kou (Indiana U)
>  * Xingjian Shi (HKUST)
>
> == Sponsors ==
>
> === Champion ===
>
> Henri Yandell (bayard at apache.org)
>
> === Nominated Mentors ===
>
> Sebastian Schelter (ssc@apache.org)
>
>
> === Sponsoring Entity ===
>
> We are requesting the Incubator to sponsor this project.
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Thanks Markus - I've added you to the proposal :)

On Fri, Jan 6, 2017 at 9:20 AM, Markus Weimer <ma...@weimo.de> wrote:

> On 2017-01-05 9:12 PM, Henri Yandell wrote:
>
>> I'd like to propose a new incubator Apache MXNet podling.
>>
>
> Awesome! If you still need a mentor, feel free to sign me up!
>
> Markus
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Sebastian <ss...@apache.org>]

Great to see you join our efforts!

On 06.01.2017 18:20, Markus Weimer wrote:
> On 2017-01-05 9:12 PM, Henri Yandell wrote:
>> I'd like to propose a new incubator Apache MXNet podling.
>
> Awesome! If you still need a mentor, feel free to sign me up!
>
> Markus
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Markus Weimer <ma...@weimo.de>]

On 2017-01-05 9:12 PM, Henri Yandell wrote:
> I'd like to propose a new incubator Apache MXNet podling.

Awesome! If you still need a mentor, feel free to sign me up!

Markus

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Thanks Ziheng :)

I've added you to the page.

Hen

On Wed, Jan 11, 2017 at 9:35 PM, ziheng <jz...@gmail.com> wrote:

> Hello
>
> Please sign me up as a committer for MXNet - I've been working on MXNet
> Profiler and NNVM-Fusion, also will work with Mu as an intern at Amazon AWS
> team in the next months, and I would love to get more involved in the
> project.
>
> GitHub ID:ZihengJiang
> Affiliations: AWS
>
>
> Thanks
>
>
>
> --
> View this message in context: http://apache-incubator-
> general.996316.n3.nabble.com/DISCUSS-Proposing-MXNet-for-
> the-Apache-Incubator-tp53250p53335.html
> Sent from the Apache Incubator - General mailing list archive at
> Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[ziheng <jz...@gmail.com>]

Hello 

Please sign me up as a committer for MXNet - I've been working on MXNet
Profiler and NNVM-Fusion, also will work with Mu as an intern at Amazon AWS
team in the next months, and I would love to get more involved in the
project. 

GitHub ID:ZihengJiang
Affiliations: AWS


Thanks



--
View this message in context: http://apache-incubator-general.996316.n3.nabble.com/DISCUSS-Proposing-MXNet-for-the-Apache-Incubator-tp53250p53335.html
Sent from the Apache Incubator - General mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Tsuyoshi Ozawa <oz...@apache.org>]

Apache SystemML(incubating) is also a flexible, scalable machine
learning system[4].

> Apache SystemML provides an optimal workplace for machine learning using big data. It can be run on top of Apache Spark, where it automatically scales your data, line by line, determining whether your code should be run on the driver or an Apache Spark cluster.

[4] https://systemml.apache.org/

Thanks,
- Tsuyoshi

On Fri, Jan 6, 2017 at 6:11 PM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi Henri,
>
> It's a great news! Looking forwarding to MXNet's coming to Apache Incubator :-)
>
> Two minor comments:
>
>> We currently use Github to maintain our source code,
>> https://github.com/MXNet
>
> In my understanding, the following url is correct one.
> https://github.com/dmlc/mxnet
>
>> === Relationship with Other Apache Products ===
>
> As far as I know, there are 2 additional machine learning libraries in
> addition to the projects you mentioned.
> Apache MADlib(incubating)[1] is a machine learning library, which can
> run on SQL system(Greenplum/Apache HAWQ(incubating)/PostgreSQL).
> Apache Hivemall(incubating)[2] are also machine learning library,
> which can run on Hadoop ecosystem: Apache Spark/Apache Hive/Apache
> Pig. Especially for Hivemall project, it has MIX server, one kind of
> parameter server to exchange parameters between mappers[3].
>
> This is just a sharing, and I don't mean you should add these projects
> to your comment.
>
> [1] http://madlib.incubator.apache.org/
> [2] https://hivemall.incubator.apache.org/
> [3] https://hivemall.incubator.apache.org/userguide/tips/mixserver.html
>
> Thanks,
> - Tsuyoshi
>
> On Fri, Jan 6, 2017 at 4:32 PM, Henry Saputra <he...@gmail.com> wrote:
>> This is great news and I am looking forward to it =)
>>
>> According to proposal, the community want to stick with Github issues for
>> tracking issues and bugs?
>> I suppose this needs a nod by Greg Stein as rep from Apache Infra to
>> confirm that this is ok for incubation and how would it impact during
>> graduation.
>>
>> - Henry
>>
>> On Thu, Jan 5, 2017 at 9:12 PM, Henri Yandell <ba...@apache.org> wrote:
>>
>>> Hello Incubator,
>>>
>>> I'd like to propose a new incubator Apache MXNet podling.
>>>
>>> The existing MXNet project (http://mxnet.io - 1.5 years old, 15
>>> committers,
>>> 200 contributors) is very interested in joining Apache. MXNet is an
>>> open-source deep learning framework that allows you to define, train, and
>>> deploy deep neural networks on a wide array of devices, from cloud
>>> infrastructure to mobile devices.
>>>
>>> The wiki proposal page is located here:
>>>
>>>   https://wiki.apache.org/incubator/MXNetProposal
>>>
>>> I've included the text below in case anyone wants to focus on parts of it
>>> in a reply.
>>>
>>> Looking forward to your thoughts, and for lots of interested Apache members
>>> to volunteer to mentor the project in addition to Sebastian and myself.
>>>
>>> Currently the list of committers is based on the current active coders, so
>>> we're also very interested in hearing from anyone else who is interested in
>>> working on the project, be they current or future contributor!
>>>
>>> Thanks,
>>>
>>> Hen
>>> On behalf of the MXNet project
>>>
>>> ---------
>>>
>>> = MXNet: Apache Incubator Proposal =
>>>
>>> == Abstract ==
>>>
>>> MXNet is a Flexible and Efficient Library for Deep Learning
>>>
>>> == Proposal ==
>>>
>>> MXNet is an open-source deep learning framework that allows you to define,
>>> train, and deploy deep neural networks on a wide array of devices, from
>>> cloud infrastructure to mobile devices. It is highly scalable, allowing for
>>> fast model training, and supports a flexible programming model and multiple
>>> languages. MXNet allows you to mix symbolic and imperative programming
>>> flavors to maximize both efficiency and productivity. MXNet is built on a
>>> dynamic dependency scheduler that automatically parallelizes both symbolic
>>> and imperative operations on the fly. A graph optimization layer on top of
>>> that makes symbolic execution fast and memory efficient. The MXNet library
>>> is portable and lightweight, and it scales to multiple GPUs and multiple
>>> machines.
>>>
>>> == Background ==
>>>
>>> Deep learning is a subset of Machine learning and refers to a class of
>>> algorithms that use a hierarchical approach with non-linearities to
>>> discover and learn representations within data. Deep Learning has recently
>>> become very popular due to its applicability and advancement of domains
>>> such as Computer Vision, Speech Recognition, Natural Language Understanding
>>> and Recommender Systems. With pervasive and cost effective cloud computing,
>>> large labeled datasets and continued algorithmic innovation, Deep Learning
>>> has become the one of the most popular classes of algorithms for machine
>>> learning practitioners in recent years.
>>>
>>> == Rational ==
>>>
>>> The adoption of deep learning is quickly expanding from initial deep domain
>>> experts rooted in academia to data scientists and developers working to
>>> deploy intelligent services and products. Deep learning however has many
>>> challenges.  These include model training time (which can take days to
>>> weeks), programmability (not everyone writes Python or C++ and like
>>> symbolic programming) and balancing production readiness (support for
>>> things like failover) with development flexibility (ability to program
>>> different ways, support for new operators and model types) and speed of
>>> execution (fast and scalable model training).  Other frameworks excel on
>>> some but not all of these aspects.
>>>
>>>
>>> == Initial Goals ==
>>>
>>> MXNet is a fairly established project on GitHub with its first code
>>> contribution in April 2015 and roughly 200 contributors. It is used by
>>> several large companies and some of the top research institutions on the
>>> planet. Initial goals would be the following:
>>>
>>>  1. Move the existing codebase(s) to Apache
>>>  1. Integrate with the Apache development process/sign CLAs
>>>  1. Ensure all dependencies are compliant with Apache License version 2.0
>>>  1. Incremental development and releases per Apache guidelines
>>>  1. Establish engineering discipline and a predictable release cadence of
>>> high quality releases
>>>  1. Expand the community beyond the current base of expert level users
>>>  1. Improve usability and the overall developer/user experience
>>>  1. Add additional functionality to address newer problem types and
>>> algorithms
>>>
>>>
>>> == Current Status ==
>>>
>>> === Meritocracy ===
>>>
>>> The MXNet project already operates on meritocratic principles. Today, MXNet
>>> has developers worldwide and has accepted multiple major patches from a
>>> diverse set of contributors within both industry and academia. We would
>>> like to follow ASF meritocratic principles to encourage more developers to
>>> contribute in this project. We know that only active and committed
>>> developers from a diverse set of backgrounds can make MXNet a successful
>>> project.  We are also improving the documentation and code to help new
>>> developers get started quickly.
>>>
>>> === Community ===
>>>
>>> Acceptance into the Apache foundation would bolster the growing user and
>>> developer community around MXNet. That community includes around 200
>>> contributors from academia and industry. The core developers of our project
>>> are listed in our contributors below and are also represented by logos on
>>> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
>>> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
>>> University of Washington and Wolfram.
>>>
>>> === Core Developers ===
>>>
>>> (with GitHub logins)
>>>
>>>  * Tianqi Chen (@tqchen)
>>>  * Mu Li (@mli)
>>>  * Junyuan Xie (@piiswrong)
>>>  * Bing Xu (@antinucleon)
>>>  * Chiyuan Zhang (@pluskid)
>>>  * Minjie Wang (@jermainewang)
>>>  * Naiyan Wang (@winstywang)
>>>  * Yizhi Liu (@javelinjs)
>>>  * Tong He (@hetong007)
>>>  * Qiang Kou (@thirdwing)
>>>  * Xingjian Shi (@sxjscience)
>>>
>>> === Alignment ===
>>>
>>> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
>>> and Mahout, each of which targets a different application domain. MXNet,
>>> being a distributed platform for large-scale deep learning, focuses on
>>> another important domain for which there still lacks a scalable,
>>> programmable, flexible and super fast open-source platform. The recent
>>> success of deep learning models especially for vision and speech
>>> recognition tasks has generated interests in both applying existing deep
>>> learning models and in developing new ones. Thus, an open-source platform
>>> for deep learning backed by some of the top industry and academic players
>>> will be able to attract a large community of users and developers. MXNet is
>>> a complex system needing many iterations of design, implementation and
>>> testing. Apache's collaboration framework which encourages active
>>> contribution from developers will inevitably help improve the quality of
>>> the system, as shown in the success of Hadoop, Spark, etc. Equally
>>> important is the community of users which helps identify real-life
>>> applications of deep learning, and helps to evaluate the system's
>>> performance and ease-of-use. We hope to leverage ASF for coordinating and
>>> promoting both communities, and in return benefit the communities with
>>> another useful tool.
>>>
>>> == Known Risks ==
>>>
>>> === Orphaned products ===
>>>
>>> Given the current level of investment in MXNet and the stakeholders using
>>> it - the risk of the project being abandoned is minimal. Amazon, for
>>> example, is in active development to use MXNet in many of its services and
>>> many large corporations use it in their production applications.
>>>
>>> === Inexperience with Open Source ===
>>>
>>> MXNet has existed as a healthy open source project for more than a year.
>>> During that time, the project has attracted 200+ contributors.
>>>
>>> === Homogenous Developers ===
>>>
>>> The initial list of committers and contributors includes developers from
>>> several institutions and industry participants (see above).
>>>
>>> === Reliance on Salaried Developers ===
>>>
>>> Like most open source projects, MXNet receives a substantial support from
>>> salaried developers. A large fraction of MXNet development is supported by
>>> graduate students at various universities in the course of research degrees
>>> - this is more a “volunteer” relationship, since in most cases students
>>> contribute vastly more than is necessary to immediately support research.
>>> In addition, those working from within corporations are devoting
>>> significant time and effort in the project - and these come from several
>>> organizations.
>>>
>>> === A Excessive Fascination with the Apache Brand ===
>>>
>>> We choose Apache not for publicity. We have two purposes. First, we hope
>>> that Apache's known best-practices for managing a mature open source
>>> project can help guide us.  For example, we are feeling the growing pains
>>> of a successful open source project as we attempt a major refactor of the
>>> internals while customers are using the system in production. We seek
>>> guidance in communicating breaking API changes and version revisions.
>>> Also, as our involvement from major corporations increases, we want to
>>> assure our users that MXNet will stay open and not favor any particular
>>> platform or environment. These are some examples of the know-how and
>>> discipline we're hoping Apache can bring to our project.
>>>
>>> Second, we want to leverage Apache's reputation to recruit more developers
>>> to create a diverse community.
>>>
>>> === Relationship with Other Apache Products ===
>>>
>>> Apache Mahout and Apache Spark's MLlib are general machine learning
>>> systems. Deep learning algorithms can thus be implemented on these two
>>> platforms as well. However, in practice, the overlap will be minimal.  Deep
>>> learning is so computationally intensive that it often requires specialized
>>> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
>>> of GPU hardware is complex because the hardware is so fast that the
>>> supporting systems around it must be carefully optimized to keep the GPU
>>> cores busy.  Extending this capability to distributed multi-GPU and
>>> multi-host environments requires great care.  This is a critical
>>> differentiator between MXNet and existing Apache machine learning systems.
>>>
>>> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
>>> This is the fundamental difference to MXNet who follows the parameter
>>> server framework. MXNet can run synchronously or asynchronously. In
>>> addition, MXNet has optimizations for training a wide range of deep
>>> learning models using a variety of approaches (e.g., model parallelism and
>>> data parallelism) which makes MXNet much more efficient (near-linear
>>> speedup on state of the art models). MXNet also supports both imperative
>>> and symbolic approaches providing ease of programming for deep learning
>>> algorithms.
>>>
>>> Other Apache projects that are potentially complimentary:
>>>
>>> Apache Arrow - read data in Apache Arrow‘s internal format from MXNet, that
>>> would allow users to run ETL/preprocessing in Spark, save the results in
>>> Arrow’s format and then run DL algorithms on it.
>>>
>>> Apache Singa - MXNet and Singa are both deep learning projects, and can
>>> benefit from a larger deep learning community at Apache.
>>>
>>> == Documentation ==
>>>
>>> Documentation has recently migrated to http://mxnet.io.  We continue to
>>> refine and improve the documentation.
>>>
>>> == Initial Source ==
>>>
>>> We currently use Github to maintain our source code,
>>> https://github.com/MXNet
>>>
>>> == Source and Intellectual Property Submission Plan ==
>>>
>>> MXNet Code is available under Apache License, Version 2.0. We will work
>>> with the committers to get CLAs signed and review previous contributions.
>>>
>>> == External Dependencies ==
>>>
>>>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
>>> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
>>> lib-zeromq), TBB
>>>  * required for GPU usage: cudnn, cuda
>>>  * required for python usage: Python 2/3
>>>  * required for R module: R, Rcpp (GPLv2 licensing)
>>>  * optional for image preparation and preprocessing: opencv
>>>  * optional dependencies for additional features: torch7, numba, cython (in
>>> NNVM branch)
>>>
>>> Rcpt and lib-zeromq are expected to be licensing discussions.
>>>
>>> == Cryptography ==
>>>
>>> Not Applicable
>>>
>>> == Required Resources ==
>>>
>>> === Mailing Lists ===
>>>
>>> There is currently no mailing list.
>>>
>>> === Issue Tracking ===
>>>
>>> Currently uses GitHub to track issues. Would like to continue to do so.
>>>
>>> == Committers and Affiliations ==
>>>
>>>  * Tianqi Chen (UW)
>>>  * Mu Li (AWS)
>>>  * Junyuan Xie (AWS)
>>>  * Bing Xu (Apple)
>>>  * Chiyuan Zhang (MIT)
>>>  * Minjie Wang (UYU)
>>>  * Naiyan Wang (Tusimple)
>>>  * Yizhi Liu (Mediav)
>>>  * Tong He (Simon Fraser University)
>>>  * Qiang Kou (Indiana U)
>>>  * Xingjian Shi (HKUST)
>>>
>>> == Sponsors ==
>>>
>>> === Champion ===
>>>
>>> Henri Yandell (bayard at apache.org)
>>>
>>> === Nominated Mentors ===
>>>
>>> Sebastian Schelter (ssc@apache.org)
>>>
>>>
>>> === Sponsoring Entity ===
>>>
>>> We are requesting the Incubator to sponsor this project.
>>>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Tsuyoshi Ozawa <oz...@apache.org>]

Hi Henri,

It's a great news! Looking forwarding to MXNet's coming to Apache Incubator :-)

Two minor comments:

> We currently use Github to maintain our source code,
> https://github.com/MXNet

In my understanding, the following url is correct one.
https://github.com/dmlc/mxnet

> === Relationship with Other Apache Products ===

As far as I know, there are 2 additional machine learning libraries in
addition to the projects you mentioned.
Apache MADlib(incubating)[1] is a machine learning library, which can
run on SQL system(Greenplum/Apache HAWQ(incubating)/PostgreSQL).
Apache Hivemall(incubating)[2] are also machine learning library,
which can run on Hadoop ecosystem: Apache Spark/Apache Hive/Apache
Pig. Especially for Hivemall project, it has MIX server, one kind of
parameter server to exchange parameters between mappers[3].

This is just a sharing, and I don't mean you should add these projects
to your comment.

[1] http://madlib.incubator.apache.org/
[2] https://hivemall.incubator.apache.org/
[3] https://hivemall.incubator.apache.org/userguide/tips/mixserver.html

Thanks,
- Tsuyoshi

On Fri, Jan 6, 2017 at 4:32 PM, Henry Saputra <he...@gmail.com> wrote:
> This is great news and I am looking forward to it =)
>
> According to proposal, the community want to stick with Github issues for
> tracking issues and bugs?
> I suppose this needs a nod by Greg Stein as rep from Apache Infra to
> confirm that this is ok for incubation and how would it impact during
> graduation.
>
> - Henry
>
> On Thu, Jan 5, 2017 at 9:12 PM, Henri Yandell <ba...@apache.org> wrote:
>
>> Hello Incubator,
>>
>> I'd like to propose a new incubator Apache MXNet podling.
>>
>> The existing MXNet project (http://mxnet.io - 1.5 years old, 15
>> committers,
>> 200 contributors) is very interested in joining Apache. MXNet is an
>> open-source deep learning framework that allows you to define, train, and
>> deploy deep neural networks on a wide array of devices, from cloud
>> infrastructure to mobile devices.
>>
>> The wiki proposal page is located here:
>>
>>   https://wiki.apache.org/incubator/MXNetProposal
>>
>> I've included the text below in case anyone wants to focus on parts of it
>> in a reply.
>>
>> Looking forward to your thoughts, and for lots of interested Apache members
>> to volunteer to mentor the project in addition to Sebastian and myself.
>>
>> Currently the list of committers is based on the current active coders, so
>> we're also very interested in hearing from anyone else who is interested in
>> working on the project, be they current or future contributor!
>>
>> Thanks,
>>
>> Hen
>> On behalf of the MXNet project
>>
>> ---------
>>
>> = MXNet: Apache Incubator Proposal =
>>
>> == Abstract ==
>>
>> MXNet is a Flexible and Efficient Library for Deep Learning
>>
>> == Proposal ==
>>
>> MXNet is an open-source deep learning framework that allows you to define,
>> train, and deploy deep neural networks on a wide array of devices, from
>> cloud infrastructure to mobile devices. It is highly scalable, allowing for
>> fast model training, and supports a flexible programming model and multiple
>> languages. MXNet allows you to mix symbolic and imperative programming
>> flavors to maximize both efficiency and productivity. MXNet is built on a
>> dynamic dependency scheduler that automatically parallelizes both symbolic
>> and imperative operations on the fly. A graph optimization layer on top of
>> that makes symbolic execution fast and memory efficient. The MXNet library
>> is portable and lightweight, and it scales to multiple GPUs and multiple
>> machines.
>>
>> == Background ==
>>
>> Deep learning is a subset of Machine learning and refers to a class of
>> algorithms that use a hierarchical approach with non-linearities to
>> discover and learn representations within data. Deep Learning has recently
>> become very popular due to its applicability and advancement of domains
>> such as Computer Vision, Speech Recognition, Natural Language Understanding
>> and Recommender Systems. With pervasive and cost effective cloud computing,
>> large labeled datasets and continued algorithmic innovation, Deep Learning
>> has become the one of the most popular classes of algorithms for machine
>> learning practitioners in recent years.
>>
>> == Rational ==
>>
>> The adoption of deep learning is quickly expanding from initial deep domain
>> experts rooted in academia to data scientists and developers working to
>> deploy intelligent services and products. Deep learning however has many
>> challenges.  These include model training time (which can take days to
>> weeks), programmability (not everyone writes Python or C++ and like
>> symbolic programming) and balancing production readiness (support for
>> things like failover) with development flexibility (ability to program
>> different ways, support for new operators and model types) and speed of
>> execution (fast and scalable model training).  Other frameworks excel on
>> some but not all of these aspects.
>>
>>
>> == Initial Goals ==
>>
>> MXNet is a fairly established project on GitHub with its first code
>> contribution in April 2015 and roughly 200 contributors. It is used by
>> several large companies and some of the top research institutions on the
>> planet. Initial goals would be the following:
>>
>>  1. Move the existing codebase(s) to Apache
>>  1. Integrate with the Apache development process/sign CLAs
>>  1. Ensure all dependencies are compliant with Apache License version 2.0
>>  1. Incremental development and releases per Apache guidelines
>>  1. Establish engineering discipline and a predictable release cadence of
>> high quality releases
>>  1. Expand the community beyond the current base of expert level users
>>  1. Improve usability and the overall developer/user experience
>>  1. Add additional functionality to address newer problem types and
>> algorithms
>>
>>
>> == Current Status ==
>>
>> === Meritocracy ===
>>
>> The MXNet project already operates on meritocratic principles. Today, MXNet
>> has developers worldwide and has accepted multiple major patches from a
>> diverse set of contributors within both industry and academia. We would
>> like to follow ASF meritocratic principles to encourage more developers to
>> contribute in this project. We know that only active and committed
>> developers from a diverse set of backgrounds can make MXNet a successful
>> project.  We are also improving the documentation and code to help new
>> developers get started quickly.
>>
>> === Community ===
>>
>> Acceptance into the Apache foundation would bolster the growing user and
>> developer community around MXNet. That community includes around 200
>> contributors from academia and industry. The core developers of our project
>> are listed in our contributors below and are also represented by logos on
>> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
>> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
>> University of Washington and Wolfram.
>>
>> === Core Developers ===
>>
>> (with GitHub logins)
>>
>>  * Tianqi Chen (@tqchen)
>>  * Mu Li (@mli)
>>  * Junyuan Xie (@piiswrong)
>>  * Bing Xu (@antinucleon)
>>  * Chiyuan Zhang (@pluskid)
>>  * Minjie Wang (@jermainewang)
>>  * Naiyan Wang (@winstywang)
>>  * Yizhi Liu (@javelinjs)
>>  * Tong He (@hetong007)
>>  * Qiang Kou (@thirdwing)
>>  * Xingjian Shi (@sxjscience)
>>
>> === Alignment ===
>>
>> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
>> and Mahout, each of which targets a different application domain. MXNet,
>> being a distributed platform for large-scale deep learning, focuses on
>> another important domain for which there still lacks a scalable,
>> programmable, flexible and super fast open-source platform. The recent
>> success of deep learning models especially for vision and speech
>> recognition tasks has generated interests in both applying existing deep
>> learning models and in developing new ones. Thus, an open-source platform
>> for deep learning backed by some of the top industry and academic players
>> will be able to attract a large community of users and developers. MXNet is
>> a complex system needing many iterations of design, implementation and
>> testing. Apache's collaboration framework which encourages active
>> contribution from developers will inevitably help improve the quality of
>> the system, as shown in the success of Hadoop, Spark, etc. Equally
>> important is the community of users which helps identify real-life
>> applications of deep learning, and helps to evaluate the system's
>> performance and ease-of-use. We hope to leverage ASF for coordinating and
>> promoting both communities, and in return benefit the communities with
>> another useful tool.
>>
>> == Known Risks ==
>>
>> === Orphaned products ===
>>
>> Given the current level of investment in MXNet and the stakeholders using
>> it - the risk of the project being abandoned is minimal. Amazon, for
>> example, is in active development to use MXNet in many of its services and
>> many large corporations use it in their production applications.
>>
>> === Inexperience with Open Source ===
>>
>> MXNet has existed as a healthy open source project for more than a year.
>> During that time, the project has attracted 200+ contributors.
>>
>> === Homogenous Developers ===
>>
>> The initial list of committers and contributors includes developers from
>> several institutions and industry participants (see above).
>>
>> === Reliance on Salaried Developers ===
>>
>> Like most open source projects, MXNet receives a substantial support from
>> salaried developers. A large fraction of MXNet development is supported by
>> graduate students at various universities in the course of research degrees
>> - this is more a “volunteer” relationship, since in most cases students
>> contribute vastly more than is necessary to immediately support research.
>> In addition, those working from within corporations are devoting
>> significant time and effort in the project - and these come from several
>> organizations.
>>
>> === A Excessive Fascination with the Apache Brand ===
>>
>> We choose Apache not for publicity. We have two purposes. First, we hope
>> that Apache's known best-practices for managing a mature open source
>> project can help guide us.  For example, we are feeling the growing pains
>> of a successful open source project as we attempt a major refactor of the
>> internals while customers are using the system in production. We seek
>> guidance in communicating breaking API changes and version revisions.
>> Also, as our involvement from major corporations increases, we want to
>> assure our users that MXNet will stay open and not favor any particular
>> platform or environment. These are some examples of the know-how and
>> discipline we're hoping Apache can bring to our project.
>>
>> Second, we want to leverage Apache's reputation to recruit more developers
>> to create a diverse community.
>>
>> === Relationship with Other Apache Products ===
>>
>> Apache Mahout and Apache Spark's MLlib are general machine learning
>> systems. Deep learning algorithms can thus be implemented on these two
>> platforms as well. However, in practice, the overlap will be minimal.  Deep
>> learning is so computationally intensive that it often requires specialized
>> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
>> of GPU hardware is complex because the hardware is so fast that the
>> supporting systems around it must be carefully optimized to keep the GPU
>> cores busy.  Extending this capability to distributed multi-GPU and
>> multi-host environments requires great care.  This is a critical
>> differentiator between MXNet and existing Apache machine learning systems.
>>
>> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
>> This is the fundamental difference to MXNet who follows the parameter
>> server framework. MXNet can run synchronously or asynchronously. In
>> addition, MXNet has optimizations for training a wide range of deep
>> learning models using a variety of approaches (e.g., model parallelism and
>> data parallelism) which makes MXNet much more efficient (near-linear
>> speedup on state of the art models). MXNet also supports both imperative
>> and symbolic approaches providing ease of programming for deep learning
>> algorithms.
>>
>> Other Apache projects that are potentially complimentary:
>>
>> Apache Arrow - read data in Apache Arrow‘s internal format from MXNet, that
>> would allow users to run ETL/preprocessing in Spark, save the results in
>> Arrow’s format and then run DL algorithms on it.
>>
>> Apache Singa - MXNet and Singa are both deep learning projects, and can
>> benefit from a larger deep learning community at Apache.
>>
>> == Documentation ==
>>
>> Documentation has recently migrated to http://mxnet.io.  We continue to
>> refine and improve the documentation.
>>
>> == Initial Source ==
>>
>> We currently use Github to maintain our source code,
>> https://github.com/MXNet
>>
>> == Source and Intellectual Property Submission Plan ==
>>
>> MXNet Code is available under Apache License, Version 2.0. We will work
>> with the committers to get CLAs signed and review previous contributions.
>>
>> == External Dependencies ==
>>
>>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
>> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
>> lib-zeromq), TBB
>>  * required for GPU usage: cudnn, cuda
>>  * required for python usage: Python 2/3
>>  * required for R module: R, Rcpp (GPLv2 licensing)
>>  * optional for image preparation and preprocessing: opencv
>>  * optional dependencies for additional features: torch7, numba, cython (in
>> NNVM branch)
>>
>> Rcpt and lib-zeromq are expected to be licensing discussions.
>>
>> == Cryptography ==
>>
>> Not Applicable
>>
>> == Required Resources ==
>>
>> === Mailing Lists ===
>>
>> There is currently no mailing list.
>>
>> === Issue Tracking ===
>>
>> Currently uses GitHub to track issues. Would like to continue to do so.
>>
>> == Committers and Affiliations ==
>>
>>  * Tianqi Chen (UW)
>>  * Mu Li (AWS)
>>  * Junyuan Xie (AWS)
>>  * Bing Xu (Apple)
>>  * Chiyuan Zhang (MIT)
>>  * Minjie Wang (UYU)
>>  * Naiyan Wang (Tusimple)
>>  * Yizhi Liu (Mediav)
>>  * Tong He (Simon Fraser University)
>>  * Qiang Kou (Indiana U)
>>  * Xingjian Shi (HKUST)
>>
>> == Sponsors ==
>>
>> === Champion ===
>>
>> Henri Yandell (bayard at apache.org)
>>
>> === Nominated Mentors ===
>>
>> Sebastian Schelter (ssc@apache.org)
>>
>>
>> === Sponsoring Entity ===
>>
>> We are requesting the Incubator to sponsor this project.
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Understood. I saw that Greg had recently approved another podling to do
this. Though, assuming approved, there will still need to be some infra
headscratching on the 3,000 issues currently on the main dmlc/mxnet repo
and how imports are best done :) The simplest would be to transfer the
current repo as is over at GitHub - not sure if that's been done before.

On Thu, Jan 5, 2017 at 11:32 PM, Henry Saputra <he...@gmail.com>
wrote:

> This is great news and I am looking forward to it =)
>
> According to proposal, the community want to stick with Github issues for
> tracking issues and bugs?
> I suppose this needs a nod by Greg Stein as rep from Apache Infra to
> confirm that this is ok for incubation and how would it impact during
> graduation.
>
> - Henry
>
> On Thu, Jan 5, 2017 at 9:12 PM, Henri Yandell <ba...@apache.org> wrote:
>
> > Hello Incubator,
> >
> > I'd like to propose a new incubator Apache MXNet podling.
> >
> > The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> > committers,
> > 200 contributors) is very interested in joining Apache. MXNet is an
> > open-source deep learning framework that allows you to define, train, and
> > deploy deep neural networks on a wide array of devices, from cloud
> > infrastructure to mobile devices.
> >
> > The wiki proposal page is located here:
> >
> >   https://wiki.apache.org/incubator/MXNetProposal
> >
> > I've included the text below in case anyone wants to focus on parts of it
> > in a reply.
> >
> > Looking forward to your thoughts, and for lots of interested Apache
> members
> > to volunteer to mentor the project in addition to Sebastian and myself.
> >
> > Currently the list of committers is based on the current active coders,
> so
> > we're also very interested in hearing from anyone else who is interested
> in
> > working on the project, be they current or future contributor!
> >
> > Thanks,
> >
> > Hen
> > On behalf of the MXNet project
> >
> > ---------
> >
> > = MXNet: Apache Incubator Proposal =
> >
> > == Abstract ==
> >
> > MXNet is a Flexible and Efficient Library for Deep Learning
> >
> > == Proposal ==
> >
> > MXNet is an open-source deep learning framework that allows you to
> define,
> > train, and deploy deep neural networks on a wide array of devices, from
> > cloud infrastructure to mobile devices. It is highly scalable, allowing
> for
> > fast model training, and supports a flexible programming model and
> multiple
> > languages. MXNet allows you to mix symbolic and imperative programming
> > flavors to maximize both efficiency and productivity. MXNet is built on a
> > dynamic dependency scheduler that automatically parallelizes both
> symbolic
> > and imperative operations on the fly. A graph optimization layer on top
> of
> > that makes symbolic execution fast and memory efficient. The MXNet
> library
> > is portable and lightweight, and it scales to multiple GPUs and multiple
> > machines.
> >
> > == Background ==
> >
> > Deep learning is a subset of Machine learning and refers to a class of
> > algorithms that use a hierarchical approach with non-linearities to
> > discover and learn representations within data. Deep Learning has
> recently
> > become very popular due to its applicability and advancement of domains
> > such as Computer Vision, Speech Recognition, Natural Language
> Understanding
> > and Recommender Systems. With pervasive and cost effective cloud
> computing,
> > large labeled datasets and continued algorithmic innovation, Deep
> Learning
> > has become the one of the most popular classes of algorithms for machine
> > learning practitioners in recent years.
> >
> > == Rational ==
> >
> > The adoption of deep learning is quickly expanding from initial deep
> domain
> > experts rooted in academia to data scientists and developers working to
> > deploy intelligent services and products. Deep learning however has many
> > challenges.  These include model training time (which can take days to
> > weeks), programmability (not everyone writes Python or C++ and like
> > symbolic programming) and balancing production readiness (support for
> > things like failover) with development flexibility (ability to program
> > different ways, support for new operators and model types) and speed of
> > execution (fast and scalable model training).  Other frameworks excel on
> > some but not all of these aspects.
> >
> >
> > == Initial Goals ==
> >
> > MXNet is a fairly established project on GitHub with its first code
> > contribution in April 2015 and roughly 200 contributors. It is used by
> > several large companies and some of the top research institutions on the
> > planet. Initial goals would be the following:
> >
> >  1. Move the existing codebase(s) to Apache
> >  1. Integrate with the Apache development process/sign CLAs
> >  1. Ensure all dependencies are compliant with Apache License version 2.0
> >  1. Incremental development and releases per Apache guidelines
> >  1. Establish engineering discipline and a predictable release cadence of
> > high quality releases
> >  1. Expand the community beyond the current base of expert level users
> >  1. Improve usability and the overall developer/user experience
> >  1. Add additional functionality to address newer problem types and
> > algorithms
> >
> >
> > == Current Status ==
> >
> > === Meritocracy ===
> >
> > The MXNet project already operates on meritocratic principles. Today,
> MXNet
> > has developers worldwide and has accepted multiple major patches from a
> > diverse set of contributors within both industry and academia. We would
> > like to follow ASF meritocratic principles to encourage more developers
> to
> > contribute in this project. We know that only active and committed
> > developers from a diverse set of backgrounds can make MXNet a successful
> > project.  We are also improving the documentation and code to help new
> > developers get started quickly.
> >
> > === Community ===
> >
> > Acceptance into the Apache foundation would bolster the growing user and
> > developer community around MXNet. That community includes around 200
> > contributors from academia and industry. The core developers of our
> project
> > are listed in our contributors below and are also represented by logos on
> > the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> > Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of
> Alberta,
> > University of Washington and Wolfram.
> >
> > === Core Developers ===
> >
> > (with GitHub logins)
> >
> >  * Tianqi Chen (@tqchen)
> >  * Mu Li (@mli)
> >  * Junyuan Xie (@piiswrong)
> >  * Bing Xu (@antinucleon)
> >  * Chiyuan Zhang (@pluskid)
> >  * Minjie Wang (@jermainewang)
> >  * Naiyan Wang (@winstywang)
> >  * Yizhi Liu (@javelinjs)
> >  * Tong He (@hetong007)
> >  * Qiang Kou (@thirdwing)
> >  * Xingjian Shi (@sxjscience)
> >
> > === Alignment ===
> >
> > ASF is already the home of many distributed platforms, e.g., Hadoop,
> Spark
> > and Mahout, each of which targets a different application domain. MXNet,
> > being a distributed platform for large-scale deep learning, focuses on
> > another important domain for which there still lacks a scalable,
> > programmable, flexible and super fast open-source platform. The recent
> > success of deep learning models especially for vision and speech
> > recognition tasks has generated interests in both applying existing deep
> > learning models and in developing new ones. Thus, an open-source platform
> > for deep learning backed by some of the top industry and academic players
> > will be able to attract a large community of users and developers. MXNet
> is
> > a complex system needing many iterations of design, implementation and
> > testing. Apache's collaboration framework which encourages active
> > contribution from developers will inevitably help improve the quality of
> > the system, as shown in the success of Hadoop, Spark, etc. Equally
> > important is the community of users which helps identify real-life
> > applications of deep learning, and helps to evaluate the system's
> > performance and ease-of-use. We hope to leverage ASF for coordinating and
> > promoting both communities, and in return benefit the communities with
> > another useful tool.
> >
> > == Known Risks ==
> >
> > === Orphaned products ===
> >
> > Given the current level of investment in MXNet and the stakeholders using
> > it - the risk of the project being abandoned is minimal. Amazon, for
> > example, is in active development to use MXNet in many of its services
> and
> > many large corporations use it in their production applications.
> >
> > === Inexperience with Open Source ===
> >
> > MXNet has existed as a healthy open source project for more than a year.
> > During that time, the project has attracted 200+ contributors.
> >
> > === Homogenous Developers ===
> >
> > The initial list of committers and contributors includes developers from
> > several institutions and industry participants (see above).
> >
> > === Reliance on Salaried Developers ===
> >
> > Like most open source projects, MXNet receives a substantial support from
> > salaried developers. A large fraction of MXNet development is supported
> by
> > graduate students at various universities in the course of research
> degrees
> > - this is more a “volunteer” relationship, since in most cases students
> > contribute vastly more than is necessary to immediately support research.
> > In addition, those working from within corporations are devoting
> > significant time and effort in the project - and these come from several
> > organizations.
> >
> > === A Excessive Fascination with the Apache Brand ===
> >
> > We choose Apache not for publicity. We have two purposes. First, we hope
> > that Apache's known best-practices for managing a mature open source
> > project can help guide us.  For example, we are feeling the growing pains
> > of a successful open source project as we attempt a major refactor of the
> > internals while customers are using the system in production. We seek
> > guidance in communicating breaking API changes and version revisions.
> > Also, as our involvement from major corporations increases, we want to
> > assure our users that MXNet will stay open and not favor any particular
> > platform or environment. These are some examples of the know-how and
> > discipline we're hoping Apache can bring to our project.
> >
> > Second, we want to leverage Apache's reputation to recruit more
> developers
> > to create a diverse community.
> >
> > === Relationship with Other Apache Products ===
> >
> > Apache Mahout and Apache Spark's MLlib are general machine learning
> > systems. Deep learning algorithms can thus be implemented on these two
> > platforms as well. However, in practice, the overlap will be minimal.
> Deep
> > learning is so computationally intensive that it often requires
> specialized
> > GPU hardware to accomplish tasks of meaningful size.  Making efficient
> use
> > of GPU hardware is complex because the hardware is so fast that the
> > supporting systems around it must be carefully optimized to keep the GPU
> > cores busy.  Extending this capability to distributed multi-GPU and
> > multi-host environments requires great care.  This is a critical
> > differentiator between MXNet and existing Apache machine learning
> systems.
> >
> > Mahout and Spark ML-LIB follow models where their nodes run
> synchronously.
> > This is the fundamental difference to MXNet who follows the parameter
> > server framework. MXNet can run synchronously or asynchronously. In
> > addition, MXNet has optimizations for training a wide range of deep
> > learning models using a variety of approaches (e.g., model parallelism
> and
> > data parallelism) which makes MXNet much more efficient (near-linear
> > speedup on state of the art models). MXNet also supports both imperative
> > and symbolic approaches providing ease of programming for deep learning
> > algorithms.
> >
> > Other Apache projects that are potentially complimentary:
> >
> > Apache Arrow - read data in Apache Arrow‘s internal format from MXNet,
> that
> > would allow users to run ETL/preprocessing in Spark, save the results in
> > Arrow’s format and then run DL algorithms on it.
> >
> > Apache Singa - MXNet and Singa are both deep learning projects, and can
> > benefit from a larger deep learning community at Apache.
> >
> > == Documentation ==
> >
> > Documentation has recently migrated to http://mxnet.io.  We continue to
> > refine and improve the documentation.
> >
> > == Initial Source ==
> >
> > We currently use Github to maintain our source code,
> > https://github.com/MXNet
> >
> > == Source and Intellectual Property Submission Plan ==
> >
> > MXNet Code is available under Apache License, Version 2.0. We will work
> > with the committers to get CLAs signed and review previous contributions.
> >
> > == External Dependencies ==
> >
> >  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> > (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> > lib-zeromq), TBB
> >  * required for GPU usage: cudnn, cuda
> >  * required for python usage: Python 2/3
> >  * required for R module: R, Rcpp (GPLv2 licensing)
> >  * optional for image preparation and preprocessing: opencv
> >  * optional dependencies for additional features: torch7, numba, cython
> (in
> > NNVM branch)
> >
> > Rcpt and lib-zeromq are expected to be licensing discussions.
> >
> > == Cryptography ==
> >
> > Not Applicable
> >
> > == Required Resources ==
> >
> > === Mailing Lists ===
> >
> > There is currently no mailing list.
> >
> > === Issue Tracking ===
> >
> > Currently uses GitHub to track issues. Would like to continue to do so.
> >
> > == Committers and Affiliations ==
> >
> >  * Tianqi Chen (UW)
> >  * Mu Li (AWS)
> >  * Junyuan Xie (AWS)
> >  * Bing Xu (Apple)
> >  * Chiyuan Zhang (MIT)
> >  * Minjie Wang (UYU)
> >  * Naiyan Wang (Tusimple)
> >  * Yizhi Liu (Mediav)
> >  * Tong He (Simon Fraser University)
> >  * Qiang Kou (Indiana U)
> >  * Xingjian Shi (HKUST)
> >
> > == Sponsors ==
> >
> > === Champion ===
> >
> > Henri Yandell (bayard at apache.org)
> >
> > === Nominated Mentors ===
> >
> > Sebastian Schelter (ssc@apache.org)
> >
> >
> > === Sponsoring Entity ===
> >
> > We are requesting the Incubator to sponsor this project.
> >
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henry Saputra <he...@gmail.com>]

This is great news and I am looking forward to it =)

According to proposal, the community want to stick with Github issues for
tracking issues and bugs?
I suppose this needs a nod by Greg Stein as rep from Apache Infra to
confirm that this is ok for incubation and how would it impact during
graduation.

- Henry

On Thu, Jan 5, 2017 at 9:12 PM, Henri Yandell <ba...@apache.org> wrote:

> Hello Incubator,
>
> I'd like to propose a new incubator Apache MXNet podling.
>
> The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> committers,
> 200 contributors) is very interested in joining Apache. MXNet is an
> open-source deep learning framework that allows you to define, train, and
> deploy deep neural networks on a wide array of devices, from cloud
> infrastructure to mobile devices.
>
> The wiki proposal page is located here:
>
>   https://wiki.apache.org/incubator/MXNetProposal
>
> I've included the text below in case anyone wants to focus on parts of it
> in a reply.
>
> Looking forward to your thoughts, and for lots of interested Apache members
> to volunteer to mentor the project in addition to Sebastian and myself.
>
> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!
>
> Thanks,
>
> Hen
> On behalf of the MXNet project
>
> ---------
>
> = MXNet: Apache Incubator Proposal =
>
> == Abstract ==
>
> MXNet is a Flexible and Efficient Library for Deep Learning
>
> == Proposal ==
>
> MXNet is an open-source deep learning framework that allows you to define,
> train, and deploy deep neural networks on a wide array of devices, from
> cloud infrastructure to mobile devices. It is highly scalable, allowing for
> fast model training, and supports a flexible programming model and multiple
> languages. MXNet allows you to mix symbolic and imperative programming
> flavors to maximize both efficiency and productivity. MXNet is built on a
> dynamic dependency scheduler that automatically parallelizes both symbolic
> and imperative operations on the fly. A graph optimization layer on top of
> that makes symbolic execution fast and memory efficient. The MXNet library
> is portable and lightweight, and it scales to multiple GPUs and multiple
> machines.
>
> == Background ==
>
> Deep learning is a subset of Machine learning and refers to a class of
> algorithms that use a hierarchical approach with non-linearities to
> discover and learn representations within data. Deep Learning has recently
> become very popular due to its applicability and advancement of domains
> such as Computer Vision, Speech Recognition, Natural Language Understanding
> and Recommender Systems. With pervasive and cost effective cloud computing,
> large labeled datasets and continued algorithmic innovation, Deep Learning
> has become the one of the most popular classes of algorithms for machine
> learning practitioners in recent years.
>
> == Rational ==
>
> The adoption of deep learning is quickly expanding from initial deep domain
> experts rooted in academia to data scientists and developers working to
> deploy intelligent services and products. Deep learning however has many
> challenges.  These include model training time (which can take days to
> weeks), programmability (not everyone writes Python or C++ and like
> symbolic programming) and balancing production readiness (support for
> things like failover) with development flexibility (ability to program
> different ways, support for new operators and model types) and speed of
> execution (fast and scalable model training).  Other frameworks excel on
> some but not all of these aspects.
>
>
> == Initial Goals ==
>
> MXNet is a fairly established project on GitHub with its first code
> contribution in April 2015 and roughly 200 contributors. It is used by
> several large companies and some of the top research institutions on the
> planet. Initial goals would be the following:
>
>  1. Move the existing codebase(s) to Apache
>  1. Integrate with the Apache development process/sign CLAs
>  1. Ensure all dependencies are compliant with Apache License version 2.0
>  1. Incremental development and releases per Apache guidelines
>  1. Establish engineering discipline and a predictable release cadence of
> high quality releases
>  1. Expand the community beyond the current base of expert level users
>  1. Improve usability and the overall developer/user experience
>  1. Add additional functionality to address newer problem types and
> algorithms
>
>
> == Current Status ==
>
> === Meritocracy ===
>
> The MXNet project already operates on meritocratic principles. Today, MXNet
> has developers worldwide and has accepted multiple major patches from a
> diverse set of contributors within both industry and academia. We would
> like to follow ASF meritocratic principles to encourage more developers to
> contribute in this project. We know that only active and committed
> developers from a diverse set of backgrounds can make MXNet a successful
> project.  We are also improving the documentation and code to help new
> developers get started quickly.
>
> === Community ===
>
> Acceptance into the Apache foundation would bolster the growing user and
> developer community around MXNet. That community includes around 200
> contributors from academia and industry. The core developers of our project
> are listed in our contributors below and are also represented by logos on
> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
> University of Washington and Wolfram.
>
> === Core Developers ===
>
> (with GitHub logins)
>
>  * Tianqi Chen (@tqchen)
>  * Mu Li (@mli)
>  * Junyuan Xie (@piiswrong)
>  * Bing Xu (@antinucleon)
>  * Chiyuan Zhang (@pluskid)
>  * Minjie Wang (@jermainewang)
>  * Naiyan Wang (@winstywang)
>  * Yizhi Liu (@javelinjs)
>  * Tong He (@hetong007)
>  * Qiang Kou (@thirdwing)
>  * Xingjian Shi (@sxjscience)
>
> === Alignment ===
>
> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
> and Mahout, each of which targets a different application domain. MXNet,
> being a distributed platform for large-scale deep learning, focuses on
> another important domain for which there still lacks a scalable,
> programmable, flexible and super fast open-source platform. The recent
> success of deep learning models especially for vision and speech
> recognition tasks has generated interests in both applying existing deep
> learning models and in developing new ones. Thus, an open-source platform
> for deep learning backed by some of the top industry and academic players
> will be able to attract a large community of users and developers. MXNet is
> a complex system needing many iterations of design, implementation and
> testing. Apache's collaboration framework which encourages active
> contribution from developers will inevitably help improve the quality of
> the system, as shown in the success of Hadoop, Spark, etc. Equally
> important is the community of users which helps identify real-life
> applications of deep learning, and helps to evaluate the system's
> performance and ease-of-use. We hope to leverage ASF for coordinating and
> promoting both communities, and in return benefit the communities with
> another useful tool.
>
> == Known Risks ==
>
> === Orphaned products ===
>
> Given the current level of investment in MXNet and the stakeholders using
> it - the risk of the project being abandoned is minimal. Amazon, for
> example, is in active development to use MXNet in many of its services and
> many large corporations use it in their production applications.
>
> === Inexperience with Open Source ===
>
> MXNet has existed as a healthy open source project for more than a year.
> During that time, the project has attracted 200+ contributors.
>
> === Homogenous Developers ===
>
> The initial list of committers and contributors includes developers from
> several institutions and industry participants (see above).
>
> === Reliance on Salaried Developers ===
>
> Like most open source projects, MXNet receives a substantial support from
> salaried developers. A large fraction of MXNet development is supported by
> graduate students at various universities in the course of research degrees
> - this is more a “volunteer” relationship, since in most cases students
> contribute vastly more than is necessary to immediately support research.
> In addition, those working from within corporations are devoting
> significant time and effort in the project - and these come from several
> organizations.
>
> === A Excessive Fascination with the Apache Brand ===
>
> We choose Apache not for publicity. We have two purposes. First, we hope
> that Apache's known best-practices for managing a mature open source
> project can help guide us.  For example, we are feeling the growing pains
> of a successful open source project as we attempt a major refactor of the
> internals while customers are using the system in production. We seek
> guidance in communicating breaking API changes and version revisions.
> Also, as our involvement from major corporations increases, we want to
> assure our users that MXNet will stay open and not favor any particular
> platform or environment. These are some examples of the know-how and
> discipline we're hoping Apache can bring to our project.
>
> Second, we want to leverage Apache's reputation to recruit more developers
> to create a diverse community.
>
> === Relationship with Other Apache Products ===
>
> Apache Mahout and Apache Spark's MLlib are general machine learning
> systems. Deep learning algorithms can thus be implemented on these two
> platforms as well. However, in practice, the overlap will be minimal.  Deep
> learning is so computationally intensive that it often requires specialized
> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
> of GPU hardware is complex because the hardware is so fast that the
> supporting systems around it must be carefully optimized to keep the GPU
> cores busy.  Extending this capability to distributed multi-GPU and
> multi-host environments requires great care.  This is a critical
> differentiator between MXNet and existing Apache machine learning systems.
>
> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
> This is the fundamental difference to MXNet who follows the parameter
> server framework. MXNet can run synchronously or asynchronously. In
> addition, MXNet has optimizations for training a wide range of deep
> learning models using a variety of approaches (e.g., model parallelism and
> data parallelism) which makes MXNet much more efficient (near-linear
> speedup on state of the art models). MXNet also supports both imperative
> and symbolic approaches providing ease of programming for deep learning
> algorithms.
>
> Other Apache projects that are potentially complimentary:
>
> Apache Arrow - read data in Apache Arrow‘s internal format from MXNet, that
> would allow users to run ETL/preprocessing in Spark, save the results in
> Arrow’s format and then run DL algorithms on it.
>
> Apache Singa - MXNet and Singa are both deep learning projects, and can
> benefit from a larger deep learning community at Apache.
>
> == Documentation ==
>
> Documentation has recently migrated to http://mxnet.io.  We continue to
> refine and improve the documentation.
>
> == Initial Source ==
>
> We currently use Github to maintain our source code,
> https://github.com/MXNet
>
> == Source and Intellectual Property Submission Plan ==
>
> MXNet Code is available under Apache License, Version 2.0. We will work
> with the committers to get CLAs signed and review previous contributions.
>
> == External Dependencies ==
>
>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> lib-zeromq), TBB
>  * required for GPU usage: cudnn, cuda
>  * required for python usage: Python 2/3
>  * required for R module: R, Rcpp (GPLv2 licensing)
>  * optional for image preparation and preprocessing: opencv
>  * optional dependencies for additional features: torch7, numba, cython (in
> NNVM branch)
>
> Rcpt and lib-zeromq are expected to be licensing discussions.
>
> == Cryptography ==
>
> Not Applicable
>
> == Required Resources ==
>
> === Mailing Lists ===
>
> There is currently no mailing list.
>
> === Issue Tracking ===
>
> Currently uses GitHub to track issues. Would like to continue to do so.
>
> == Committers and Affiliations ==
>
>  * Tianqi Chen (UW)
>  * Mu Li (AWS)
>  * Junyuan Xie (AWS)
>  * Bing Xu (Apple)
>  * Chiyuan Zhang (MIT)
>  * Minjie Wang (UYU)
>  * Naiyan Wang (Tusimple)
>  * Yizhi Liu (Mediav)
>  * Tong He (Simon Fraser University)
>  * Qiang Kou (Indiana U)
>  * Xingjian Shi (HKUST)
>
> == Sponsors ==
>
> === Champion ===
>
> Henri Yandell (bayard at apache.org)
>
> === Nominated Mentors ===
>
> Sebastian Schelter (ssc@apache.org)
>
>
> === Sponsoring Entity ===
>
> We are requesting the Incubator to sponsor this project.
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[YiZhi Liu <ja...@gmail.com>]

Confirmed, and please update my affiliation to 'Qihoo 360'. Thanks.

2017-01-14 16:08 GMT+08:00 梁德澎 <li...@gmail.com>:
> Hi,
>
> I’ve been working on the MXNet-ScalaPkg for a while with Yizhi Liu
> (@javelinjs).
> Please sign me up as a committer of MxNet.
>
> GitHub ID: Ldpe2G
> Email: liangdepeng@gmail.com
> Affiliations: Sun Yat-sen University
>
> 2017-01-14 13:49 GMT+08:00 Henri Yandell <ba...@apache.org>:
>
>> Thanks for all the feedback and interested parties :)
>>
>> My aim is to propose a vote on Monday, unless someone raises an issue
>> before then.
>>
>> Hen
>>
>> On Thu, Jan 5, 2017 at 9:12 PM, Henri Yandell <ba...@apache.org> wrote:
>>
>> > Hello Incubator,
>> >
>> > I'd like to propose a new incubator Apache MXNet podling.
>> >
>> > The existing MXNet project (http://mxnet.io - 1.5 years old, 15
>> > committers, 200 contributors) is very interested in joining Apache. MXNet
>> > is an open-source deep learning framework that allows you to define,
>> train,
>> > and deploy deep neural networks on a wide array of devices, from cloud
>> > infrastructure to mobile devices.
>> >
>> > The wiki proposal page is located here:
>> >
>> >   https://wiki.apache.org/incubator/MXNetProposal
>> >
>> > I've included the text below in case anyone wants to focus on parts of it
>> > in a reply.
>> >
>> > Looking forward to your thoughts, and for lots of interested Apache
>> > members to volunteer to mentor the project in addition to Sebastian and
>> > myself.
>> >
>> > Currently the list of committers is based on the current active coders,
>> so
>> > we're also very interested in hearing from anyone else who is interested
>> in
>> > working on the project, be they current or future contributor!
>> >
>> > Thanks,
>> >
>> > Hen
>> > On behalf of the MXNet project
>> >
>> > ---------
>> >
>> > = MXNet: Apache Incubator Proposal =
>> >
>> > == Abstract ==
>> >
>> > MXNet is a Flexible and Efficient Library for Deep Learning
>> >
>> > == Proposal ==
>> >
>> > MXNet is an open-source deep learning framework that allows you to
>> define,
>> > train, and deploy deep neural networks on a wide array of devices, from
>> > cloud infrastructure to mobile devices. It is highly scalable, allowing
>> for
>> > fast model training, and supports a flexible programming model and
>> multiple
>> > languages. MXNet allows you to mix symbolic and imperative programming
>> > flavors to maximize both efficiency and productivity. MXNet is built on a
>> > dynamic dependency scheduler that automatically parallelizes both
>> symbolic
>> > and imperative operations on the fly. A graph optimization layer on top
>> of
>> > that makes symbolic execution fast and memory efficient. The MXNet
>> library
>> > is portable and lightweight, and it scales to multiple GPUs and multiple
>> > machines.
>> >
>> > == Background ==
>> >
>> > Deep learning is a subset of Machine learning and refers to a class of
>> > algorithms that use a hierarchical approach with non-linearities to
>> > discover and learn representations within data. Deep Learning has
>> recently
>> > become very popular due to its applicability and advancement of domains
>> > such as Computer Vision, Speech Recognition, Natural Language
>> Understanding
>> > and Recommender Systems. With pervasive and cost effective cloud
>> computing,
>> > large labeled datasets and continued algorithmic innovation, Deep
>> Learning
>> > has become the one of the most popular classes of algorithms for machine
>> > learning practitioners in recent years.
>> >
>> > == Rational ==
>> >
>> > The adoption of deep learning is quickly expanding from initial deep
>> > domain experts rooted in academia to data scientists and developers
>> working
>> > to deploy intelligent services and products. Deep learning however has
>> many
>> > challenges.  These include model training time (which can take days to
>> > weeks), programmability (not everyone writes Python or C++ and like
>> > symbolic programming) and balancing production readiness (support for
>> > things like failover) with development flexibility (ability to program
>> > different ways, support for new operators and model types) and speed of
>> > execution (fast and scalable model training).  Other frameworks excel on
>> > some but not all of these aspects.
>> >
>> >
>> > == Initial Goals ==
>> >
>> > MXNet is a fairly established project on GitHub with its first code
>> > contribution in April 2015 and roughly 200 contributors. It is used by
>> > several large companies and some of the top research institutions on the
>> > planet. Initial goals would be the following:
>> >
>> >  1. Move the existing codebase(s) to Apache
>> >  1. Integrate with the Apache development process/sign CLAs
>> >  1. Ensure all dependencies are compliant with Apache License version 2.0
>> >  1. Incremental development and releases per Apache guidelines
>> >  1. Establish engineering discipline and a predictable release cadence of
>> > high quality releases
>> >  1. Expand the community beyond the current base of expert level users
>> >  1. Improve usability and the overall developer/user experience
>> >  1. Add additional functionality to address newer problem types and
>> > algorithms
>> >
>> >
>> > == Current Status ==
>> >
>> > === Meritocracy ===
>> >
>> > The MXNet project already operates on meritocratic principles. Today,
>> > MXNet has developers worldwide and has accepted multiple major patches
>> from
>> > a diverse set of contributors within both industry and academia. We would
>> > like to follow ASF meritocratic principles to encourage more developers
>> to
>> > contribute in this project. We know that only active and committed
>> > developers from a diverse set of backgrounds can make MXNet a successful
>> > project.  We are also improving the documentation and code to help new
>> > developers get started quickly.
>> >
>> > === Community ===
>> >
>> > Acceptance into the Apache foundation would bolster the growing user and
>> > developer community around MXNet. That community includes around 200
>> > contributors from academia and industry. The core developers of our
>> project
>> > are listed in our contributors below and are also represented by logos on
>> > the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
>> > Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of
>> Alberta,
>> > University of Washington and Wolfram.
>> >
>> > === Core Developers ===
>> >
>> > (with GitHub logins)
>> >
>> >  * Tianqi Chen (@tqchen)
>> >  * Mu Li (@mli)
>> >  * Junyuan Xie (@piiswrong)
>> >  * Bing Xu (@antinucleon)
>> >  * Chiyuan Zhang (@pluskid)
>> >  * Minjie Wang (@jermainewang)
>> >  * Naiyan Wang (@winstywang)
>> >  * Yizhi Liu (@javelinjs)
>> >  * Tong He (@hetong007)
>> >  * Qiang Kou (@thirdwing)
>> >  * Xingjian Shi (@sxjscience)
>> >
>> > === Alignment ===
>> >
>> > ASF is already the home of many distributed platforms, e.g., Hadoop,
>> Spark
>> > and Mahout, each of which targets a different application domain. MXNet,
>> > being a distributed platform for large-scale deep learning, focuses on
>> > another important domain for which there still lacks a scalable,
>> > programmable, flexible and super fast open-source platform. The recent
>> > success of deep learning models especially for vision and speech
>> > recognition tasks has generated interests in both applying existing deep
>> > learning models and in developing new ones. Thus, an open-source platform
>> > for deep learning backed by some of the top industry and academic players
>> > will be able to attract a large community of users and developers. MXNet
>> is
>> > a complex system needing many iterations of design, implementation and
>> > testing. Apache's collaboration framework which encourages active
>> > contribution from developers will inevitably help improve the quality of
>> > the system, as shown in the success of Hadoop, Spark, etc. Equally
>> > important is the community of users which helps identify real-life
>> > applications of deep learning, and helps to evaluate the system's
>> > performance and ease-of-use. We hope to leverage ASF for coordinating and
>> > promoting both communities, and in return benefit the communities with
>> > another useful tool.
>> >
>> > == Known Risks ==
>> >
>> > === Orphaned products ===
>> >
>> > Given the current level of investment in MXNet and the stakeholders using
>> > it - the risk of the project being abandoned is minimal. Amazon, for
>> > example, is in active development to use MXNet in many of its services
>> and
>> > many large corporations use it in their production applications.
>> >
>> > === Inexperience with Open Source ===
>> >
>> > MXNet has existed as a healthy open source project for more than a year.
>> > During that time, the project has attracted 200+ contributors.
>> >
>> > === Homogenous Developers ===
>> >
>> > The initial list of committers and contributors includes developers from
>> > several institutions and industry participants (see above).
>> >
>> > === Reliance on Salaried Developers ===
>> >
>> > Like most open source projects, MXNet receives a substantial support from
>> > salaried developers. A large fraction of MXNet development is supported
>> by
>> > graduate students at various universities in the course of research
>> degrees
>> > - this is more a “volunteer” relationship, since in most cases students
>> > contribute vastly more than is necessary to immediately support research.
>> > In addition, those working from within corporations are devoting
>> > significant time and effort in the project - and these come from several
>> > organizations.
>> >
>> > === A Excessive Fascination with the Apache Brand ===
>> >
>> > We choose Apache not for publicity. We have two purposes. First, we hope
>> > that Apache's known best-practices for managing a mature open source
>> > project can help guide us.  For example, we are feeling the growing pains
>> > of a successful open source project as we attempt a major refactor of the
>> > internals while customers are using the system in production. We seek
>> > guidance in communicating breaking API changes and version revisions.
>> > Also, as our involvement from major corporations increases, we want to
>> > assure our users that MXNet will stay open and not favor any particular
>> > platform or environment. These are some examples of the know-how and
>> > discipline we're hoping Apache can bring to our project.
>> >
>> > Second, we want to leverage Apache's reputation to recruit more
>> developers
>> > to create a diverse community.
>> >
>> > === Relationship with Other Apache Products ===
>> >
>> > Apache Mahout and Apache Spark's MLlib are general machine learning
>> > systems. Deep learning algorithms can thus be implemented on these two
>> > platforms as well. However, in practice, the overlap will be minimal.
>> Deep
>> > learning is so computationally intensive that it often requires
>> specialized
>> > GPU hardware to accomplish tasks of meaningful size.  Making efficient
>> use
>> > of GPU hardware is complex because the hardware is so fast that the
>> > supporting systems around it must be carefully optimized to keep the GPU
>> > cores busy.  Extending this capability to distributed multi-GPU and
>> > multi-host environments requires great care.  This is a critical
>> > differentiator between MXNet and existing Apache machine learning
>> systems.
>> >
>> > Mahout and Spark ML-LIB follow models where their nodes run
>> synchronously.
>> > This is the fundamental difference to MXNet who follows the parameter
>> > server framework. MXNet can run synchronously or asynchronously. In
>> > addition, MXNet has optimizations for training a wide range of deep
>> > learning models using a variety of approaches (e.g., model parallelism
>> and
>> > data parallelism) which makes MXNet much more efficient (near-linear
>> > speedup on state of the art models). MXNet also supports both imperative
>> > and symbolic approaches providing ease of programming for deep learning
>> > algorithms.
>> >
>> > Other Apache projects that are potentially complimentary:
>> >
>> > Apache Arrow - read data in Apache Arrow‘s internal format from MXNet,
>> > that would allow users to run ETL/preprocessing in Spark, save the
>> results
>> > in Arrow’s format and then run DL algorithms on it.
>> >
>> > Apache Singa - MXNet and Singa are both deep learning projects, and can
>> > benefit from a larger deep learning community at Apache.
>> >
>> > == Documentation ==
>> >
>> > Documentation has recently migrated to http://mxnet.io.  We continue to
>> > refine and improve the documentation.
>> >
>> > == Initial Source ==
>> >
>> > We currently use Github to maintain our source code,
>> > https://github.com/MXNet
>> >
>> > == Source and Intellectual Property Submission Plan ==
>> >
>> > MXNet Code is available under Apache License, Version 2.0. We will work
>> > with the committers to get CLAs signed and review previous contributions.
>> >
>> > == External Dependencies ==
>> >
>> >  * required by the core code base: GCC or CLOM, Clang, any BLAS library
>> > (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
>> > lib-zeromq), TBB
>> >  * required for GPU usage: cudnn, cuda
>> >  * required for python usage: Python 2/3
>> >  * required for R module: R, Rcpp (GPLv2 licensing)
>> >  * optional for image preparation and preprocessing: opencv
>> >  * optional dependencies for additional features: torch7, numba, cython
>> > (in NNVM branch)
>> >
>> > Rcpt and lib-zeromq are expected to be licensing discussions.
>> >
>> > == Cryptography ==
>> >
>> > Not Applicable
>> >
>> > == Required Resources ==
>> >
>> > === Mailing Lists ===
>> >
>> > There is currently no mailing list.
>> >
>> > === Issue Tracking ===
>> >
>> > Currently uses GitHub to track issues. Would like to continue to do so.
>> >
>> > == Committers and Affiliations ==
>> >
>> >  * Tianqi Chen (UW)
>> >  * Mu Li (AWS)
>> >  * Junyuan Xie (AWS)
>> >  * Bing Xu (Apple)
>> >  * Chiyuan Zhang (MIT)
>> >  * Minjie Wang (UYU)
>> >  * Naiyan Wang (Tusimple)
>> >  * Yizhi Liu (Mediav)
>> >  * Tong He (Simon Fraser University)
>> >  * Qiang Kou (Indiana U)
>> >  * Xingjian Shi (HKUST)
>> >
>> > == Sponsors ==
>> >
>> > === Champion ===
>> >
>> > Henri Yandell (bayard at apache.org)
>> >
>> > === Nominated Mentors ===
>> >
>> > Sebastian Schelter (ssc@apache.org)
>> >
>> >
>> > === Sponsoring Entity ===
>> >
>> > We are requesting the Incubator to sponsor this project.
>> >
>> >
>>



-- 
Yizhi Liu
DMLC member
Technical Manager
Qihoo 360 Inc, Shanghai, China

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Added. Apologies if Liang DePeng is the incorrect anglicization of your
name.

Hen

On Sat, Jan 14, 2017 at 12:08 AM, 梁德澎 <li...@gmail.com> wrote:

> Hi,
>
> I’ve been working on the MXNet-ScalaPkg for a while with Yizhi Liu
> (@javelinjs).
> Please sign me up as a committer of MxNet.
>
> GitHub ID: Ldpe2G
> Email: liangdepeng@gmail.com
> Affiliations: Sun Yat-sen University
>
> 2017-01-14 13:49 GMT+08:00 Henri Yandell <ba...@apache.org>:
>
> > Thanks for all the feedback and interested parties :)
> >
> > My aim is to propose a vote on Monday, unless someone raises an issue
> > before then.
> >
> > Hen
> >
> > On Thu, Jan 5, 2017 at 9:12 PM, Henri Yandell <ba...@apache.org> wrote:
> >
> > > Hello Incubator,
> > >
> > > I'd like to propose a new incubator Apache MXNet podling.
> > >
> > > The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> > > committers, 200 contributors) is very interested in joining Apache.
> MXNet
> > > is an open-source deep learning framework that allows you to define,
> > train,
> > > and deploy deep neural networks on a wide array of devices, from cloud
> > > infrastructure to mobile devices.
> > >
> > > The wiki proposal page is located here:
> > >
> > >   https://wiki.apache.org/incubator/MXNetProposal
> > >
> > > I've included the text below in case anyone wants to focus on parts of
> it
> > > in a reply.
> > >
> > > Looking forward to your thoughts, and for lots of interested Apache
> > > members to volunteer to mentor the project in addition to Sebastian and
> > > myself.
> > >
> > > Currently the list of committers is based on the current active coders,
> > so
> > > we're also very interested in hearing from anyone else who is
> interested
> > in
> > > working on the project, be they current or future contributor!
> > >
> > > Thanks,
> > >
> > > Hen
> > > On behalf of the MXNet project
> > >
> > > ---------
> > >
> > > = MXNet: Apache Incubator Proposal =
> > >
> > > == Abstract ==
> > >
> > > MXNet is a Flexible and Efficient Library for Deep Learning
> > >
> > > == Proposal ==
> > >
> > > MXNet is an open-source deep learning framework that allows you to
> > define,
> > > train, and deploy deep neural networks on a wide array of devices, from
> > > cloud infrastructure to mobile devices. It is highly scalable, allowing
> > for
> > > fast model training, and supports a flexible programming model and
> > multiple
> > > languages. MXNet allows you to mix symbolic and imperative programming
> > > flavors to maximize both efficiency and productivity. MXNet is built
> on a
> > > dynamic dependency scheduler that automatically parallelizes both
> > symbolic
> > > and imperative operations on the fly. A graph optimization layer on top
> > of
> > > that makes symbolic execution fast and memory efficient. The MXNet
> > library
> > > is portable and lightweight, and it scales to multiple GPUs and
> multiple
> > > machines.
> > >
> > > == Background ==
> > >
> > > Deep learning is a subset of Machine learning and refers to a class of
> > > algorithms that use a hierarchical approach with non-linearities to
> > > discover and learn representations within data. Deep Learning has
> > recently
> > > become very popular due to its applicability and advancement of domains
> > > such as Computer Vision, Speech Recognition, Natural Language
> > Understanding
> > > and Recommender Systems. With pervasive and cost effective cloud
> > computing,
> > > large labeled datasets and continued algorithmic innovation, Deep
> > Learning
> > > has become the one of the most popular classes of algorithms for
> machine
> > > learning practitioners in recent years.
> > >
> > > == Rational ==
> > >
> > > The adoption of deep learning is quickly expanding from initial deep
> > > domain experts rooted in academia to data scientists and developers
> > working
> > > to deploy intelligent services and products. Deep learning however has
> > many
> > > challenges.  These include model training time (which can take days to
> > > weeks), programmability (not everyone writes Python or C++ and like
> > > symbolic programming) and balancing production readiness (support for
> > > things like failover) with development flexibility (ability to program
> > > different ways, support for new operators and model types) and speed of
> > > execution (fast and scalable model training).  Other frameworks excel
> on
> > > some but not all of these aspects.
> > >
> > >
> > > == Initial Goals ==
> > >
> > > MXNet is a fairly established project on GitHub with its first code
> > > contribution in April 2015 and roughly 200 contributors. It is used by
> > > several large companies and some of the top research institutions on
> the
> > > planet. Initial goals would be the following:
> > >
> > >  1. Move the existing codebase(s) to Apache
> > >  1. Integrate with the Apache development process/sign CLAs
> > >  1. Ensure all dependencies are compliant with Apache License version
> 2.0
> > >  1. Incremental development and releases per Apache guidelines
> > >  1. Establish engineering discipline and a predictable release cadence
> of
> > > high quality releases
> > >  1. Expand the community beyond the current base of expert level users
> > >  1. Improve usability and the overall developer/user experience
> > >  1. Add additional functionality to address newer problem types and
> > > algorithms
> > >
> > >
> > > == Current Status ==
> > >
> > > === Meritocracy ===
> > >
> > > The MXNet project already operates on meritocratic principles. Today,
> > > MXNet has developers worldwide and has accepted multiple major patches
> > from
> > > a diverse set of contributors within both industry and academia. We
> would
> > > like to follow ASF meritocratic principles to encourage more developers
> > to
> > > contribute in this project. We know that only active and committed
> > > developers from a diverse set of backgrounds can make MXNet a
> successful
> > > project.  We are also improving the documentation and code to help new
> > > developers get started quickly.
> > >
> > > === Community ===
> > >
> > > Acceptance into the Apache foundation would bolster the growing user
> and
> > > developer community around MXNet. That community includes around 200
> > > contributors from academia and industry. The core developers of our
> > project
> > > are listed in our contributors below and are also represented by logos
> on
> > > the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> > > Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of
> > Alberta,
> > > University of Washington and Wolfram.
> > >
> > > === Core Developers ===
> > >
> > > (with GitHub logins)
> > >
> > >  * Tianqi Chen (@tqchen)
> > >  * Mu Li (@mli)
> > >  * Junyuan Xie (@piiswrong)
> > >  * Bing Xu (@antinucleon)
> > >  * Chiyuan Zhang (@pluskid)
> > >  * Minjie Wang (@jermainewang)
> > >  * Naiyan Wang (@winstywang)
> > >  * Yizhi Liu (@javelinjs)
> > >  * Tong He (@hetong007)
> > >  * Qiang Kou (@thirdwing)
> > >  * Xingjian Shi (@sxjscience)
> > >
> > > === Alignment ===
> > >
> > > ASF is already the home of many distributed platforms, e.g., Hadoop,
> > Spark
> > > and Mahout, each of which targets a different application domain.
> MXNet,
> > > being a distributed platform for large-scale deep learning, focuses on
> > > another important domain for which there still lacks a scalable,
> > > programmable, flexible and super fast open-source platform. The recent
> > > success of deep learning models especially for vision and speech
> > > recognition tasks has generated interests in both applying existing
> deep
> > > learning models and in developing new ones. Thus, an open-source
> platform
> > > for deep learning backed by some of the top industry and academic
> players
> > > will be able to attract a large community of users and developers.
> MXNet
> > is
> > > a complex system needing many iterations of design, implementation and
> > > testing. Apache's collaboration framework which encourages active
> > > contribution from developers will inevitably help improve the quality
> of
> > > the system, as shown in the success of Hadoop, Spark, etc. Equally
> > > important is the community of users which helps identify real-life
> > > applications of deep learning, and helps to evaluate the system's
> > > performance and ease-of-use. We hope to leverage ASF for coordinating
> and
> > > promoting both communities, and in return benefit the communities with
> > > another useful tool.
> > >
> > > == Known Risks ==
> > >
> > > === Orphaned products ===
> > >
> > > Given the current level of investment in MXNet and the stakeholders
> using
> > > it - the risk of the project being abandoned is minimal. Amazon, for
> > > example, is in active development to use MXNet in many of its services
> > and
> > > many large corporations use it in their production applications.
> > >
> > > === Inexperience with Open Source ===
> > >
> > > MXNet has existed as a healthy open source project for more than a
> year.
> > > During that time, the project has attracted 200+ contributors.
> > >
> > > === Homogenous Developers ===
> > >
> > > The initial list of committers and contributors includes developers
> from
> > > several institutions and industry participants (see above).
> > >
> > > === Reliance on Salaried Developers ===
> > >
> > > Like most open source projects, MXNet receives a substantial support
> from
> > > salaried developers. A large fraction of MXNet development is supported
> > by
> > > graduate students at various universities in the course of research
> > degrees
> > > - this is more a “volunteer” relationship, since in most cases students
> > > contribute vastly more than is necessary to immediately support
> research.
> > > In addition, those working from within corporations are devoting
> > > significant time and effort in the project - and these come from
> several
> > > organizations.
> > >
> > > === A Excessive Fascination with the Apache Brand ===
> > >
> > > We choose Apache not for publicity. We have two purposes. First, we
> hope
> > > that Apache's known best-practices for managing a mature open source
> > > project can help guide us.  For example, we are feeling the growing
> pains
> > > of a successful open source project as we attempt a major refactor of
> the
> > > internals while customers are using the system in production. We seek
> > > guidance in communicating breaking API changes and version revisions.
> > > Also, as our involvement from major corporations increases, we want to
> > > assure our users that MXNet will stay open and not favor any particular
> > > platform or environment. These are some examples of the know-how and
> > > discipline we're hoping Apache can bring to our project.
> > >
> > > Second, we want to leverage Apache's reputation to recruit more
> > developers
> > > to create a diverse community.
> > >
> > > === Relationship with Other Apache Products ===
> > >
> > > Apache Mahout and Apache Spark's MLlib are general machine learning
> > > systems. Deep learning algorithms can thus be implemented on these two
> > > platforms as well. However, in practice, the overlap will be minimal.
> > Deep
> > > learning is so computationally intensive that it often requires
> > specialized
> > > GPU hardware to accomplish tasks of meaningful size.  Making efficient
> > use
> > > of GPU hardware is complex because the hardware is so fast that the
> > > supporting systems around it must be carefully optimized to keep the
> GPU
> > > cores busy.  Extending this capability to distributed multi-GPU and
> > > multi-host environments requires great care.  This is a critical
> > > differentiator between MXNet and existing Apache machine learning
> > systems.
> > >
> > > Mahout and Spark ML-LIB follow models where their nodes run
> > synchronously.
> > > This is the fundamental difference to MXNet who follows the parameter
> > > server framework. MXNet can run synchronously or asynchronously. In
> > > addition, MXNet has optimizations for training a wide range of deep
> > > learning models using a variety of approaches (e.g., model parallelism
> > and
> > > data parallelism) which makes MXNet much more efficient (near-linear
> > > speedup on state of the art models). MXNet also supports both
> imperative
> > > and symbolic approaches providing ease of programming for deep learning
> > > algorithms.
> > >
> > > Other Apache projects that are potentially complimentary:
> > >
> > > Apache Arrow - read data in Apache Arrow‘s internal format from MXNet,
> > > that would allow users to run ETL/preprocessing in Spark, save the
> > results
> > > in Arrow’s format and then run DL algorithms on it.
> > >
> > > Apache Singa - MXNet and Singa are both deep learning projects, and can
> > > benefit from a larger deep learning community at Apache.
> > >
> > > == Documentation ==
> > >
> > > Documentation has recently migrated to http://mxnet.io.  We continue
> to
> > > refine and improve the documentation.
> > >
> > > == Initial Source ==
> > >
> > > We currently use Github to maintain our source code,
> > > https://github.com/MXNet
> > >
> > > == Source and Intellectual Property Submission Plan ==
> > >
> > > MXNet Code is available under Apache License, Version 2.0. We will work
> > > with the committers to get CLAs signed and review previous
> contributions.
> > >
> > > == External Dependencies ==
> > >
> > >  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> > > (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> > > lib-zeromq), TBB
> > >  * required for GPU usage: cudnn, cuda
> > >  * required for python usage: Python 2/3
> > >  * required for R module: R, Rcpp (GPLv2 licensing)
> > >  * optional for image preparation and preprocessing: opencv
> > >  * optional dependencies for additional features: torch7, numba, cython
> > > (in NNVM branch)
> > >
> > > Rcpt and lib-zeromq are expected to be licensing discussions.
> > >
> > > == Cryptography ==
> > >
> > > Not Applicable
> > >
> > > == Required Resources ==
> > >
> > > === Mailing Lists ===
> > >
> > > There is currently no mailing list.
> > >
> > > === Issue Tracking ===
> > >
> > > Currently uses GitHub to track issues. Would like to continue to do so.
> > >
> > > == Committers and Affiliations ==
> > >
> > >  * Tianqi Chen (UW)
> > >  * Mu Li (AWS)
> > >  * Junyuan Xie (AWS)
> > >  * Bing Xu (Apple)
> > >  * Chiyuan Zhang (MIT)
> > >  * Minjie Wang (UYU)
> > >  * Naiyan Wang (Tusimple)
> > >  * Yizhi Liu (Mediav)
> > >  * Tong He (Simon Fraser University)
> > >  * Qiang Kou (Indiana U)
> > >  * Xingjian Shi (HKUST)
> > >
> > > == Sponsors ==
> > >
> > > === Champion ===
> > >
> > > Henri Yandell (bayard at apache.org)
> > >
> > > === Nominated Mentors ===
> > >
> > > Sebastian Schelter (ssc@apache.org)
> > >
> > >
> > > === Sponsoring Entity ===
> > >
> > > We are requesting the Incubator to sponsor this project.
> > >
> > >
> >
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[梁德澎 <li...@gmail.com>]

Hi,

I’ve been working on the MXNet-ScalaPkg for a while with Yizhi Liu
(@javelinjs).
Please sign me up as a committer of MxNet.

GitHub ID: Ldpe2G
Email: liangdepeng@gmail.com
Affiliations: Sun Yat-sen University

2017-01-14 13:49 GMT+08:00 Henri Yandell <ba...@apache.org>:

> Thanks for all the feedback and interested parties :)
>
> My aim is to propose a vote on Monday, unless someone raises an issue
> before then.
>
> Hen
>
> On Thu, Jan 5, 2017 at 9:12 PM, Henri Yandell <ba...@apache.org> wrote:
>
> > Hello Incubator,
> >
> > I'd like to propose a new incubator Apache MXNet podling.
> >
> > The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> > committers, 200 contributors) is very interested in joining Apache. MXNet
> > is an open-source deep learning framework that allows you to define,
> train,
> > and deploy deep neural networks on a wide array of devices, from cloud
> > infrastructure to mobile devices.
> >
> > The wiki proposal page is located here:
> >
> >   https://wiki.apache.org/incubator/MXNetProposal
> >
> > I've included the text below in case anyone wants to focus on parts of it
> > in a reply.
> >
> > Looking forward to your thoughts, and for lots of interested Apache
> > members to volunteer to mentor the project in addition to Sebastian and
> > myself.
> >
> > Currently the list of committers is based on the current active coders,
> so
> > we're also very interested in hearing from anyone else who is interested
> in
> > working on the project, be they current or future contributor!
> >
> > Thanks,
> >
> > Hen
> > On behalf of the MXNet project
> >
> > ---------
> >
> > = MXNet: Apache Incubator Proposal =
> >
> > == Abstract ==
> >
> > MXNet is a Flexible and Efficient Library for Deep Learning
> >
> > == Proposal ==
> >
> > MXNet is an open-source deep learning framework that allows you to
> define,
> > train, and deploy deep neural networks on a wide array of devices, from
> > cloud infrastructure to mobile devices. It is highly scalable, allowing
> for
> > fast model training, and supports a flexible programming model and
> multiple
> > languages. MXNet allows you to mix symbolic and imperative programming
> > flavors to maximize both efficiency and productivity. MXNet is built on a
> > dynamic dependency scheduler that automatically parallelizes both
> symbolic
> > and imperative operations on the fly. A graph optimization layer on top
> of
> > that makes symbolic execution fast and memory efficient. The MXNet
> library
> > is portable and lightweight, and it scales to multiple GPUs and multiple
> > machines.
> >
> > == Background ==
> >
> > Deep learning is a subset of Machine learning and refers to a class of
> > algorithms that use a hierarchical approach with non-linearities to
> > discover and learn representations within data. Deep Learning has
> recently
> > become very popular due to its applicability and advancement of domains
> > such as Computer Vision, Speech Recognition, Natural Language
> Understanding
> > and Recommender Systems. With pervasive and cost effective cloud
> computing,
> > large labeled datasets and continued algorithmic innovation, Deep
> Learning
> > has become the one of the most popular classes of algorithms for machine
> > learning practitioners in recent years.
> >
> > == Rational ==
> >
> > The adoption of deep learning is quickly expanding from initial deep
> > domain experts rooted in academia to data scientists and developers
> working
> > to deploy intelligent services and products. Deep learning however has
> many
> > challenges.  These include model training time (which can take days to
> > weeks), programmability (not everyone writes Python or C++ and like
> > symbolic programming) and balancing production readiness (support for
> > things like failover) with development flexibility (ability to program
> > different ways, support for new operators and model types) and speed of
> > execution (fast and scalable model training).  Other frameworks excel on
> > some but not all of these aspects.
> >
> >
> > == Initial Goals ==
> >
> > MXNet is a fairly established project on GitHub with its first code
> > contribution in April 2015 and roughly 200 contributors. It is used by
> > several large companies and some of the top research institutions on the
> > planet. Initial goals would be the following:
> >
> >  1. Move the existing codebase(s) to Apache
> >  1. Integrate with the Apache development process/sign CLAs
> >  1. Ensure all dependencies are compliant with Apache License version 2.0
> >  1. Incremental development and releases per Apache guidelines
> >  1. Establish engineering discipline and a predictable release cadence of
> > high quality releases
> >  1. Expand the community beyond the current base of expert level users
> >  1. Improve usability and the overall developer/user experience
> >  1. Add additional functionality to address newer problem types and
> > algorithms
> >
> >
> > == Current Status ==
> >
> > === Meritocracy ===
> >
> > The MXNet project already operates on meritocratic principles. Today,
> > MXNet has developers worldwide and has accepted multiple major patches
> from
> > a diverse set of contributors within both industry and academia. We would
> > like to follow ASF meritocratic principles to encourage more developers
> to
> > contribute in this project. We know that only active and committed
> > developers from a diverse set of backgrounds can make MXNet a successful
> > project.  We are also improving the documentation and code to help new
> > developers get started quickly.
> >
> > === Community ===
> >
> > Acceptance into the Apache foundation would bolster the growing user and
> > developer community around MXNet. That community includes around 200
> > contributors from academia and industry. The core developers of our
> project
> > are listed in our contributors below and are also represented by logos on
> > the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> > Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of
> Alberta,
> > University of Washington and Wolfram.
> >
> > === Core Developers ===
> >
> > (with GitHub logins)
> >
> >  * Tianqi Chen (@tqchen)
> >  * Mu Li (@mli)
> >  * Junyuan Xie (@piiswrong)
> >  * Bing Xu (@antinucleon)
> >  * Chiyuan Zhang (@pluskid)
> >  * Minjie Wang (@jermainewang)
> >  * Naiyan Wang (@winstywang)
> >  * Yizhi Liu (@javelinjs)
> >  * Tong He (@hetong007)
> >  * Qiang Kou (@thirdwing)
> >  * Xingjian Shi (@sxjscience)
> >
> > === Alignment ===
> >
> > ASF is already the home of many distributed platforms, e.g., Hadoop,
> Spark
> > and Mahout, each of which targets a different application domain. MXNet,
> > being a distributed platform for large-scale deep learning, focuses on
> > another important domain for which there still lacks a scalable,
> > programmable, flexible and super fast open-source platform. The recent
> > success of deep learning models especially for vision and speech
> > recognition tasks has generated interests in both applying existing deep
> > learning models and in developing new ones. Thus, an open-source platform
> > for deep learning backed by some of the top industry and academic players
> > will be able to attract a large community of users and developers. MXNet
> is
> > a complex system needing many iterations of design, implementation and
> > testing. Apache's collaboration framework which encourages active
> > contribution from developers will inevitably help improve the quality of
> > the system, as shown in the success of Hadoop, Spark, etc. Equally
> > important is the community of users which helps identify real-life
> > applications of deep learning, and helps to evaluate the system's
> > performance and ease-of-use. We hope to leverage ASF for coordinating and
> > promoting both communities, and in return benefit the communities with
> > another useful tool.
> >
> > == Known Risks ==
> >
> > === Orphaned products ===
> >
> > Given the current level of investment in MXNet and the stakeholders using
> > it - the risk of the project being abandoned is minimal. Amazon, for
> > example, is in active development to use MXNet in many of its services
> and
> > many large corporations use it in their production applications.
> >
> > === Inexperience with Open Source ===
> >
> > MXNet has existed as a healthy open source project for more than a year.
> > During that time, the project has attracted 200+ contributors.
> >
> > === Homogenous Developers ===
> >
> > The initial list of committers and contributors includes developers from
> > several institutions and industry participants (see above).
> >
> > === Reliance on Salaried Developers ===
> >
> > Like most open source projects, MXNet receives a substantial support from
> > salaried developers. A large fraction of MXNet development is supported
> by
> > graduate students at various universities in the course of research
> degrees
> > - this is more a “volunteer” relationship, since in most cases students
> > contribute vastly more than is necessary to immediately support research.
> > In addition, those working from within corporations are devoting
> > significant time and effort in the project - and these come from several
> > organizations.
> >
> > === A Excessive Fascination with the Apache Brand ===
> >
> > We choose Apache not for publicity. We have two purposes. First, we hope
> > that Apache's known best-practices for managing a mature open source
> > project can help guide us.  For example, we are feeling the growing pains
> > of a successful open source project as we attempt a major refactor of the
> > internals while customers are using the system in production. We seek
> > guidance in communicating breaking API changes and version revisions.
> > Also, as our involvement from major corporations increases, we want to
> > assure our users that MXNet will stay open and not favor any particular
> > platform or environment. These are some examples of the know-how and
> > discipline we're hoping Apache can bring to our project.
> >
> > Second, we want to leverage Apache's reputation to recruit more
> developers
> > to create a diverse community.
> >
> > === Relationship with Other Apache Products ===
> >
> > Apache Mahout and Apache Spark's MLlib are general machine learning
> > systems. Deep learning algorithms can thus be implemented on these two
> > platforms as well. However, in practice, the overlap will be minimal.
> Deep
> > learning is so computationally intensive that it often requires
> specialized
> > GPU hardware to accomplish tasks of meaningful size.  Making efficient
> use
> > of GPU hardware is complex because the hardware is so fast that the
> > supporting systems around it must be carefully optimized to keep the GPU
> > cores busy.  Extending this capability to distributed multi-GPU and
> > multi-host environments requires great care.  This is a critical
> > differentiator between MXNet and existing Apache machine learning
> systems.
> >
> > Mahout and Spark ML-LIB follow models where their nodes run
> synchronously.
> > This is the fundamental difference to MXNet who follows the parameter
> > server framework. MXNet can run synchronously or asynchronously. In
> > addition, MXNet has optimizations for training a wide range of deep
> > learning models using a variety of approaches (e.g., model parallelism
> and
> > data parallelism) which makes MXNet much more efficient (near-linear
> > speedup on state of the art models). MXNet also supports both imperative
> > and symbolic approaches providing ease of programming for deep learning
> > algorithms.
> >
> > Other Apache projects that are potentially complimentary:
> >
> > Apache Arrow - read data in Apache Arrow‘s internal format from MXNet,
> > that would allow users to run ETL/preprocessing in Spark, save the
> results
> > in Arrow’s format and then run DL algorithms on it.
> >
> > Apache Singa - MXNet and Singa are both deep learning projects, and can
> > benefit from a larger deep learning community at Apache.
> >
> > == Documentation ==
> >
> > Documentation has recently migrated to http://mxnet.io.  We continue to
> > refine and improve the documentation.
> >
> > == Initial Source ==
> >
> > We currently use Github to maintain our source code,
> > https://github.com/MXNet
> >
> > == Source and Intellectual Property Submission Plan ==
> >
> > MXNet Code is available under Apache License, Version 2.0. We will work
> > with the committers to get CLAs signed and review previous contributions.
> >
> > == External Dependencies ==
> >
> >  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> > (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> > lib-zeromq), TBB
> >  * required for GPU usage: cudnn, cuda
> >  * required for python usage: Python 2/3
> >  * required for R module: R, Rcpp (GPLv2 licensing)
> >  * optional for image preparation and preprocessing: opencv
> >  * optional dependencies for additional features: torch7, numba, cython
> > (in NNVM branch)
> >
> > Rcpt and lib-zeromq are expected to be licensing discussions.
> >
> > == Cryptography ==
> >
> > Not Applicable
> >
> > == Required Resources ==
> >
> > === Mailing Lists ===
> >
> > There is currently no mailing list.
> >
> > === Issue Tracking ===
> >
> > Currently uses GitHub to track issues. Would like to continue to do so.
> >
> > == Committers and Affiliations ==
> >
> >  * Tianqi Chen (UW)
> >  * Mu Li (AWS)
> >  * Junyuan Xie (AWS)
> >  * Bing Xu (Apple)
> >  * Chiyuan Zhang (MIT)
> >  * Minjie Wang (UYU)
> >  * Naiyan Wang (Tusimple)
> >  * Yizhi Liu (Mediav)
> >  * Tong He (Simon Fraser University)
> >  * Qiang Kou (Indiana U)
> >  * Xingjian Shi (HKUST)
> >
> > == Sponsors ==
> >
> > === Champion ===
> >
> > Henri Yandell (bayard at apache.org)
> >
> > === Nominated Mentors ===
> >
> > Sebastian Schelter (ssc@apache.org)
> >
> >
> > === Sponsoring Entity ===
> >
> > We are requesting the Incubator to sponsor this project.
> >
> >
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Thanks for all the feedback and interested parties :)

My aim is to propose a vote on Monday, unless someone raises an issue
before then.

Hen

On Thu, Jan 5, 2017 at 9:12 PM, Henri Yandell <ba...@apache.org> wrote:

> Hello Incubator,
>
> I'd like to propose a new incubator Apache MXNet podling.
>
> The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> committers, 200 contributors) is very interested in joining Apache. MXNet
> is an open-source deep learning framework that allows you to define, train,
> and deploy deep neural networks on a wide array of devices, from cloud
> infrastructure to mobile devices.
>
> The wiki proposal page is located here:
>
>   https://wiki.apache.org/incubator/MXNetProposal
>
> I've included the text below in case anyone wants to focus on parts of it
> in a reply.
>
> Looking forward to your thoughts, and for lots of interested Apache
> members to volunteer to mentor the project in addition to Sebastian and
> myself.
>
> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!
>
> Thanks,
>
> Hen
> On behalf of the MXNet project
>
> ---------
>
> = MXNet: Apache Incubator Proposal =
>
> == Abstract ==
>
> MXNet is a Flexible and Efficient Library for Deep Learning
>
> == Proposal ==
>
> MXNet is an open-source deep learning framework that allows you to define,
> train, and deploy deep neural networks on a wide array of devices, from
> cloud infrastructure to mobile devices. It is highly scalable, allowing for
> fast model training, and supports a flexible programming model and multiple
> languages. MXNet allows you to mix symbolic and imperative programming
> flavors to maximize both efficiency and productivity. MXNet is built on a
> dynamic dependency scheduler that automatically parallelizes both symbolic
> and imperative operations on the fly. A graph optimization layer on top of
> that makes symbolic execution fast and memory efficient. The MXNet library
> is portable and lightweight, and it scales to multiple GPUs and multiple
> machines.
>
> == Background ==
>
> Deep learning is a subset of Machine learning and refers to a class of
> algorithms that use a hierarchical approach with non-linearities to
> discover and learn representations within data. Deep Learning has recently
> become very popular due to its applicability and advancement of domains
> such as Computer Vision, Speech Recognition, Natural Language Understanding
> and Recommender Systems. With pervasive and cost effective cloud computing,
> large labeled datasets and continued algorithmic innovation, Deep Learning
> has become the one of the most popular classes of algorithms for machine
> learning practitioners in recent years.
>
> == Rational ==
>
> The adoption of deep learning is quickly expanding from initial deep
> domain experts rooted in academia to data scientists and developers working
> to deploy intelligent services and products. Deep learning however has many
> challenges.  These include model training time (which can take days to
> weeks), programmability (not everyone writes Python or C++ and like
> symbolic programming) and balancing production readiness (support for
> things like failover) with development flexibility (ability to program
> different ways, support for new operators and model types) and speed of
> execution (fast and scalable model training).  Other frameworks excel on
> some but not all of these aspects.
>
>
> == Initial Goals ==
>
> MXNet is a fairly established project on GitHub with its first code
> contribution in April 2015 and roughly 200 contributors. It is used by
> several large companies and some of the top research institutions on the
> planet. Initial goals would be the following:
>
>  1. Move the existing codebase(s) to Apache
>  1. Integrate with the Apache development process/sign CLAs
>  1. Ensure all dependencies are compliant with Apache License version 2.0
>  1. Incremental development and releases per Apache guidelines
>  1. Establish engineering discipline and a predictable release cadence of
> high quality releases
>  1. Expand the community beyond the current base of expert level users
>  1. Improve usability and the overall developer/user experience
>  1. Add additional functionality to address newer problem types and
> algorithms
>
>
> == Current Status ==
>
> === Meritocracy ===
>
> The MXNet project already operates on meritocratic principles. Today,
> MXNet has developers worldwide and has accepted multiple major patches from
> a diverse set of contributors within both industry and academia. We would
> like to follow ASF meritocratic principles to encourage more developers to
> contribute in this project. We know that only active and committed
> developers from a diverse set of backgrounds can make MXNet a successful
> project.  We are also improving the documentation and code to help new
> developers get started quickly.
>
> === Community ===
>
> Acceptance into the Apache foundation would bolster the growing user and
> developer community around MXNet. That community includes around 200
> contributors from academia and industry. The core developers of our project
> are listed in our contributors below and are also represented by logos on
> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
> University of Washington and Wolfram.
>
> === Core Developers ===
>
> (with GitHub logins)
>
>  * Tianqi Chen (@tqchen)
>  * Mu Li (@mli)
>  * Junyuan Xie (@piiswrong)
>  * Bing Xu (@antinucleon)
>  * Chiyuan Zhang (@pluskid)
>  * Minjie Wang (@jermainewang)
>  * Naiyan Wang (@winstywang)
>  * Yizhi Liu (@javelinjs)
>  * Tong He (@hetong007)
>  * Qiang Kou (@thirdwing)
>  * Xingjian Shi (@sxjscience)
>
> === Alignment ===
>
> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
> and Mahout, each of which targets a different application domain. MXNet,
> being a distributed platform for large-scale deep learning, focuses on
> another important domain for which there still lacks a scalable,
> programmable, flexible and super fast open-source platform. The recent
> success of deep learning models especially for vision and speech
> recognition tasks has generated interests in both applying existing deep
> learning models and in developing new ones. Thus, an open-source platform
> for deep learning backed by some of the top industry and academic players
> will be able to attract a large community of users and developers. MXNet is
> a complex system needing many iterations of design, implementation and
> testing. Apache's collaboration framework which encourages active
> contribution from developers will inevitably help improve the quality of
> the system, as shown in the success of Hadoop, Spark, etc. Equally
> important is the community of users which helps identify real-life
> applications of deep learning, and helps to evaluate the system's
> performance and ease-of-use. We hope to leverage ASF for coordinating and
> promoting both communities, and in return benefit the communities with
> another useful tool.
>
> == Known Risks ==
>
> === Orphaned products ===
>
> Given the current level of investment in MXNet and the stakeholders using
> it - the risk of the project being abandoned is minimal. Amazon, for
> example, is in active development to use MXNet in many of its services and
> many large corporations use it in their production applications.
>
> === Inexperience with Open Source ===
>
> MXNet has existed as a healthy open source project for more than a year.
> During that time, the project has attracted 200+ contributors.
>
> === Homogenous Developers ===
>
> The initial list of committers and contributors includes developers from
> several institutions and industry participants (see above).
>
> === Reliance on Salaried Developers ===
>
> Like most open source projects, MXNet receives a substantial support from
> salaried developers. A large fraction of MXNet development is supported by
> graduate students at various universities in the course of research degrees
> - this is more a “volunteer” relationship, since in most cases students
> contribute vastly more than is necessary to immediately support research.
> In addition, those working from within corporations are devoting
> significant time and effort in the project - and these come from several
> organizations.
>
> === A Excessive Fascination with the Apache Brand ===
>
> We choose Apache not for publicity. We have two purposes. First, we hope
> that Apache's known best-practices for managing a mature open source
> project can help guide us.  For example, we are feeling the growing pains
> of a successful open source project as we attempt a major refactor of the
> internals while customers are using the system in production. We seek
> guidance in communicating breaking API changes and version revisions.
> Also, as our involvement from major corporations increases, we want to
> assure our users that MXNet will stay open and not favor any particular
> platform or environment. These are some examples of the know-how and
> discipline we're hoping Apache can bring to our project.
>
> Second, we want to leverage Apache's reputation to recruit more developers
> to create a diverse community.
>
> === Relationship with Other Apache Products ===
>
> Apache Mahout and Apache Spark's MLlib are general machine learning
> systems. Deep learning algorithms can thus be implemented on these two
> platforms as well. However, in practice, the overlap will be minimal.  Deep
> learning is so computationally intensive that it often requires specialized
> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
> of GPU hardware is complex because the hardware is so fast that the
> supporting systems around it must be carefully optimized to keep the GPU
> cores busy.  Extending this capability to distributed multi-GPU and
> multi-host environments requires great care.  This is a critical
> differentiator between MXNet and existing Apache machine learning systems.
>
> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
> This is the fundamental difference to MXNet who follows the parameter
> server framework. MXNet can run synchronously or asynchronously. In
> addition, MXNet has optimizations for training a wide range of deep
> learning models using a variety of approaches (e.g., model parallelism and
> data parallelism) which makes MXNet much more efficient (near-linear
> speedup on state of the art models). MXNet also supports both imperative
> and symbolic approaches providing ease of programming for deep learning
> algorithms.
>
> Other Apache projects that are potentially complimentary:
>
> Apache Arrow - read data in Apache Arrow‘s internal format from MXNet,
> that would allow users to run ETL/preprocessing in Spark, save the results
> in Arrow’s format and then run DL algorithms on it.
>
> Apache Singa - MXNet and Singa are both deep learning projects, and can
> benefit from a larger deep learning community at Apache.
>
> == Documentation ==
>
> Documentation has recently migrated to http://mxnet.io.  We continue to
> refine and improve the documentation.
>
> == Initial Source ==
>
> We currently use Github to maintain our source code,
> https://github.com/MXNet
>
> == Source and Intellectual Property Submission Plan ==
>
> MXNet Code is available under Apache License, Version 2.0. We will work
> with the committers to get CLAs signed and review previous contributions.
>
> == External Dependencies ==
>
>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> lib-zeromq), TBB
>  * required for GPU usage: cudnn, cuda
>  * required for python usage: Python 2/3
>  * required for R module: R, Rcpp (GPLv2 licensing)
>  * optional for image preparation and preprocessing: opencv
>  * optional dependencies for additional features: torch7, numba, cython
> (in NNVM branch)
>
> Rcpt and lib-zeromq are expected to be licensing discussions.
>
> == Cryptography ==
>
> Not Applicable
>
> == Required Resources ==
>
> === Mailing Lists ===
>
> There is currently no mailing list.
>
> === Issue Tracking ===
>
> Currently uses GitHub to track issues. Would like to continue to do so.
>
> == Committers and Affiliations ==
>
>  * Tianqi Chen (UW)
>  * Mu Li (AWS)
>  * Junyuan Xie (AWS)
>  * Bing Xu (Apple)
>  * Chiyuan Zhang (MIT)
>  * Minjie Wang (UYU)
>  * Naiyan Wang (Tusimple)
>  * Yizhi Liu (Mediav)
>  * Tong He (Simon Fraser University)
>  * Qiang Kou (Indiana U)
>  * Xingjian Shi (HKUST)
>
> == Sponsors ==
>
> === Champion ===
>
> Henri Yandell (bayard at apache.org)
>
> === Nominated Mentors ===
>
> Sebastian Schelter (ssc@apache.org)
>
>
> === Sponsoring Entity ===
>
> We are requesting the Incubator to sponsor this project.
>
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Added :)

On Wed, Jan 11, 2017 at 10:43 PM, 项亮 <xl...@gmail.com> wrote:

> I would like to volunteer as a committer for MXNet
>
> github id: xlvector
> email: xlvector@gmail.com
>
> Liang Xiang from Toutiao Lab
>
> On 2017-01-06 13:12 (+0800), Henri Yandell <ba...@apache.org> wrote:
> > Hello Incubator,
> >
> > I'd like to propose a new incubator Apache MXNet podling.
> >
> > The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> committers,
> > 200 contributors) is very interested in joining Apache. MXNet is an
> > open-source deep learning framework that allows you to define, train, and
> > deploy deep neural networks on a wide array of devices, from cloud
> > infrastructure to mobile devices.
> >
> > The wiki proposal page is located here:
> >
> >   https://wiki.apache.org/incubator/MXNetProposal
> >
> > I've included the text below in case anyone wants to focus on parts of it
> > in a reply.
> >
> > Looking forward to your thoughts, and for lots of interested Apache
> members
> > to volunteer to mentor the project in addition to Sebastian and myself.
> >
> > Currently the list of committers is based on the current active coders,
> so
> > we're also very interested in hearing from anyone else who is interested
> in
> > working on the project, be they current or future contributor!
> >
> > Thanks,
> >
> > Hen
> > On behalf of the MXNet project
> >
> > ---------
> >
> > = MXNet: Apache Incubator Proposal =
> >
> > == Abstract ==
> >
> > MXNet is a Flexible and Efficient Library for Deep Learning
> >
> > == Proposal ==
> >
> > MXNet is an open-source deep learning framework that allows you to
> define,
> > train, and deploy deep neural networks on a wide array of devices, from
> > cloud infrastructure to mobile devices. It is highly scalable, allowing
> for
> > fast model training, and supports a flexible programming model and
> multiple
> > languages. MXNet allows you to mix symbolic and imperative programming
> > flavors to maximize both efficiency and productivity. MXNet is built on a
> > dynamic dependency scheduler that automatically parallelizes both
> symbolic
> > and imperative operations on the fly. A graph optimization layer on top
> of
> > that makes symbolic execution fast and memory efficient. The MXNet
> library
> > is portable and lightweight, and it scales to multiple GPUs and multiple
> > machines.
> >
> > == Background ==
> >
> > Deep learning is a subset of Machine learning and refers to a class of
> > algorithms that use a hierarchical approach with non-linearities to
> > discover and learn representations within data. Deep Learning has
> recently
> > become very popular due to its applicability and advancement of domains
> > such as Computer Vision, Speech Recognition, Natural Language
> Understanding
> > and Recommender Systems. With pervasive and cost effective cloud
> computing,
> > large labeled datasets and continued algorithmic innovation, Deep
> Learning
> > has become the one of the most popular classes of algorithms for machine
> > learning practitioners in recent years.
> >
> > == Rational ==
> >
> > The adoption of deep learning is quickly expanding from initial deep
> domain
> > experts rooted in academia to data scientists and developers working to
> > deploy intelligent services and products. Deep learning however has many
> > challenges.  These include model training time (which can take days to
> > weeks), programmability (not everyone writes Python or C++ and like
> > symbolic programming) and balancing production readiness (support for
> > things like failover) with development flexibility (ability to program
> > different ways, support for new operators and model types) and speed of
> > execution (fast and scalable model training).  Other frameworks excel on
> > some but not all of these aspects.
> >
> >
> > == Initial Goals ==
> >
> > MXNet is a fairly established project on GitHub with its first code
> > contribution in April 2015 and roughly 200 contributors. It is used by
> > several large companies and some of the top research institutions on the
> > planet. Initial goals would be the following:
> >
> >  1. Move the existing codebase(s) to Apache
> >  1. Integrate with the Apache development process/sign CLAs
> >  1. Ensure all dependencies are compliant with Apache License version 2.0
> >  1. Incremental development and releases per Apache guidelines
> >  1. Establish engineering discipline and a predictable release cadence of
> > high quality releases
> >  1. Expand the community beyond the current base of expert level users
> >  1. Improve usability and the overall developer/user experience
> >  1. Add additional functionality to address newer problem types and
> > algorithms
> >
> >
> > == Current Status ==
> >
> > === Meritocracy ===
> >
> > The MXNet project already operates on meritocratic principles. Today,
> MXNet
> > has developers worldwide and has accepted multiple major patches from a
> > diverse set of contributors within both industry and academia. We would
> > like to follow ASF meritocratic principles to encourage more developers
> to
> > contribute in this project. We know that only active and committed
> > developers from a diverse set of backgrounds can make MXNet a successful
> > project.  We are also improving the documentation and code to help new
> > developers get started quickly.
> >
> > === Community ===
> >
> > Acceptance into the Apache foundation would bolster the growing user and
> > developer community around MXNet. That community includes around 200
> > contributors from academia and industry. The core developers of our
> project
> > are listed in our contributors below and are also represented by logos on
> > the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> > Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of
> Alberta,
> > University of Washington and Wolfram.
> >
> > === Core Developers ===
> >
> > (with GitHub logins)
> >
> >  * Tianqi Chen (@tqchen)
> >  * Mu Li (@mli)
> >  * Junyuan Xie (@piiswrong)
> >  * Bing Xu (@antinucleon)
> >  * Chiyuan Zhang (@pluskid)
> >  * Minjie Wang (@jermainewang)
> >  * Naiyan Wang (@winstywang)
> >  * Yizhi Liu (@javelinjs)
> >  * Tong He (@hetong007)
> >  * Qiang Kou (@thirdwing)
> >  * Xingjian Shi (@sxjscience)
> >
> > === Alignment ===
> >
> > ASF is already the home of many distributed platforms, e.g., Hadoop,
> Spark
> > and Mahout, each of which targets a different application domain. MXNet,
> > being a distributed platform for large-scale deep learning, focuses on
> > another important domain for which there still lacks a scalable,
> > programmable, flexible and super fast open-source platform. The recent
> > success of deep learning models especially for vision and speech
> > recognition tasks has generated interests in both applying existing deep
> > learning models and in developing new ones. Thus, an open-source platform
> > for deep learning backed by some of the top industry and academic players
> > will be able to attract a large community of users and developers. MXNet
> is
> > a complex system needing many iterations of design, implementation and
> > testing. Apache's collaboration framework which encourages active
> > contribution from developers will inevitably help improve the quality of
> > the system, as shown in the success of Hadoop, Spark, etc. Equally
> > important is the community of users which helps identify real-life
> > applications of deep learning, and helps to evaluate the system's
> > performance and ease-of-use. We hope to leverage ASF for coordinating and
> > promoting both communities, and in return benefit the communities with
> > another useful tool.
> >
> > == Known Risks ==
> >
> > === Orphaned products ===
> >
> > Given the current level of investment in MXNet and the stakeholders using
> > it - the risk of the project being abandoned is minimal. Amazon, for
> > example, is in active development to use MXNet in many of its services
> and
> > many large corporations use it in their production applications.
> >
> > === Inexperience with Open Source ===
> >
> > MXNet has existed as a healthy open source project for more than a year.
> > During that time, the project has attracted 200+ contributors.
> >
> > === Homogenous Developers ===
> >
> > The initial list of committers and contributors includes developers from
> > several institutions and industry participants (see above).
> >
> > === Reliance on Salaried Developers ===
> >
> > Like most open source projects, MXNet receives a substantial support from
> > salaried developers. A large fraction of MXNet development is supported
> by
> > graduate students at various universities in the course of research
> degrees
> > - this is more a “volunteer” relationship, since in most cases students
> > contribute vastly more than is necessary to immediately support research.
> > In addition, those working from within corporations are devoting
> > significant time and effort in the project - and these come from several
> > organizations.
> >
> > === A Excessive Fascination with the Apache Brand ===
> >
> > We choose Apache not for publicity. We have two purposes. First, we hope
> > that Apache's known best-practices for managing a mature open source
> > project can help guide us.  For example, we are feeling the growing pains
> > of a successful open source project as we attempt a major refactor of the
> > internals while customers are using the system in production. We seek
> > guidance in communicating breaking API changes and version revisions.
> > Also, as our involvement from major corporations increases, we want to
> > assure our users that MXNet will stay open and not favor any particular
> > platform or environment. These are some examples of the know-how and
> > discipline we're hoping Apache can bring to our project.
> >
> > Second, we want to leverage Apache's reputation to recruit more
> developers
> > to create a diverse community.
> >
> > === Relationship with Other Apache Products ===
> >
> > Apache Mahout and Apache Spark's MLlib are general machine learning
> > systems. Deep learning algorithms can thus be implemented on these two
> > platforms as well. However, in practice, the overlap will be minimal.
> Deep
> > learning is so computationally intensive that it often requires
> specialized
> > GPU hardware to accomplish tasks of meaningful size.  Making efficient
> use
> > of GPU hardware is complex because the hardware is so fast that the
> > supporting systems around it must be carefully optimized to keep the GPU
> > cores busy.  Extending this capability to distributed multi-GPU and
> > multi-host environments requires great care.  This is a critical
> > differentiator between MXNet and existing Apache machine learning
> systems.
> >
> > Mahout and Spark ML-LIB follow models where their nodes run
> synchronously.
> > This is the fundamental difference to MXNet who follows the parameter
> > server framework. MXNet can run synchronously or asynchronously. In
> > addition, MXNet has optimizations for training a wide range of deep
> > learning models using a variety of approaches (e.g., model parallelism
> and
> > data parallelism) which makes MXNet much more efficient (near-linear
> > speedup on state of the art models). MXNet also supports both imperative
> > and symbolic approaches providing ease of programming for deep learning
> > algorithms.
> >
> > Other Apache projects that are potentially complimentary:
> >
> > Apache Arrow - read data in Apache Arrow‘s internal format from MXNet,
> that
> > would allow users to run ETL/preprocessing in Spark, save the results in
> > Arrow’s format and then run DL algorithms on it.
> >
> > Apache Singa - MXNet and Singa are both deep learning projects, and can
> > benefit from a larger deep learning community at Apache.
> >
> > == Documentation ==
> >
> > Documentation has recently migrated to http://mxnet.io.  We continue to
> > refine and improve the documentation.
> >
> > == Initial Source ==
> >
> > We currently use Github to maintain our source code,
> > https://github.com/MXNet
> >
> > == Source and Intellectual Property Submission Plan ==
> >
> > MXNet Code is available under Apache License, Version 2.0. We will work
> > with the committers to get CLAs signed and review previous contributions.
> >
> > == External Dependencies ==
> >
> >  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> > (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> > lib-zeromq), TBB
> >  * required for GPU usage: cudnn, cuda
> >  * required for python usage: Python 2/3
> >  * required for R module: R, Rcpp (GPLv2 licensing)
> >  * optional for image preparation and preprocessing: opencv
> >  * optional dependencies for additional features: torch7, numba, cython
> (in
> > NNVM branch)
> >
> > Rcpt and lib-zeromq are expected to be licensing discussions.
> >
> > == Cryptography ==
> >
> > Not Applicable
> >
> > == Required Resources ==
> >
> > === Mailing Lists ===
> >
> > There is currently no mailing list.
> >
> > === Issue Tracking ===
> >
> > Currently uses GitHub to track issues. Would like to continue to do so.
> >
> > == Committers and Affiliations ==
> >
> >  * Tianqi Chen (UW)
> >  * Mu Li (AWS)
> >  * Junyuan Xie (AWS)
> >  * Bing Xu (Apple)
> >  * Chiyuan Zhang (MIT)
> >  * Minjie Wang (UYU)
> >  * Naiyan Wang (Tusimple)
> >  * Yizhi Liu (Mediav)
> >  * Tong He (Simon Fraser University)
> >  * Qiang Kou (Indiana U)
> >  * Xingjian Shi (HKUST)
> >
> > == Sponsors ==
> >
> > === Champion ===
> >
> > Henri Yandell (bayard at apache.org)
> >
> > === Nominated Mentors ===
> >
> > Sebastian Schelter (ssc@apache.org)
> >
> >
> > === Sponsoring Entity ===
> >
> > We are requesting the Incubator to sponsor this project.
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Added :)

On Thu, Jan 12, 2017 at 1:43 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:

> Hi Henri,
>
> My previous comment was just a review comment against the proposal,
> but I forgot to mentioning importance thing.
>
> > Currently the list of committers is based on the current active coders,
> so
> > we're also very interested in hearing from anyone else who is interested
> in
> > working on the project, be they current or future contributor!
>
> I'm also interested in working on MXNet :-)
>
> Thanks,
> - Tsuyoshi
>
> On Thu, Jan 12, 2017 at 3:43 PM, 项亮 <xl...@gmail.com> wrote:
> > I would like to volunteer as a committer for MXNet
> >
> > github id: xlvector
> > email: xlvector@gmail.com
> >
> > Liang Xiang from Toutiao Lab
> >
> > On 2017-01-06 13:12 (+0800), Henri Yandell <ba...@apache.org> wrote:
> >> Hello Incubator,
> >>
> >> I'd like to propose a new incubator Apache MXNet podling.
> >>
> >> The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> committers,
> >> 200 contributors) is very interested in joining Apache. MXNet is an
> >> open-source deep learning framework that allows you to define, train,
> and
> >> deploy deep neural networks on a wide array of devices, from cloud
> >> infrastructure to mobile devices.
> >>
> >> The wiki proposal page is located here:
> >>
> >>   https://wiki.apache.org/incubator/MXNetProposal
> >>
> >> I've included the text below in case anyone wants to focus on parts of
> it
> >> in a reply.
> >>
> >> Looking forward to your thoughts, and for lots of interested Apache
> members
> >> to volunteer to mentor the project in addition to Sebastian and myself.
> >>
> >> Currently the list of committers is based on the current active coders,
> so
> >> we're also very interested in hearing from anyone else who is
> interested in
> >> working on the project, be they current or future contributor!
> >>
> >> Thanks,
> >>
> >> Hen
> >> On behalf of the MXNet project
> >>
> >> ---------
> >>
> >> = MXNet: Apache Incubator Proposal =
> >>
> >> == Abstract ==
> >>
> >> MXNet is a Flexible and Efficient Library for Deep Learning
> >>
> >> == Proposal ==
> >>
> >> MXNet is an open-source deep learning framework that allows you to
> define,
> >> train, and deploy deep neural networks on a wide array of devices, from
> >> cloud infrastructure to mobile devices. It is highly scalable, allowing
> for
> >> fast model training, and supports a flexible programming model and
> multiple
> >> languages. MXNet allows you to mix symbolic and imperative programming
> >> flavors to maximize both efficiency and productivity. MXNet is built on
> a
> >> dynamic dependency scheduler that automatically parallelizes both
> symbolic
> >> and imperative operations on the fly. A graph optimization layer on top
> of
> >> that makes symbolic execution fast and memory efficient. The MXNet
> library
> >> is portable and lightweight, and it scales to multiple GPUs and multiple
> >> machines.
> >>
> >> == Background ==
> >>
> >> Deep learning is a subset of Machine learning and refers to a class of
> >> algorithms that use a hierarchical approach with non-linearities to
> >> discover and learn representations within data. Deep Learning has
> recently
> >> become very popular due to its applicability and advancement of domains
> >> such as Computer Vision, Speech Recognition, Natural Language
> Understanding
> >> and Recommender Systems. With pervasive and cost effective cloud
> computing,
> >> large labeled datasets and continued algorithmic innovation, Deep
> Learning
> >> has become the one of the most popular classes of algorithms for machine
> >> learning practitioners in recent years.
> >>
> >> == Rational ==
> >>
> >> The adoption of deep learning is quickly expanding from initial deep
> domain
> >> experts rooted in academia to data scientists and developers working to
> >> deploy intelligent services and products. Deep learning however has many
> >> challenges.  These include model training time (which can take days to
> >> weeks), programmability (not everyone writes Python or C++ and like
> >> symbolic programming) and balancing production readiness (support for
> >> things like failover) with development flexibility (ability to program
> >> different ways, support for new operators and model types) and speed of
> >> execution (fast and scalable model training).  Other frameworks excel on
> >> some but not all of these aspects.
> >>
> >>
> >> == Initial Goals ==
> >>
> >> MXNet is a fairly established project on GitHub with its first code
> >> contribution in April 2015 and roughly 200 contributors. It is used by
> >> several large companies and some of the top research institutions on the
> >> planet. Initial goals would be the following:
> >>
> >>  1. Move the existing codebase(s) to Apache
> >>  1. Integrate with the Apache development process/sign CLAs
> >>  1. Ensure all dependencies are compliant with Apache License version
> 2.0
> >>  1. Incremental development and releases per Apache guidelines
> >>  1. Establish engineering discipline and a predictable release cadence
> of
> >> high quality releases
> >>  1. Expand the community beyond the current base of expert level users
> >>  1. Improve usability and the overall developer/user experience
> >>  1. Add additional functionality to address newer problem types and
> >> algorithms
> >>
> >>
> >> == Current Status ==
> >>
> >> === Meritocracy ===
> >>
> >> The MXNet project already operates on meritocratic principles. Today,
> MXNet
> >> has developers worldwide and has accepted multiple major patches from a
> >> diverse set of contributors within both industry and academia. We would
> >> like to follow ASF meritocratic principles to encourage more developers
> to
> >> contribute in this project. We know that only active and committed
> >> developers from a diverse set of backgrounds can make MXNet a successful
> >> project.  We are also improving the documentation and code to help new
> >> developers get started quickly.
> >>
> >> === Community ===
> >>
> >> Acceptance into the Apache foundation would bolster the growing user and
> >> developer community around MXNet. That community includes around 200
> >> contributors from academia and industry. The core developers of our
> project
> >> are listed in our contributors below and are also represented by logos
> on
> >> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> >> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of
> Alberta,
> >> University of Washington and Wolfram.
> >>
> >> === Core Developers ===
> >>
> >> (with GitHub logins)
> >>
> >>  * Tianqi Chen (@tqchen)
> >>  * Mu Li (@mli)
> >>  * Junyuan Xie (@piiswrong)
> >>  * Bing Xu (@antinucleon)
> >>  * Chiyuan Zhang (@pluskid)
> >>  * Minjie Wang (@jermainewang)
> >>  * Naiyan Wang (@winstywang)
> >>  * Yizhi Liu (@javelinjs)
> >>  * Tong He (@hetong007)
> >>  * Qiang Kou (@thirdwing)
> >>  * Xingjian Shi (@sxjscience)
> >>
> >> === Alignment ===
> >>
> >> ASF is already the home of many distributed platforms, e.g., Hadoop,
> Spark
> >> and Mahout, each of which targets a different application domain. MXNet,
> >> being a distributed platform for large-scale deep learning, focuses on
> >> another important domain for which there still lacks a scalable,
> >> programmable, flexible and super fast open-source platform. The recent
> >> success of deep learning models especially for vision and speech
> >> recognition tasks has generated interests in both applying existing deep
> >> learning models and in developing new ones. Thus, an open-source
> platform
> >> for deep learning backed by some of the top industry and academic
> players
> >> will be able to attract a large community of users and developers.
> MXNet is
> >> a complex system needing many iterations of design, implementation and
> >> testing. Apache's collaboration framework which encourages active
> >> contribution from developers will inevitably help improve the quality of
> >> the system, as shown in the success of Hadoop, Spark, etc. Equally
> >> important is the community of users which helps identify real-life
> >> applications of deep learning, and helps to evaluate the system's
> >> performance and ease-of-use. We hope to leverage ASF for coordinating
> and
> >> promoting both communities, and in return benefit the communities with
> >> another useful tool.
> >>
> >> == Known Risks ==
> >>
> >> === Orphaned products ===
> >>
> >> Given the current level of investment in MXNet and the stakeholders
> using
> >> it - the risk of the project being abandoned is minimal. Amazon, for
> >> example, is in active development to use MXNet in many of its services
> and
> >> many large corporations use it in their production applications.
> >>
> >> === Inexperience with Open Source ===
> >>
> >> MXNet has existed as a healthy open source project for more than a year.
> >> During that time, the project has attracted 200+ contributors.
> >>
> >> === Homogenous Developers ===
> >>
> >> The initial list of committers and contributors includes developers from
> >> several institutions and industry participants (see above).
> >>
> >> === Reliance on Salaried Developers ===
> >>
> >> Like most open source projects, MXNet receives a substantial support
> from
> >> salaried developers. A large fraction of MXNet development is supported
> by
> >> graduate students at various universities in the course of research
> degrees
> >> - this is more a “volunteer” relationship, since in most cases students
> >> contribute vastly more than is necessary to immediately support
> research.
> >> In addition, those working from within corporations are devoting
> >> significant time and effort in the project - and these come from several
> >> organizations.
> >>
> >> === A Excessive Fascination with the Apache Brand ===
> >>
> >> We choose Apache not for publicity. We have two purposes. First, we hope
> >> that Apache's known best-practices for managing a mature open source
> >> project can help guide us.  For example, we are feeling the growing
> pains
> >> of a successful open source project as we attempt a major refactor of
> the
> >> internals while customers are using the system in production. We seek
> >> guidance in communicating breaking API changes and version revisions.
> >> Also, as our involvement from major corporations increases, we want to
> >> assure our users that MXNet will stay open and not favor any particular
> >> platform or environment. These are some examples of the know-how and
> >> discipline we're hoping Apache can bring to our project.
> >>
> >> Second, we want to leverage Apache's reputation to recruit more
> developers
> >> to create a diverse community.
> >>
> >> === Relationship with Other Apache Products ===
> >>
> >> Apache Mahout and Apache Spark's MLlib are general machine learning
> >> systems. Deep learning algorithms can thus be implemented on these two
> >> platforms as well. However, in practice, the overlap will be minimal.
> Deep
> >> learning is so computationally intensive that it often requires
> specialized
> >> GPU hardware to accomplish tasks of meaningful size.  Making efficient
> use
> >> of GPU hardware is complex because the hardware is so fast that the
> >> supporting systems around it must be carefully optimized to keep the GPU
> >> cores busy.  Extending this capability to distributed multi-GPU and
> >> multi-host environments requires great care.  This is a critical
> >> differentiator between MXNet and existing Apache machine learning
> systems.
> >>
> >> Mahout and Spark ML-LIB follow models where their nodes run
> synchronously.
> >> This is the fundamental difference to MXNet who follows the parameter
> >> server framework. MXNet can run synchronously or asynchronously. In
> >> addition, MXNet has optimizations for training a wide range of deep
> >> learning models using a variety of approaches (e.g., model parallelism
> and
> >> data parallelism) which makes MXNet much more efficient (near-linear
> >> speedup on state of the art models). MXNet also supports both imperative
> >> and symbolic approaches providing ease of programming for deep learning
> >> algorithms.
> >>
> >> Other Apache projects that are potentially complimentary:
> >>
> >> Apache Arrow - read data in Apache Arrow‘s internal format from MXNet,
> that
> >> would allow users to run ETL/preprocessing in Spark, save the results in
> >> Arrow’s format and then run DL algorithms on it.
> >>
> >> Apache Singa - MXNet and Singa are both deep learning projects, and can
> >> benefit from a larger deep learning community at Apache.
> >>
> >> == Documentation ==
> >>
> >> Documentation has recently migrated to http://mxnet.io.  We continue to
> >> refine and improve the documentation.
> >>
> >> == Initial Source ==
> >>
> >> We currently use Github to maintain our source code,
> >> https://github.com/MXNet
> >>
> >> == Source and Intellectual Property Submission Plan ==
> >>
> >> MXNet Code is available under Apache License, Version 2.0. We will work
> >> with the committers to get CLAs signed and review previous
> contributions.
> >>
> >> == External Dependencies ==
> >>
> >>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> >> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> >> lib-zeromq), TBB
> >>  * required for GPU usage: cudnn, cuda
> >>  * required for python usage: Python 2/3
> >>  * required for R module: R, Rcpp (GPLv2 licensing)
> >>  * optional for image preparation and preprocessing: opencv
> >>  * optional dependencies for additional features: torch7, numba, cython
> (in
> >> NNVM branch)
> >>
> >> Rcpt and lib-zeromq are expected to be licensing discussions.
> >>
> >> == Cryptography ==
> >>
> >> Not Applicable
> >>
> >> == Required Resources ==
> >>
> >> === Mailing Lists ===
> >>
> >> There is currently no mailing list.
> >>
> >> === Issue Tracking ===
> >>
> >> Currently uses GitHub to track issues. Would like to continue to do so.
> >>
> >> == Committers and Affiliations ==
> >>
> >>  * Tianqi Chen (UW)
> >>  * Mu Li (AWS)
> >>  * Junyuan Xie (AWS)
> >>  * Bing Xu (Apple)
> >>  * Chiyuan Zhang (MIT)
> >>  * Minjie Wang (UYU)
> >>  * Naiyan Wang (Tusimple)
> >>  * Yizhi Liu (Mediav)
> >>  * Tong He (Simon Fraser University)
> >>  * Qiang Kou (Indiana U)
> >>  * Xingjian Shi (HKUST)
> >>
> >> == Sponsors ==
> >>
> >> === Champion ===
> >>
> >> Henri Yandell (bayard at apache.org)
> >>
> >> === Nominated Mentors ===
> >>
> >> Sebastian Schelter (ssc@apache.org)
> >>
> >>
> >> === Sponsoring Entity ===
> >>
> >> We are requesting the Incubator to sponsor this project.
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: general-help@incubator.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Tsuyoshi Ozawa <oz...@apache.org>]

Hi Henri,

My previous comment was just a review comment against the proposal,
but I forgot to mentioning importance thing.

> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!

I'm also interested in working on MXNet :-)

Thanks,
- Tsuyoshi

On Thu, Jan 12, 2017 at 3:43 PM, 项亮 <xl...@gmail.com> wrote:
> I would like to volunteer as a committer for MXNet
>
> github id: xlvector
> email: xlvector@gmail.com
>
> Liang Xiang from Toutiao Lab
>
> On 2017-01-06 13:12 (+0800), Henri Yandell <ba...@apache.org> wrote:
>> Hello Incubator,
>>
>> I'd like to propose a new incubator Apache MXNet podling.
>>
>> The existing MXNet project (http://mxnet.io - 1.5 years old, 15 committers,
>> 200 contributors) is very interested in joining Apache. MXNet is an
>> open-source deep learning framework that allows you to define, train, and
>> deploy deep neural networks on a wide array of devices, from cloud
>> infrastructure to mobile devices.
>>
>> The wiki proposal page is located here:
>>
>>   https://wiki.apache.org/incubator/MXNetProposal
>>
>> I've included the text below in case anyone wants to focus on parts of it
>> in a reply.
>>
>> Looking forward to your thoughts, and for lots of interested Apache members
>> to volunteer to mentor the project in addition to Sebastian and myself.
>>
>> Currently the list of committers is based on the current active coders, so
>> we're also very interested in hearing from anyone else who is interested in
>> working on the project, be they current or future contributor!
>>
>> Thanks,
>>
>> Hen
>> On behalf of the MXNet project
>>
>> ---------
>>
>> = MXNet: Apache Incubator Proposal =
>>
>> == Abstract ==
>>
>> MXNet is a Flexible and Efficient Library for Deep Learning
>>
>> == Proposal ==
>>
>> MXNet is an open-source deep learning framework that allows you to define,
>> train, and deploy deep neural networks on a wide array of devices, from
>> cloud infrastructure to mobile devices. It is highly scalable, allowing for
>> fast model training, and supports a flexible programming model and multiple
>> languages. MXNet allows you to mix symbolic and imperative programming
>> flavors to maximize both efficiency and productivity. MXNet is built on a
>> dynamic dependency scheduler that automatically parallelizes both symbolic
>> and imperative operations on the fly. A graph optimization layer on top of
>> that makes symbolic execution fast and memory efficient. The MXNet library
>> is portable and lightweight, and it scales to multiple GPUs and multiple
>> machines.
>>
>> == Background ==
>>
>> Deep learning is a subset of Machine learning and refers to a class of
>> algorithms that use a hierarchical approach with non-linearities to
>> discover and learn representations within data. Deep Learning has recently
>> become very popular due to its applicability and advancement of domains
>> such as Computer Vision, Speech Recognition, Natural Language Understanding
>> and Recommender Systems. With pervasive and cost effective cloud computing,
>> large labeled datasets and continued algorithmic innovation, Deep Learning
>> has become the one of the most popular classes of algorithms for machine
>> learning practitioners in recent years.
>>
>> == Rational ==
>>
>> The adoption of deep learning is quickly expanding from initial deep domain
>> experts rooted in academia to data scientists and developers working to
>> deploy intelligent services and products. Deep learning however has many
>> challenges.  These include model training time (which can take days to
>> weeks), programmability (not everyone writes Python or C++ and like
>> symbolic programming) and balancing production readiness (support for
>> things like failover) with development flexibility (ability to program
>> different ways, support for new operators and model types) and speed of
>> execution (fast and scalable model training).  Other frameworks excel on
>> some but not all of these aspects.
>>
>>
>> == Initial Goals ==
>>
>> MXNet is a fairly established project on GitHub with its first code
>> contribution in April 2015 and roughly 200 contributors. It is used by
>> several large companies and some of the top research institutions on the
>> planet. Initial goals would be the following:
>>
>>  1. Move the existing codebase(s) to Apache
>>  1. Integrate with the Apache development process/sign CLAs
>>  1. Ensure all dependencies are compliant with Apache License version 2.0
>>  1. Incremental development and releases per Apache guidelines
>>  1. Establish engineering discipline and a predictable release cadence of
>> high quality releases
>>  1. Expand the community beyond the current base of expert level users
>>  1. Improve usability and the overall developer/user experience
>>  1. Add additional functionality to address newer problem types and
>> algorithms
>>
>>
>> == Current Status ==
>>
>> === Meritocracy ===
>>
>> The MXNet project already operates on meritocratic principles. Today, MXNet
>> has developers worldwide and has accepted multiple major patches from a
>> diverse set of contributors within both industry and academia. We would
>> like to follow ASF meritocratic principles to encourage more developers to
>> contribute in this project. We know that only active and committed
>> developers from a diverse set of backgrounds can make MXNet a successful
>> project.  We are also improving the documentation and code to help new
>> developers get started quickly.
>>
>> === Community ===
>>
>> Acceptance into the Apache foundation would bolster the growing user and
>> developer community around MXNet. That community includes around 200
>> contributors from academia and industry. The core developers of our project
>> are listed in our contributors below and are also represented by logos on
>> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
>> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
>> University of Washington and Wolfram.
>>
>> === Core Developers ===
>>
>> (with GitHub logins)
>>
>>  * Tianqi Chen (@tqchen)
>>  * Mu Li (@mli)
>>  * Junyuan Xie (@piiswrong)
>>  * Bing Xu (@antinucleon)
>>  * Chiyuan Zhang (@pluskid)
>>  * Minjie Wang (@jermainewang)
>>  * Naiyan Wang (@winstywang)
>>  * Yizhi Liu (@javelinjs)
>>  * Tong He (@hetong007)
>>  * Qiang Kou (@thirdwing)
>>  * Xingjian Shi (@sxjscience)
>>
>> === Alignment ===
>>
>> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
>> and Mahout, each of which targets a different application domain. MXNet,
>> being a distributed platform for large-scale deep learning, focuses on
>> another important domain for which there still lacks a scalable,
>> programmable, flexible and super fast open-source platform. The recent
>> success of deep learning models especially for vision and speech
>> recognition tasks has generated interests in both applying existing deep
>> learning models and in developing new ones. Thus, an open-source platform
>> for deep learning backed by some of the top industry and academic players
>> will be able to attract a large community of users and developers. MXNet is
>> a complex system needing many iterations of design, implementation and
>> testing. Apache's collaboration framework which encourages active
>> contribution from developers will inevitably help improve the quality of
>> the system, as shown in the success of Hadoop, Spark, etc. Equally
>> important is the community of users which helps identify real-life
>> applications of deep learning, and helps to evaluate the system's
>> performance and ease-of-use. We hope to leverage ASF for coordinating and
>> promoting both communities, and in return benefit the communities with
>> another useful tool.
>>
>> == Known Risks ==
>>
>> === Orphaned products ===
>>
>> Given the current level of investment in MXNet and the stakeholders using
>> it - the risk of the project being abandoned is minimal. Amazon, for
>> example, is in active development to use MXNet in many of its services and
>> many large corporations use it in their production applications.
>>
>> === Inexperience with Open Source ===
>>
>> MXNet has existed as a healthy open source project for more than a year.
>> During that time, the project has attracted 200+ contributors.
>>
>> === Homogenous Developers ===
>>
>> The initial list of committers and contributors includes developers from
>> several institutions and industry participants (see above).
>>
>> === Reliance on Salaried Developers ===
>>
>> Like most open source projects, MXNet receives a substantial support from
>> salaried developers. A large fraction of MXNet development is supported by
>> graduate students at various universities in the course of research degrees
>> - this is more a “volunteer” relationship, since in most cases students
>> contribute vastly more than is necessary to immediately support research.
>> In addition, those working from within corporations are devoting
>> significant time and effort in the project - and these come from several
>> organizations.
>>
>> === A Excessive Fascination with the Apache Brand ===
>>
>> We choose Apache not for publicity. We have two purposes. First, we hope
>> that Apache's known best-practices for managing a mature open source
>> project can help guide us.  For example, we are feeling the growing pains
>> of a successful open source project as we attempt a major refactor of the
>> internals while customers are using the system in production. We seek
>> guidance in communicating breaking API changes and version revisions.
>> Also, as our involvement from major corporations increases, we want to
>> assure our users that MXNet will stay open and not favor any particular
>> platform or environment. These are some examples of the know-how and
>> discipline we're hoping Apache can bring to our project.
>>
>> Second, we want to leverage Apache's reputation to recruit more developers
>> to create a diverse community.
>>
>> === Relationship with Other Apache Products ===
>>
>> Apache Mahout and Apache Spark's MLlib are general machine learning
>> systems. Deep learning algorithms can thus be implemented on these two
>> platforms as well. However, in practice, the overlap will be minimal.  Deep
>> learning is so computationally intensive that it often requires specialized
>> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
>> of GPU hardware is complex because the hardware is so fast that the
>> supporting systems around it must be carefully optimized to keep the GPU
>> cores busy.  Extending this capability to distributed multi-GPU and
>> multi-host environments requires great care.  This is a critical
>> differentiator between MXNet and existing Apache machine learning systems.
>>
>> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
>> This is the fundamental difference to MXNet who follows the parameter
>> server framework. MXNet can run synchronously or asynchronously. In
>> addition, MXNet has optimizations for training a wide range of deep
>> learning models using a variety of approaches (e.g., model parallelism and
>> data parallelism) which makes MXNet much more efficient (near-linear
>> speedup on state of the art models). MXNet also supports both imperative
>> and symbolic approaches providing ease of programming for deep learning
>> algorithms.
>>
>> Other Apache projects that are potentially complimentary:
>>
>> Apache Arrow - read data in Apache Arrow‘s internal format from MXNet, that
>> would allow users to run ETL/preprocessing in Spark, save the results in
>> Arrow’s format and then run DL algorithms on it.
>>
>> Apache Singa - MXNet and Singa are both deep learning projects, and can
>> benefit from a larger deep learning community at Apache.
>>
>> == Documentation ==
>>
>> Documentation has recently migrated to http://mxnet.io.  We continue to
>> refine and improve the documentation.
>>
>> == Initial Source ==
>>
>> We currently use Github to maintain our source code,
>> https://github.com/MXNet
>>
>> == Source and Intellectual Property Submission Plan ==
>>
>> MXNet Code is available under Apache License, Version 2.0. We will work
>> with the committers to get CLAs signed and review previous contributions.
>>
>> == External Dependencies ==
>>
>>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
>> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
>> lib-zeromq), TBB
>>  * required for GPU usage: cudnn, cuda
>>  * required for python usage: Python 2/3
>>  * required for R module: R, Rcpp (GPLv2 licensing)
>>  * optional for image preparation and preprocessing: opencv
>>  * optional dependencies for additional features: torch7, numba, cython (in
>> NNVM branch)
>>
>> Rcpt and lib-zeromq are expected to be licensing discussions.
>>
>> == Cryptography ==
>>
>> Not Applicable
>>
>> == Required Resources ==
>>
>> === Mailing Lists ===
>>
>> There is currently no mailing list.
>>
>> === Issue Tracking ===
>>
>> Currently uses GitHub to track issues. Would like to continue to do so.
>>
>> == Committers and Affiliations ==
>>
>>  * Tianqi Chen (UW)
>>  * Mu Li (AWS)
>>  * Junyuan Xie (AWS)
>>  * Bing Xu (Apple)
>>  * Chiyuan Zhang (MIT)
>>  * Minjie Wang (UYU)
>>  * Naiyan Wang (Tusimple)
>>  * Yizhi Liu (Mediav)
>>  * Tong He (Simon Fraser University)
>>  * Qiang Kou (Indiana U)
>>  * Xingjian Shi (HKUST)
>>
>> == Sponsors ==
>>
>> === Champion ===
>>
>> Henri Yandell (bayard at apache.org)
>>
>> === Nominated Mentors ===
>>
>> Sebastian Schelter (ssc@apache.org)
>>
>>
>> === Sponsoring Entity ===
>>
>> We are requesting the Incubator to sponsor this project.
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[项亮 <xl...@gmail.com>]

I would like to volunteer as a committer for MXNet

github id: xlvector
email: xlvector@gmail.com

Liang Xiang from Toutiao Lab

On 2017-01-06 13:12 (+0800), Henri Yandell <ba...@apache.org> wrote: 
> Hello Incubator,
> 
> I'd like to propose a new incubator Apache MXNet podling.
> 
> The existing MXNet project (http://mxnet.io - 1.5 years old, 15 committers,
> 200 contributors) is very interested in joining Apache. MXNet is an
> open-source deep learning framework that allows you to define, train, and
> deploy deep neural networks on a wide array of devices, from cloud
> infrastructure to mobile devices.
> 
> The wiki proposal page is located here:
> 
>   https://wiki.apache.org/incubator/MXNetProposal
> 
> I've included the text below in case anyone wants to focus on parts of it
> in a reply.
> 
> Looking forward to your thoughts, and for lots of interested Apache members
> to volunteer to mentor the project in addition to Sebastian and myself.
> 
> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!
> 
> Thanks,
> 
> Hen
> On behalf of the MXNet project
> 
> ---------
> 
> = MXNet: Apache Incubator Proposal =
> 
> == Abstract ==
> 
> MXNet is a Flexible and Efficient Library for Deep Learning
> 
> == Proposal ==
> 
> MXNet is an open-source deep learning framework that allows you to define,
> train, and deploy deep neural networks on a wide array of devices, from
> cloud infrastructure to mobile devices. It is highly scalable, allowing for
> fast model training, and supports a flexible programming model and multiple
> languages. MXNet allows you to mix symbolic and imperative programming
> flavors to maximize both efficiency and productivity. MXNet is built on a
> dynamic dependency scheduler that automatically parallelizes both symbolic
> and imperative operations on the fly. A graph optimization layer on top of
> that makes symbolic execution fast and memory efficient. The MXNet library
> is portable and lightweight, and it scales to multiple GPUs and multiple
> machines.
> 
> == Background ==
> 
> Deep learning is a subset of Machine learning and refers to a class of
> algorithms that use a hierarchical approach with non-linearities to
> discover and learn representations within data. Deep Learning has recently
> become very popular due to its applicability and advancement of domains
> such as Computer Vision, Speech Recognition, Natural Language Understanding
> and Recommender Systems. With pervasive and cost effective cloud computing,
> large labeled datasets and continued algorithmic innovation, Deep Learning
> has become the one of the most popular classes of algorithms for machine
> learning practitioners in recent years.
> 
> == Rational ==
> 
> The adoption of deep learning is quickly expanding from initial deep domain
> experts rooted in academia to data scientists and developers working to
> deploy intelligent services and products. Deep learning however has many
> challenges.  These include model training time (which can take days to
> weeks), programmability (not everyone writes Python or C++ and like
> symbolic programming) and balancing production readiness (support for
> things like failover) with development flexibility (ability to program
> different ways, support for new operators and model types) and speed of
> execution (fast and scalable model training).  Other frameworks excel on
> some but not all of these aspects.
> 
> 
> == Initial Goals ==
> 
> MXNet is a fairly established project on GitHub with its first code
> contribution in April 2015 and roughly 200 contributors. It is used by
> several large companies and some of the top research institutions on the
> planet. Initial goals would be the following:
> 
>  1. Move the existing codebase(s) to Apache
>  1. Integrate with the Apache development process/sign CLAs
>  1. Ensure all dependencies are compliant with Apache License version 2.0
>  1. Incremental development and releases per Apache guidelines
>  1. Establish engineering discipline and a predictable release cadence of
> high quality releases
>  1. Expand the community beyond the current base of expert level users
>  1. Improve usability and the overall developer/user experience
>  1. Add additional functionality to address newer problem types and
> algorithms
> 
> 
> == Current Status ==
> 
> === Meritocracy ===
> 
> The MXNet project already operates on meritocratic principles. Today, MXNet
> has developers worldwide and has accepted multiple major patches from a
> diverse set of contributors within both industry and academia. We would
> like to follow ASF meritocratic principles to encourage more developers to
> contribute in this project. We know that only active and committed
> developers from a diverse set of backgrounds can make MXNet a successful
> project.  We are also improving the documentation and code to help new
> developers get started quickly.
> 
> === Community ===
> 
> Acceptance into the Apache foundation would bolster the growing user and
> developer community around MXNet. That community includes around 200
> contributors from academia and industry. The core developers of our project
> are listed in our contributors below and are also represented by logos on
> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
> University of Washington and Wolfram.
> 
> === Core Developers ===
> 
> (with GitHub logins)
> 
>  * Tianqi Chen (@tqchen)
>  * Mu Li (@mli)
>  * Junyuan Xie (@piiswrong)
>  * Bing Xu (@antinucleon)
>  * Chiyuan Zhang (@pluskid)
>  * Minjie Wang (@jermainewang)
>  * Naiyan Wang (@winstywang)
>  * Yizhi Liu (@javelinjs)
>  * Tong He (@hetong007)
>  * Qiang Kou (@thirdwing)
>  * Xingjian Shi (@sxjscience)
> 
> === Alignment ===
> 
> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
> and Mahout, each of which targets a different application domain. MXNet,
> being a distributed platform for large-scale deep learning, focuses on
> another important domain for which there still lacks a scalable,
> programmable, flexible and super fast open-source platform. The recent
> success of deep learning models especially for vision and speech
> recognition tasks has generated interests in both applying existing deep
> learning models and in developing new ones. Thus, an open-source platform
> for deep learning backed by some of the top industry and academic players
> will be able to attract a large community of users and developers. MXNet is
> a complex system needing many iterations of design, implementation and
> testing. Apache's collaboration framework which encourages active
> contribution from developers will inevitably help improve the quality of
> the system, as shown in the success of Hadoop, Spark, etc. Equally
> important is the community of users which helps identify real-life
> applications of deep learning, and helps to evaluate the system's
> performance and ease-of-use. We hope to leverage ASF for coordinating and
> promoting both communities, and in return benefit the communities with
> another useful tool.
> 
> == Known Risks ==
> 
> === Orphaned products ===
> 
> Given the current level of investment in MXNet and the stakeholders using
> it - the risk of the project being abandoned is minimal. Amazon, for
> example, is in active development to use MXNet in many of its services and
> many large corporations use it in their production applications.
> 
> === Inexperience with Open Source ===
> 
> MXNet has existed as a healthy open source project for more than a year.
> During that time, the project has attracted 200+ contributors.
> 
> === Homogenous Developers ===
> 
> The initial list of committers and contributors includes developers from
> several institutions and industry participants (see above).
> 
> === Reliance on Salaried Developers ===
> 
> Like most open source projects, MXNet receives a substantial support from
> salaried developers. A large fraction of MXNet development is supported by
> graduate students at various universities in the course of research degrees
> - this is more a \u201cvolunteer\u201d relationship, since in most cases students
> contribute vastly more than is necessary to immediately support research.
> In addition, those working from within corporations are devoting
> significant time and effort in the project - and these come from several
> organizations.
> 
> === A Excessive Fascination with the Apache Brand ===
> 
> We choose Apache not for publicity. We have two purposes. First, we hope
> that Apache's known best-practices for managing a mature open source
> project can help guide us.  For example, we are feeling the growing pains
> of a successful open source project as we attempt a major refactor of the
> internals while customers are using the system in production. We seek
> guidance in communicating breaking API changes and version revisions.
> Also, as our involvement from major corporations increases, we want to
> assure our users that MXNet will stay open and not favor any particular
> platform or environment. These are some examples of the know-how and
> discipline we're hoping Apache can bring to our project.
> 
> Second, we want to leverage Apache's reputation to recruit more developers
> to create a diverse community.
> 
> === Relationship with Other Apache Products ===
> 
> Apache Mahout and Apache Spark's MLlib are general machine learning
> systems. Deep learning algorithms can thus be implemented on these two
> platforms as well. However, in practice, the overlap will be minimal.  Deep
> learning is so computationally intensive that it often requires specialized
> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
> of GPU hardware is complex because the hardware is so fast that the
> supporting systems around it must be carefully optimized to keep the GPU
> cores busy.  Extending this capability to distributed multi-GPU and
> multi-host environments requires great care.  This is a critical
> differentiator between MXNet and existing Apache machine learning systems.
> 
> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
> This is the fundamental difference to MXNet who follows the parameter
> server framework. MXNet can run synchronously or asynchronously. In
> addition, MXNet has optimizations for training a wide range of deep
> learning models using a variety of approaches (e.g., model parallelism and
> data parallelism) which makes MXNet much more efficient (near-linear
> speedup on state of the art models). MXNet also supports both imperative
> and symbolic approaches providing ease of programming for deep learning
> algorithms.
> 
> Other Apache projects that are potentially complimentary:
> 
> Apache Arrow - read data in Apache Arrow\u2018s internal format from MXNet, that
> would allow users to run ETL/preprocessing in Spark, save the results in
> Arrow\u2019s format and then run DL algorithms on it.
> 
> Apache Singa - MXNet and Singa are both deep learning projects, and can
> benefit from a larger deep learning community at Apache.
> 
> == Documentation ==
> 
> Documentation has recently migrated to http://mxnet.io.  We continue to
> refine and improve the documentation.
> 
> == Initial Source ==
> 
> We currently use Github to maintain our source code,
> https://github.com/MXNet
> 
> == Source and Intellectual Property Submission Plan ==
> 
> MXNet Code is available under Apache License, Version 2.0. We will work
> with the committers to get CLAs signed and review previous contributions.
> 
> == External Dependencies ==
> 
>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> lib-zeromq), TBB
>  * required for GPU usage: cudnn, cuda
>  * required for python usage: Python 2/3
>  * required for R module: R, Rcpp (GPLv2 licensing)
>  * optional for image preparation and preprocessing: opencv
>  * optional dependencies for additional features: torch7, numba, cython (in
> NNVM branch)
> 
> Rcpt and lib-zeromq are expected to be licensing discussions.
> 
> == Cryptography ==
> 
> Not Applicable
> 
> == Required Resources ==
> 
> === Mailing Lists ===
> 
> There is currently no mailing list.
> 
> === Issue Tracking ===
> 
> Currently uses GitHub to track issues. Would like to continue to do so.
> 
> == Committers and Affiliations ==
> 
>  * Tianqi Chen (UW)
>  * Mu Li (AWS)
>  * Junyuan Xie (AWS)
>  * Bing Xu (Apple)
>  * Chiyuan Zhang (MIT)
>  * Minjie Wang (UYU)
>  * Naiyan Wang (Tusimple)
>  * Yizhi Liu (Mediav)
>  * Tong He (Simon Fraser University)
>  * Qiang Kou (Indiana U)
>  * Xingjian Shi (HKUST)
> 
> == Sponsors ==
> 
> === Champion ===
> 
> Henri Yandell (bayard at apache.org)
> 
> === Nominated Mentors ===
> 
> Sebastian Schelter (ssc@apache.org)
> 
> 
> === Sponsoring Entity ===
> 
> We are requesting the Incubator to sponsor this project.
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

Added :)

On Fri, Jan 6, 2017 at 1:11 PM, Indhu Bharathi <in...@gmail.com>
wrote:

> Please sign me up as a committer - I've been working with Mu at work on
> MXNet (Amazon) and would love to get more involved in the project.
> GitHub ID:  indhub
>
> Thanks,
> Indu
>
> On 2017-01-05 21:12 (-0800), Henri Yandell <ba...@apache.org> wrote:
> > Hello Incubator,
> >
> > I'd like to propose a new incubator Apache MXNet podling.
> >
> > The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> committers,
> > 200 contributors) is very interested in joining Apache. MXNet is an
> > open-source deep learning framework that allows you to define, train, and
> > deploy deep neural networks on a wide array of devices, from cloud
> > infrastructure to mobile devices.
> >
> > The wiki proposal page is located here:
> >
> >   https://wiki.apache.org/incubator/MXNetProposal
> >
> > I've included the text below in case anyone wants to focus on parts of it
> > in a reply.
> >
> > Looking forward to your thoughts, and for lots of interested Apache
> members
> > to volunteer to mentor the project in addition to Sebastian and myself.
> >
> > Currently the list of committers is based on the current active coders,
> so
> > we're also very interested in hearing from anyone else who is interested
> in
> > working on the project, be they current or future contributor!
> >
> > Thanks,
> >
> > Hen
> > On behalf of the MXNet project
> >
> > ---------
> >
> > = MXNet: Apache Incubator Proposal =
> >
> > == Abstract ==
> >
> > MXNet is a Flexible and Efficient Library for Deep Learning
> >
> > == Proposal ==
> >
> > MXNet is an open-source deep learning framework that allows you to
> define,
> > train, and deploy deep neural networks on a wide array of devices, from
> > cloud infrastructure to mobile devices. It is highly scalable, allowing
> for
> > fast model training, and supports a flexible programming model and
> multiple
> > languages. MXNet allows you to mix symbolic and imperative programming
> > flavors to maximize both efficiency and productivity. MXNet is built on a
> > dynamic dependency scheduler that automatically parallelizes both
> symbolic
> > and imperative operations on the fly. A graph optimization layer on top
> of
> > that makes symbolic execution fast and memory efficient. The MXNet
> library
> > is portable and lightweight, and it scales to multiple GPUs and multiple
> > machines.
> >
> > == Background ==
> >
> > Deep learning is a subset of Machine learning and refers to a class of
> > algorithms that use a hierarchical approach with non-linearities to
> > discover and learn representations within data. Deep Learning has
> recently
> > become very popular due to its applicability and advancement of domains
> > such as Computer Vision, Speech Recognition, Natural Language
> Understanding
> > and Recommender Systems. With pervasive and cost effective cloud
> computing,
> > large labeled datasets and continued algorithmic innovation, Deep
> Learning
> > has become the one of the most popular classes of algorithms for machine
> > learning practitioners in recent years.
> >
> > == Rational ==
> >
> > The adoption of deep learning is quickly expanding from initial deep
> domain
> > experts rooted in academia to data scientists and developers working to
> > deploy intelligent services and products. Deep learning however has many
> > challenges.  These include model training time (which can take days to
> > weeks), programmability (not everyone writes Python or C++ and like
> > symbolic programming) and balancing production readiness (support for
> > things like failover) with development flexibility (ability to program
> > different ways, support for new operators and model types) and speed of
> > execution (fast and scalable model training).  Other frameworks excel on
> > some but not all of these aspects.
> >
> >
> > == Initial Goals ==
> >
> > MXNet is a fairly established project on GitHub with its first code
> > contribution in April 2015 and roughly 200 contributors. It is used by
> > several large companies and some of the top research institutions on the
> > planet. Initial goals would be the following:
> >
> >  1. Move the existing codebase(s) to Apache
> >  1. Integrate with the Apache development process/sign CLAs
> >  1. Ensure all dependencies are compliant with Apache License version 2.0
> >  1. Incremental development and releases per Apache guidelines
> >  1. Establish engineering discipline and a predictable release cadence of
> > high quality releases
> >  1. Expand the community beyond the current base of expert level users
> >  1. Improve usability and the overall developer/user experience
> >  1. Add additional functionality to address newer problem types and
> > algorithms
> >
> >
> > == Current Status ==
> >
> > === Meritocracy ===
> >
> > The MXNet project already operates on meritocratic principles. Today,
> MXNet
> > has developers worldwide and has accepted multiple major patches from a
> > diverse set of contributors within both industry and academia. We would
> > like to follow ASF meritocratic principles to encourage more developers
> to
> > contribute in this project. We know that only active and committed
> > developers from a diverse set of backgrounds can make MXNet a successful
> > project.  We are also improving the documentation and code to help new
> > developers get started quickly.
> >
> > === Community ===
> >
> > Acceptance into the Apache foundation would bolster the growing user and
> > developer community around MXNet. That community includes around 200
> > contributors from academia and industry. The core developers of our
> project
> > are listed in our contributors below and are also represented by logos on
> > the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> > Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of
> Alberta,
> > University of Washington and Wolfram.
> >
> > === Core Developers ===
> >
> > (with GitHub logins)
> >
> >  * Tianqi Chen (@tqchen)
> >  * Mu Li (@mli)
> >  * Junyuan Xie (@piiswrong)
> >  * Bing Xu (@antinucleon)
> >  * Chiyuan Zhang (@pluskid)
> >  * Minjie Wang (@jermainewang)
> >  * Naiyan Wang (@winstywang)
> >  * Yizhi Liu (@javelinjs)
> >  * Tong He (@hetong007)
> >  * Qiang Kou (@thirdwing)
> >  * Xingjian Shi (@sxjscience)
> >
> > === Alignment ===
> >
> > ASF is already the home of many distributed platforms, e.g., Hadoop,
> Spark
> > and Mahout, each of which targets a different application domain. MXNet,
> > being a distributed platform for large-scale deep learning, focuses on
> > another important domain for which there still lacks a scalable,
> > programmable, flexible and super fast open-source platform. The recent
> > success of deep learning models especially for vision and speech
> > recognition tasks has generated interests in both applying existing deep
> > learning models and in developing new ones. Thus, an open-source platform
> > for deep learning backed by some of the top industry and academic players
> > will be able to attract a large community of users and developers. MXNet
> is
> > a complex system needing many iterations of design, implementation and
> > testing. Apache's collaboration framework which encourages active
> > contribution from developers will inevitably help improve the quality of
> > the system, as shown in the success of Hadoop, Spark, etc. Equally
> > important is the community of users which helps identify real-life
> > applications of deep learning, and helps to evaluate the system's
> > performance and ease-of-use. We hope to leverage ASF for coordinating and
> > promoting both communities, and in return benefit the communities with
> > another useful tool.
> >
> > == Known Risks ==
> >
> > === Orphaned products ===
> >
> > Given the current level of investment in MXNet and the stakeholders using
> > it - the risk of the project being abandoned is minimal. Amazon, for
> > example, is in active development to use MXNet in many of its services
> and
> > many large corporations use it in their production applications.
> >
> > === Inexperience with Open Source ===
> >
> > MXNet has existed as a healthy open source project for more than a year.
> > During that time, the project has attracted 200+ contributors.
> >
> > === Homogenous Developers ===
> >
> > The initial list of committers and contributors includes developers from
> > several institutions and industry participants (see above).
> >
> > === Reliance on Salaried Developers ===
> >
> > Like most open source projects, MXNet receives a substantial support from
> > salaried developers. A large fraction of MXNet development is supported
> by
> > graduate students at various universities in the course of research
> degrees
> > - this is more a “volunteer” relationship, since in most cases students
> > contribute vastly more than is necessary to immediately support research.
> > In addition, those working from within corporations are devoting
> > significant time and effort in the project - and these come from several
> > organizations.
> >
> > === A Excessive Fascination with the Apache Brand ===
> >
> > We choose Apache not for publicity. We have two purposes. First, we hope
> > that Apache's known best-practices for managing a mature open source
> > project can help guide us.  For example, we are feeling the growing pains
> > of a successful open source project as we attempt a major refactor of the
> > internals while customers are using the system in production. We seek
> > guidance in communicating breaking API changes and version revisions.
> > Also, as our involvement from major corporations increases, we want to
> > assure our users that MXNet will stay open and not favor any particular
> > platform or environment. These are some examples of the know-how and
> > discipline we're hoping Apache can bring to our project.
> >
> > Second, we want to leverage Apache's reputation to recruit more
> developers
> > to create a diverse community.
> >
> > === Relationship with Other Apache Products ===
> >
> > Apache Mahout and Apache Spark's MLlib are general machine learning
> > systems. Deep learning algorithms can thus be implemented on these two
> > platforms as well. However, in practice, the overlap will be minimal.
> Deep
> > learning is so computationally intensive that it often requires
> specialized
> > GPU hardware to accomplish tasks of meaningful size.  Making efficient
> use
> > of GPU hardware is complex because the hardware is so fast that the
> > supporting systems around it must be carefully optimized to keep the GPU
> > cores busy.  Extending this capability to distributed multi-GPU and
> > multi-host environments requires great care.  This is a critical
> > differentiator between MXNet and existing Apache machine learning
> systems.
> >
> > Mahout and Spark ML-LIB follow models where their nodes run
> synchronously.
> > This is the fundamental difference to MXNet who follows the parameter
> > server framework. MXNet can run synchronously or asynchronously. In
> > addition, MXNet has optimizations for training a wide range of deep
> > learning models using a variety of approaches (e.g., model parallelism
> and
> > data parallelism) which makes MXNet much more efficient (near-linear
> > speedup on state of the art models). MXNet also supports both imperative
> > and symbolic approaches providing ease of programming for deep learning
> > algorithms.
> >
> > Other Apache projects that are potentially complimentary:
> >
> > Apache Arrow - read data in Apache Arrow‘s internal format from MXNet,
> that
> > would allow users to run ETL/preprocessing in Spark, save the results in
> > Arrow’s format and then run DL algorithms on it.
> >
> > Apache Singa - MXNet and Singa are both deep learning projects, and can
> > benefit from a larger deep learning community at Apache.
> >
> > == Documentation ==
> >
> > Documentation has recently migrated to http://mxnet.io.  We continue to
> > refine and improve the documentation.
> >
> > == Initial Source ==
> >
> > We currently use Github to maintain our source code,
> > https://github.com/MXNet
> >
> > == Source and Intellectual Property Submission Plan ==
> >
> > MXNet Code is available under Apache License, Version 2.0. We will work
> > with the committers to get CLAs signed and review previous contributions.
> >
> > == External Dependencies ==
> >
> >  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> > (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> > lib-zeromq), TBB
> >  * required for GPU usage: cudnn, cuda
> >  * required for python usage: Python 2/3
> >  * required for R module: R, Rcpp (GPLv2 licensing)
> >  * optional for image preparation and preprocessing: opencv
> >  * optional dependencies for additional features: torch7, numba, cython
> (in
> > NNVM branch)
> >
> > Rcpt and lib-zeromq are expected to be licensing discussions.
> >
> > == Cryptography ==
> >
> > Not Applicable
> >
> > == Required Resources ==
> >
> > === Mailing Lists ===
> >
> > There is currently no mailing list.
> >
> > === Issue Tracking ===
> >
> > Currently uses GitHub to track issues. Would like to continue to do so.
> >
> > == Committers and Affiliations ==
> >
> >  * Tianqi Chen (UW)
> >  * Mu Li (AWS)
> >  * Junyuan Xie (AWS)
> >  * Bing Xu (Apple)
> >  * Chiyuan Zhang (MIT)
> >  * Minjie Wang (UYU)
> >  * Naiyan Wang (Tusimple)
> >  * Yizhi Liu (Mediav)
> >  * Tong He (Simon Fraser University)
> >  * Qiang Kou (Indiana U)
> >  * Xingjian Shi (HKUST)
> >
> > == Sponsors ==
> >
> > === Champion ===
> >
> > Henri Yandell (bayard at apache.org)
> >
> > === Nominated Mentors ===
> >
> > Sebastian Schelter (ssc@apache.org)
> >
> >
> > === Sponsoring Entity ===
> >
> > We are requesting the Incubator to sponsor this project.
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Indhu Bharathi <in...@gmail.com>]

Please sign me up as a committer - I've been working with Mu at work on MXNet (Amazon) and would love to get more involved in the project.
GitHub ID:  indhub

Thanks,
Indu

On 2017-01-05 21:12 (-0800), Henri Yandell <ba...@apache.org> wrote: 
> Hello Incubator,
> 
> I'd like to propose a new incubator Apache MXNet podling.
> 
> The existing MXNet project (http://mxnet.io - 1.5 years old, 15 committers,
> 200 contributors) is very interested in joining Apache. MXNet is an
> open-source deep learning framework that allows you to define, train, and
> deploy deep neural networks on a wide array of devices, from cloud
> infrastructure to mobile devices.
> 
> The wiki proposal page is located here:
> 
>   https://wiki.apache.org/incubator/MXNetProposal
> 
> I've included the text below in case anyone wants to focus on parts of it
> in a reply.
> 
> Looking forward to your thoughts, and for lots of interested Apache members
> to volunteer to mentor the project in addition to Sebastian and myself.
> 
> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!
> 
> Thanks,
> 
> Hen
> On behalf of the MXNet project
> 
> ---------
> 
> = MXNet: Apache Incubator Proposal =
> 
> == Abstract ==
> 
> MXNet is a Flexible and Efficient Library for Deep Learning
> 
> == Proposal ==
> 
> MXNet is an open-source deep learning framework that allows you to define,
> train, and deploy deep neural networks on a wide array of devices, from
> cloud infrastructure to mobile devices. It is highly scalable, allowing for
> fast model training, and supports a flexible programming model and multiple
> languages. MXNet allows you to mix symbolic and imperative programming
> flavors to maximize both efficiency and productivity. MXNet is built on a
> dynamic dependency scheduler that automatically parallelizes both symbolic
> and imperative operations on the fly. A graph optimization layer on top of
> that makes symbolic execution fast and memory efficient. The MXNet library
> is portable and lightweight, and it scales to multiple GPUs and multiple
> machines.
> 
> == Background ==
> 
> Deep learning is a subset of Machine learning and refers to a class of
> algorithms that use a hierarchical approach with non-linearities to
> discover and learn representations within data. Deep Learning has recently
> become very popular due to its applicability and advancement of domains
> such as Computer Vision, Speech Recognition, Natural Language Understanding
> and Recommender Systems. With pervasive and cost effective cloud computing,
> large labeled datasets and continued algorithmic innovation, Deep Learning
> has become the one of the most popular classes of algorithms for machine
> learning practitioners in recent years.
> 
> == Rational ==
> 
> The adoption of deep learning is quickly expanding from initial deep domain
> experts rooted in academia to data scientists and developers working to
> deploy intelligent services and products. Deep learning however has many
> challenges.  These include model training time (which can take days to
> weeks), programmability (not everyone writes Python or C++ and like
> symbolic programming) and balancing production readiness (support for
> things like failover) with development flexibility (ability to program
> different ways, support for new operators and model types) and speed of
> execution (fast and scalable model training).  Other frameworks excel on
> some but not all of these aspects.
> 
> 
> == Initial Goals ==
> 
> MXNet is a fairly established project on GitHub with its first code
> contribution in April 2015 and roughly 200 contributors. It is used by
> several large companies and some of the top research institutions on the
> planet. Initial goals would be the following:
> 
>  1. Move the existing codebase(s) to Apache
>  1. Integrate with the Apache development process/sign CLAs
>  1. Ensure all dependencies are compliant with Apache License version 2.0
>  1. Incremental development and releases per Apache guidelines
>  1. Establish engineering discipline and a predictable release cadence of
> high quality releases
>  1. Expand the community beyond the current base of expert level users
>  1. Improve usability and the overall developer/user experience
>  1. Add additional functionality to address newer problem types and
> algorithms
> 
> 
> == Current Status ==
> 
> === Meritocracy ===
> 
> The MXNet project already operates on meritocratic principles. Today, MXNet
> has developers worldwide and has accepted multiple major patches from a
> diverse set of contributors within both industry and academia. We would
> like to follow ASF meritocratic principles to encourage more developers to
> contribute in this project. We know that only active and committed
> developers from a diverse set of backgrounds can make MXNet a successful
> project.  We are also improving the documentation and code to help new
> developers get started quickly.
> 
> === Community ===
> 
> Acceptance into the Apache foundation would bolster the growing user and
> developer community around MXNet. That community includes around 200
> contributors from academia and industry. The core developers of our project
> are listed in our contributors below and are also represented by logos on
> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
> University of Washington and Wolfram.
> 
> === Core Developers ===
> 
> (with GitHub logins)
> 
>  * Tianqi Chen (@tqchen)
>  * Mu Li (@mli)
>  * Junyuan Xie (@piiswrong)
>  * Bing Xu (@antinucleon)
>  * Chiyuan Zhang (@pluskid)
>  * Minjie Wang (@jermainewang)
>  * Naiyan Wang (@winstywang)
>  * Yizhi Liu (@javelinjs)
>  * Tong He (@hetong007)
>  * Qiang Kou (@thirdwing)
>  * Xingjian Shi (@sxjscience)
> 
> === Alignment ===
> 
> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
> and Mahout, each of which targets a different application domain. MXNet,
> being a distributed platform for large-scale deep learning, focuses on
> another important domain for which there still lacks a scalable,
> programmable, flexible and super fast open-source platform. The recent
> success of deep learning models especially for vision and speech
> recognition tasks has generated interests in both applying existing deep
> learning models and in developing new ones. Thus, an open-source platform
> for deep learning backed by some of the top industry and academic players
> will be able to attract a large community of users and developers. MXNet is
> a complex system needing many iterations of design, implementation and
> testing. Apache's collaboration framework which encourages active
> contribution from developers will inevitably help improve the quality of
> the system, as shown in the success of Hadoop, Spark, etc. Equally
> important is the community of users which helps identify real-life
> applications of deep learning, and helps to evaluate the system's
> performance and ease-of-use. We hope to leverage ASF for coordinating and
> promoting both communities, and in return benefit the communities with
> another useful tool.
> 
> == Known Risks ==
> 
> === Orphaned products ===
> 
> Given the current level of investment in MXNet and the stakeholders using
> it - the risk of the project being abandoned is minimal. Amazon, for
> example, is in active development to use MXNet in many of its services and
> many large corporations use it in their production applications.
> 
> === Inexperience with Open Source ===
> 
> MXNet has existed as a healthy open source project for more than a year.
> During that time, the project has attracted 200+ contributors.
> 
> === Homogenous Developers ===
> 
> The initial list of committers and contributors includes developers from
> several institutions and industry participants (see above).
> 
> === Reliance on Salaried Developers ===
> 
> Like most open source projects, MXNet receives a substantial support from
> salaried developers. A large fraction of MXNet development is supported by
> graduate students at various universities in the course of research degrees
> - this is more a \u201cvolunteer\u201d relationship, since in most cases students
> contribute vastly more than is necessary to immediately support research.
> In addition, those working from within corporations are devoting
> significant time and effort in the project - and these come from several
> organizations.
> 
> === A Excessive Fascination with the Apache Brand ===
> 
> We choose Apache not for publicity. We have two purposes. First, we hope
> that Apache's known best-practices for managing a mature open source
> project can help guide us.  For example, we are feeling the growing pains
> of a successful open source project as we attempt a major refactor of the
> internals while customers are using the system in production. We seek
> guidance in communicating breaking API changes and version revisions.
> Also, as our involvement from major corporations increases, we want to
> assure our users that MXNet will stay open and not favor any particular
> platform or environment. These are some examples of the know-how and
> discipline we're hoping Apache can bring to our project.
> 
> Second, we want to leverage Apache's reputation to recruit more developers
> to create a diverse community.
> 
> === Relationship with Other Apache Products ===
> 
> Apache Mahout and Apache Spark's MLlib are general machine learning
> systems. Deep learning algorithms can thus be implemented on these two
> platforms as well. However, in practice, the overlap will be minimal.  Deep
> learning is so computationally intensive that it often requires specialized
> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
> of GPU hardware is complex because the hardware is so fast that the
> supporting systems around it must be carefully optimized to keep the GPU
> cores busy.  Extending this capability to distributed multi-GPU and
> multi-host environments requires great care.  This is a critical
> differentiator between MXNet and existing Apache machine learning systems.
> 
> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
> This is the fundamental difference to MXNet who follows the parameter
> server framework. MXNet can run synchronously or asynchronously. In
> addition, MXNet has optimizations for training a wide range of deep
> learning models using a variety of approaches (e.g., model parallelism and
> data parallelism) which makes MXNet much more efficient (near-linear
> speedup on state of the art models). MXNet also supports both imperative
> and symbolic approaches providing ease of programming for deep learning
> algorithms.
> 
> Other Apache projects that are potentially complimentary:
> 
> Apache Arrow - read data in Apache Arrow\u2018s internal format from MXNet, that
> would allow users to run ETL/preprocessing in Spark, save the results in
> Arrow\u2019s format and then run DL algorithms on it.
> 
> Apache Singa - MXNet and Singa are both deep learning projects, and can
> benefit from a larger deep learning community at Apache.
> 
> == Documentation ==
> 
> Documentation has recently migrated to http://mxnet.io.  We continue to
> refine and improve the documentation.
> 
> == Initial Source ==
> 
> We currently use Github to maintain our source code,
> https://github.com/MXNet
> 
> == Source and Intellectual Property Submission Plan ==
> 
> MXNet Code is available under Apache License, Version 2.0. We will work
> with the committers to get CLAs signed and review previous contributions.
> 
> == External Dependencies ==
> 
>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> lib-zeromq), TBB
>  * required for GPU usage: cudnn, cuda
>  * required for python usage: Python 2/3
>  * required for R module: R, Rcpp (GPLv2 licensing)
>  * optional for image preparation and preprocessing: opencv
>  * optional dependencies for additional features: torch7, numba, cython (in
> NNVM branch)
> 
> Rcpt and lib-zeromq are expected to be licensing discussions.
> 
> == Cryptography ==
> 
> Not Applicable
> 
> == Required Resources ==
> 
> === Mailing Lists ===
> 
> There is currently no mailing list.
> 
> === Issue Tracking ===
> 
> Currently uses GitHub to track issues. Would like to continue to do so.
> 
> == Committers and Affiliations ==
> 
>  * Tianqi Chen (UW)
>  * Mu Li (AWS)
>  * Junyuan Xie (AWS)
>  * Bing Xu (Apple)
>  * Chiyuan Zhang (MIT)
>  * Minjie Wang (UYU)
>  * Naiyan Wang (Tusimple)
>  * Yizhi Liu (Mediav)
>  * Tong He (Simon Fraser University)
>  * Qiang Kou (Indiana U)
>  * Xingjian Shi (HKUST)
> 
> == Sponsors ==
> 
> === Champion ===
> 
> Henri Yandell (bayard at apache.org)
> 
> === Nominated Mentors ===
> 
> Sebastian Schelter (ssc@apache.org)
> 
> 
> === Sponsoring Entity ===
> 
> We are requesting the Incubator to sponsor this project.
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[sandeep krishnamurthy <sa...@gmail.com>]

On 2017-01-05 21:12 (-0800), Henri Yandell <ba...@apache.org> wrote: 
> Hello Incubator,
> 
> I'd like to propose a new incubator Apache MXNet podling.
> 
> The existing MXNet project (http://mxnet.io - 1.5 years old, 15 committers,
> 200 contributors) is very interested in joining Apache. MXNet is an
> open-source deep learning framework that allows you to define, train, and
> deploy deep neural networks on a wide array of devices, from cloud
> infrastructure to mobile devices.
> 
> The wiki proposal page is located here:
> 
>   https://wiki.apache.org/incubator/MXNetProposal
> 
> I've included the text below in case anyone wants to focus on parts of it
> in a reply.
> 
> Looking forward to your thoughts, and for lots of interested Apache members
> to volunteer to mentor the project in addition to Sebastian and myself.
> 
> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!
> 
> Thanks,
> 
> Hen
> On behalf of the MXNet project
> 
> ---------
> 
> = MXNet: Apache Incubator Proposal =
> 
> == Abstract ==
> 
> MXNet is a Flexible and Efficient Library for Deep Learning
> 
> == Proposal ==
> 
> MXNet is an open-source deep learning framework that allows you to define,
> train, and deploy deep neural networks on a wide array of devices, from
> cloud infrastructure to mobile devices. It is highly scalable, allowing for
> fast model training, and supports a flexible programming model and multiple
> languages. MXNet allows you to mix symbolic and imperative programming
> flavors to maximize both efficiency and productivity. MXNet is built on a
> dynamic dependency scheduler that automatically parallelizes both symbolic
> and imperative operations on the fly. A graph optimization layer on top of
> that makes symbolic execution fast and memory efficient. The MXNet library
> is portable and lightweight, and it scales to multiple GPUs and multiple
> machines.
> 
> == Background ==
> 
> Deep learning is a subset of Machine learning and refers to a class of
> algorithms that use a hierarchical approach with non-linearities to
> discover and learn representations within data. Deep Learning has recently
> become very popular due to its applicability and advancement of domains
> such as Computer Vision, Speech Recognition, Natural Language Understanding
> and Recommender Systems. With pervasive and cost effective cloud computing,
> large labeled datasets and continued algorithmic innovation, Deep Learning
> has become the one of the most popular classes of algorithms for machine
> learning practitioners in recent years.
> 
> == Rational ==
> 
> The adoption of deep learning is quickly expanding from initial deep domain
> experts rooted in academia to data scientists and developers working to
> deploy intelligent services and products. Deep learning however has many
> challenges.  These include model training time (which can take days to
> weeks), programmability (not everyone writes Python or C++ and like
> symbolic programming) and balancing production readiness (support for
> things like failover) with development flexibility (ability to program
> different ways, support for new operators and model types) and speed of
> execution (fast and scalable model training).  Other frameworks excel on
> some but not all of these aspects.
> 
> 
> == Initial Goals ==
> 
> MXNet is a fairly established project on GitHub with its first code
> contribution in April 2015 and roughly 200 contributors. It is used by
> several large companies and some of the top research institutions on the
> planet. Initial goals would be the following:
> 
>  1. Move the existing codebase(s) to Apache
>  1. Integrate with the Apache development process/sign CLAs
>  1. Ensure all dependencies are compliant with Apache License version 2.0
>  1. Incremental development and releases per Apache guidelines
>  1. Establish engineering discipline and a predictable release cadence of
> high quality releases
>  1. Expand the community beyond the current base of expert level users
>  1. Improve usability and the overall developer/user experience
>  1. Add additional functionality to address newer problem types and
> algorithms
> 
> 
> == Current Status ==
> 
> === Meritocracy ===
> 
> The MXNet project already operates on meritocratic principles. Today, MXNet
> has developers worldwide and has accepted multiple major patches from a
> diverse set of contributors within both industry and academia. We would
> like to follow ASF meritocratic principles to encourage more developers to
> contribute in this project. We know that only active and committed
> developers from a diverse set of backgrounds can make MXNet a successful
> project.  We are also improving the documentation and code to help new
> developers get started quickly.
> 
> === Community ===
> 
> Acceptance into the Apache foundation would bolster the growing user and
> developer community around MXNet. That community includes around 200
> contributors from academia and industry. The core developers of our project
> are listed in our contributors below and are also represented by logos on
> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
> University of Washington and Wolfram.
> 
> === Core Developers ===
> 
> (with GitHub logins)
> 
>  * Tianqi Chen (@tqchen)
>  * Mu Li (@mli)
>  * Junyuan Xie (@piiswrong)
>  * Bing Xu (@antinucleon)
>  * Chiyuan Zhang (@pluskid)
>  * Minjie Wang (@jermainewang)
>  * Naiyan Wang (@winstywang)
>  * Yizhi Liu (@javelinjs)
>  * Tong He (@hetong007)
>  * Qiang Kou (@thirdwing)
>  * Xingjian Shi (@sxjscience)
> 
> === Alignment ===
> 
> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
> and Mahout, each of which targets a different application domain. MXNet,
> being a distributed platform for large-scale deep learning, focuses on
> another important domain for which there still lacks a scalable,
> programmable, flexible and super fast open-source platform. The recent
> success of deep learning models especially for vision and speech
> recognition tasks has generated interests in both applying existing deep
> learning models and in developing new ones. Thus, an open-source platform
> for deep learning backed by some of the top industry and academic players
> will be able to attract a large community of users and developers. MXNet is
> a complex system needing many iterations of design, implementation and
> testing. Apache's collaboration framework which encourages active
> contribution from developers will inevitably help improve the quality of
> the system, as shown in the success of Hadoop, Spark, etc. Equally
> important is the community of users which helps identify real-life
> applications of deep learning, and helps to evaluate the system's
> performance and ease-of-use. We hope to leverage ASF for coordinating and
> promoting both communities, and in return benefit the communities with
> another useful tool.
> 
> == Known Risks ==
> 
> === Orphaned products ===
> 
> Given the current level of investment in MXNet and the stakeholders using
> it - the risk of the project being abandoned is minimal. Amazon, for
> example, is in active development to use MXNet in many of its services and
> many large corporations use it in their production applications.
> 
> === Inexperience with Open Source ===
> 
> MXNet has existed as a healthy open source project for more than a year.
> During that time, the project has attracted 200+ contributors.
> 
> === Homogenous Developers ===
> 
> The initial list of committers and contributors includes developers from
> several institutions and industry participants (see above).
> 
> === Reliance on Salaried Developers ===
> 
> Like most open source projects, MXNet receives a substantial support from
> salaried developers. A large fraction of MXNet development is supported by
> graduate students at various universities in the course of research degrees
> - this is more a \u201cvolunteer\u201d relationship, since in most cases students
> contribute vastly more than is necessary to immediately support research.
> In addition, those working from within corporations are devoting
> significant time and effort in the project - and these come from several
> organizations.
> 
> === A Excessive Fascination with the Apache Brand ===
> 
> We choose Apache not for publicity. We have two purposes. First, we hope
> that Apache's known best-practices for managing a mature open source
> project can help guide us.  For example, we are feeling the growing pains
> of a successful open source project as we attempt a major refactor of the
> internals while customers are using the system in production. We seek
> guidance in communicating breaking API changes and version revisions.
> Also, as our involvement from major corporations increases, we want to
> assure our users that MXNet will stay open and not favor any particular
> platform or environment. These are some examples of the know-how and
> discipline we're hoping Apache can bring to our project.
> 
> Second, we want to leverage Apache's reputation to recruit more developers
> to create a diverse community.
> 
> === Relationship with Other Apache Products ===
> 
> Apache Mahout and Apache Spark's MLlib are general machine learning
> systems. Deep learning algorithms can thus be implemented on these two
> platforms as well. However, in practice, the overlap will be minimal.  Deep
> learning is so computationally intensive that it often requires specialized
> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
> of GPU hardware is complex because the hardware is so fast that the
> supporting systems around it must be carefully optimized to keep the GPU
> cores busy.  Extending this capability to distributed multi-GPU and
> multi-host environments requires great care.  This is a critical
> differentiator between MXNet and existing Apache machine learning systems.
> 
> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
> This is the fundamental difference to MXNet who follows the parameter
> server framework. MXNet can run synchronously or asynchronously. In
> addition, MXNet has optimizations for training a wide range of deep
> learning models using a variety of approaches (e.g., model parallelism and
> data parallelism) which makes MXNet much more efficient (near-linear
> speedup on state of the art models). MXNet also supports both imperative
> and symbolic approaches providing ease of programming for deep learning
> algorithms.
> 
> Other Apache projects that are potentially complimentary:
> 
> Apache Arrow - read data in Apache Arrow\u2018s internal format from MXNet, that
> would allow users to run ETL/preprocessing in Spark, save the results in
> Arrow\u2019s format and then run DL algorithms on it.
> 
> Apache Singa - MXNet and Singa are both deep learning projects, and can
> benefit from a larger deep learning community at Apache.
> 
> == Documentation ==
> 
> Documentation has recently migrated to http://mxnet.io.  We continue to
> refine and improve the documentation.
> 
> == Initial Source ==
> 
> We currently use Github to maintain our source code,
> https://github.com/MXNet
> 
> == Source and Intellectual Property Submission Plan ==
> 
> MXNet Code is available under Apache License, Version 2.0. We will work
> with the committers to get CLAs signed and review previous contributions.
> 
> == External Dependencies ==
> 
>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> lib-zeromq), TBB
>  * required for GPU usage: cudnn, cuda
>  * required for python usage: Python 2/3
>  * required for R module: R, Rcpp (GPLv2 licensing)
>  * optional for image preparation and preprocessing: opencv
>  * optional dependencies for additional features: torch7, numba, cython (in
> NNVM branch)
> 
> Rcpt and lib-zeromq are expected to be licensing discussions.
> 
> == Cryptography ==
> 
> Not Applicable
> 
> == Required Resources ==
> 
> === Mailing Lists ===
> 
> There is currently no mailing list.
> 
> === Issue Tracking ===
> 
> Currently uses GitHub to track issues. Would like to continue to do so.
> 
> == Committers and Affiliations ==
> 
>  * Tianqi Chen (UW)
>  * Mu Li (AWS)
>  * Junyuan Xie (AWS)
>  * Bing Xu (Apple)
>  * Chiyuan Zhang (MIT)
>  * Minjie Wang (UYU)
>  * Naiyan Wang (Tusimple)
>  * Yizhi Liu (Mediav)
>  * Tong He (Simon Fraser University)
>  * Qiang Kou (Indiana U)
>  * Xingjian Shi (HKUST)
> 
> == Sponsors ==
> 
> === Champion ===
> 
> Henri Yandell (bayard at apache.org)
> 
> === Nominated Mentors ===
> 
> Sebastian Schelter (ssc@apache.org)
> 
> 
> === Sponsoring Entity ===
> 
> We are requesting the Incubator to sponsor this project.
> 

Hello,

I am committing actively for all mxnet.io documentation changes. I am working with Mu at work (Amazon) and would like to be involved more with the project. Please add me as a committer.

Github ID: sandeep-krishnamurthy
Gmail: sandeep.krishna98@gmail.com

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[sandeep krishnamurthy <sa...@gmail.com>]

On 2017-01-05 21:12 (-0800), Henri Yandell <ba...@apache.org> wrote: 
> Hello Incubator,
> 
> I'd like to propose a new incubator Apache MXNet podling.
> 
> The existing MXNet project (http://mxnet.io - 1.5 years old, 15 committers,
> 200 contributors) is very interested in joining Apache. MXNet is an
> open-source deep learning framework that allows you to define, train, and
> deploy deep neural networks on a wide array of devices, from cloud
> infrastructure to mobile devices.
> 
> The wiki proposal page is located here:
> 
>   https://wiki.apache.org/incubator/MXNetProposal
> 
> I've included the text below in case anyone wants to focus on parts of it
> in a reply.
> 
> Looking forward to your thoughts, and for lots of interested Apache members
> to volunteer to mentor the project in addition to Sebastian and myself.
> 
> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!
> 
> Thanks,
> 
> Hen
> On behalf of the MXNet project
> 
> ---------
> 
> = MXNet: Apache Incubator Proposal =
> 
> == Abstract ==
> 
> MXNet is a Flexible and Efficient Library for Deep Learning
> 
> == Proposal ==
> 
> MXNet is an open-source deep learning framework that allows you to define,
> train, and deploy deep neural networks on a wide array of devices, from
> cloud infrastructure to mobile devices. It is highly scalable, allowing for
> fast model training, and supports a flexible programming model and multiple
> languages. MXNet allows you to mix symbolic and imperative programming
> flavors to maximize both efficiency and productivity. MXNet is built on a
> dynamic dependency scheduler that automatically parallelizes both symbolic
> and imperative operations on the fly. A graph optimization layer on top of
> that makes symbolic execution fast and memory efficient. The MXNet library
> is portable and lightweight, and it scales to multiple GPUs and multiple
> machines.
> 
> == Background ==
> 
> Deep learning is a subset of Machine learning and refers to a class of
> algorithms that use a hierarchical approach with non-linearities to
> discover and learn representations within data. Deep Learning has recently
> become very popular due to its applicability and advancement of domains
> such as Computer Vision, Speech Recognition, Natural Language Understanding
> and Recommender Systems. With pervasive and cost effective cloud computing,
> large labeled datasets and continued algorithmic innovation, Deep Learning
> has become the one of the most popular classes of algorithms for machine
> learning practitioners in recent years.
> 
> == Rational ==
> 
> The adoption of deep learning is quickly expanding from initial deep domain
> experts rooted in academia to data scientists and developers working to
> deploy intelligent services and products. Deep learning however has many
> challenges.  These include model training time (which can take days to
> weeks), programmability (not everyone writes Python or C++ and like
> symbolic programming) and balancing production readiness (support for
> things like failover) with development flexibility (ability to program
> different ways, support for new operators and model types) and speed of
> execution (fast and scalable model training).  Other frameworks excel on
> some but not all of these aspects.
> 
> 
> == Initial Goals ==
> 
> MXNet is a fairly established project on GitHub with its first code
> contribution in April 2015 and roughly 200 contributors. It is used by
> several large companies and some of the top research institutions on the
> planet. Initial goals would be the following:
> 
>  1. Move the existing codebase(s) to Apache
>  1. Integrate with the Apache development process/sign CLAs
>  1. Ensure all dependencies are compliant with Apache License version 2.0
>  1. Incremental development and releases per Apache guidelines
>  1. Establish engineering discipline and a predictable release cadence of
> high quality releases
>  1. Expand the community beyond the current base of expert level users
>  1. Improve usability and the overall developer/user experience
>  1. Add additional functionality to address newer problem types and
> algorithms
> 
> 
> == Current Status ==
> 
> === Meritocracy ===
> 
> The MXNet project already operates on meritocratic principles. Today, MXNet
> has developers worldwide and has accepted multiple major patches from a
> diverse set of contributors within both industry and academia. We would
> like to follow ASF meritocratic principles to encourage more developers to
> contribute in this project. We know that only active and committed
> developers from a diverse set of backgrounds can make MXNet a successful
> project.  We are also improving the documentation and code to help new
> developers get started quickly.
> 
> === Community ===
> 
> Acceptance into the Apache foundation would bolster the growing user and
> developer community around MXNet. That community includes around 200
> contributors from academia and industry. The core developers of our project
> are listed in our contributors below and are also represented by logos on
> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
> University of Washington and Wolfram.
> 
> === Core Developers ===
> 
> (with GitHub logins)
> 
>  * Tianqi Chen (@tqchen)
>  * Mu Li (@mli)
>  * Junyuan Xie (@piiswrong)
>  * Bing Xu (@antinucleon)
>  * Chiyuan Zhang (@pluskid)
>  * Minjie Wang (@jermainewang)
>  * Naiyan Wang (@winstywang)
>  * Yizhi Liu (@javelinjs)
>  * Tong He (@hetong007)
>  * Qiang Kou (@thirdwing)
>  * Xingjian Shi (@sxjscience)
> 
> === Alignment ===
> 
> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
> and Mahout, each of which targets a different application domain. MXNet,
> being a distributed platform for large-scale deep learning, focuses on
> another important domain for which there still lacks a scalable,
> programmable, flexible and super fast open-source platform. The recent
> success of deep learning models especially for vision and speech
> recognition tasks has generated interests in both applying existing deep
> learning models and in developing new ones. Thus, an open-source platform
> for deep learning backed by some of the top industry and academic players
> will be able to attract a large community of users and developers. MXNet is
> a complex system needing many iterations of design, implementation and
> testing. Apache's collaboration framework which encourages active
> contribution from developers will inevitably help improve the quality of
> the system, as shown in the success of Hadoop, Spark, etc. Equally
> important is the community of users which helps identify real-life
> applications of deep learning, and helps to evaluate the system's
> performance and ease-of-use. We hope to leverage ASF for coordinating and
> promoting both communities, and in return benefit the communities with
> another useful tool.
> 
> == Known Risks ==
> 
> === Orphaned products ===
> 
> Given the current level of investment in MXNet and the stakeholders using
> it - the risk of the project being abandoned is minimal. Amazon, for
> example, is in active development to use MXNet in many of its services and
> many large corporations use it in their production applications.
> 
> === Inexperience with Open Source ===
> 
> MXNet has existed as a healthy open source project for more than a year.
> During that time, the project has attracted 200+ contributors.
> 
> === Homogenous Developers ===
> 
> The initial list of committers and contributors includes developers from
> several institutions and industry participants (see above).
> 
> === Reliance on Salaried Developers ===
> 
> Like most open source projects, MXNet receives a substantial support from
> salaried developers. A large fraction of MXNet development is supported by
> graduate students at various universities in the course of research degrees
> - this is more a \u201cvolunteer\u201d relationship, since in most cases students
> contribute vastly more than is necessary to immediately support research.
> In addition, those working from within corporations are devoting
> significant time and effort in the project - and these come from several
> organizations.
> 
> === A Excessive Fascination with the Apache Brand ===
> 
> We choose Apache not for publicity. We have two purposes. First, we hope
> that Apache's known best-practices for managing a mature open source
> project can help guide us.  For example, we are feeling the growing pains
> of a successful open source project as we attempt a major refactor of the
> internals while customers are using the system in production. We seek
> guidance in communicating breaking API changes and version revisions.
> Also, as our involvement from major corporations increases, we want to
> assure our users that MXNet will stay open and not favor any particular
> platform or environment. These are some examples of the know-how and
> discipline we're hoping Apache can bring to our project.
> 
> Second, we want to leverage Apache's reputation to recruit more developers
> to create a diverse community.
> 
> === Relationship with Other Apache Products ===
> 
> Apache Mahout and Apache Spark's MLlib are general machine learning
> systems. Deep learning algorithms can thus be implemented on these two
> platforms as well. However, in practice, the overlap will be minimal.  Deep
> learning is so computationally intensive that it often requires specialized
> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
> of GPU hardware is complex because the hardware is so fast that the
> supporting systems around it must be carefully optimized to keep the GPU
> cores busy.  Extending this capability to distributed multi-GPU and
> multi-host environments requires great care.  This is a critical
> differentiator between MXNet and existing Apache machine learning systems.
> 
> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
> This is the fundamental difference to MXNet who follows the parameter
> server framework. MXNet can run synchronously or asynchronously. In
> addition, MXNet has optimizations for training a wide range of deep
> learning models using a variety of approaches (e.g., model parallelism and
> data parallelism) which makes MXNet much more efficient (near-linear
> speedup on state of the art models). MXNet also supports both imperative
> and symbolic approaches providing ease of programming for deep learning
> algorithms.
> 
> Other Apache projects that are potentially complimentary:
> 
> Apache Arrow - read data in Apache Arrow\u2018s internal format from MXNet, that
> would allow users to run ETL/preprocessing in Spark, save the results in
> Arrow\u2019s format and then run DL algorithms on it.
> 
> Apache Singa - MXNet and Singa are both deep learning projects, and can
> benefit from a larger deep learning community at Apache.
> 
> == Documentation ==
> 
> Documentation has recently migrated to http://mxnet.io.  We continue to
> refine and improve the documentation.
> 
> == Initial Source ==
> 
> We currently use Github to maintain our source code,
> https://github.com/MXNet
> 
> == Source and Intellectual Property Submission Plan ==
> 
> MXNet Code is available under Apache License, Version 2.0. We will work
> with the committers to get CLAs signed and review previous contributions.
> 
> == External Dependencies ==
> 
>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> lib-zeromq), TBB
>  * required for GPU usage: cudnn, cuda
>  * required for python usage: Python 2/3
>  * required for R module: R, Rcpp (GPLv2 licensing)
>  * optional for image preparation and preprocessing: opencv
>  * optional dependencies for additional features: torch7, numba, cython (in
> NNVM branch)
> 
> Rcpt and lib-zeromq are expected to be licensing discussions.
> 
> == Cryptography ==
> 
> Not Applicable
> 
> == Required Resources ==
> 
> === Mailing Lists ===
> 
> There is currently no mailing list.
> 
> === Issue Tracking ===
> 
> Currently uses GitHub to track issues. Would like to continue to do so.
> 
> == Committers and Affiliations ==
> 
>  * Tianqi Chen (UW)
>  * Mu Li (AWS)
>  * Junyuan Xie (AWS)
>  * Bing Xu (Apple)
>  * Chiyuan Zhang (MIT)
>  * Minjie Wang (UYU)
>  * Naiyan Wang (Tusimple)
>  * Yizhi Liu (Mediav)
>  * Tong He (Simon Fraser University)
>  * Qiang Kou (Indiana U)
>  * Xingjian Shi (HKUST)
> 
> == Sponsors ==
> 
> === Champion ===
> 
> Henri Yandell (bayard at apache.org)
> 
> === Nominated Mentors ===
> 
> Sebastian Schelter (ssc@apache.org)
> 
> 
> === Sponsoring Entity ===
> 
> We are requesting the Incubator to sponsor this project.
> 

==========================================

Hello,

Please sign me up as a committer - I've been working with Mu at work on MXNet (Amazon) and would love to get more involved in the project.

GitHub ID: sandeep-krishnamurthy
Gmail ID: sandeep.krishna98@gmail.com

Thanks,
Sandeep

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: Committer for MXNet.

[Henri Yandell <ba...@apache.org>]

Added (after confirming offline with Chris that his affiliation is AWS).

On Fri, Jan 6, 2017 at 10:59 AM, Chris Olivier <cj...@gmail.com>
wrote:

> I would like to volunteer as a committer for MXNet.
>
> -Chris Olivier
> cjolivier01@gmail.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: Committer for MXNet.

[Henri Yandell <ba...@apache.org>]

Thanks :)

https://www.linkedin.com/in/chris-olivier-81953223 I presume?

Part of the proposal is to identify folks' affiliations so any dependence
on a single paying employer is clear.

On Fri, Jan 6, 2017 at 10:59 AM, Chris Olivier <cj...@gmail.com>
wrote:

> I would like to volunteer as a committer for MXNet.
>
> -Chris Olivier
> cjolivier01@gmail.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Committer for MXNet.

[Chris Olivier <cj...@gmail.com>]

I would like to volunteer as a committer for MXNet.

-Chris Olivier
cjolivier01@gmail.com

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Henri Yandell <ba...@apache.org>]

On Fri, Jan 6, 2017 at 3:52 AM, John D. Ament <jo...@apache.org> wrote:

> There seem to be some discrepancies in the proposal vs what they currently
> have.  That and some comments in line.
>
> On Fri, Jan 6, 2017 at 12:12 AM Henri Yandell <ba...@apache.org> wrote:
>
> > Hello Incubator,
> >
> > I'd like to propose a new incubator Apache MXNet podling.
> >
> > The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> > committers,
> > 200 contributors) is very interested in joining Apache. MXNet is an
> > open-source deep learning framework that allows you to define, train, and
> > deploy deep neural networks on a wide array of devices, from cloud
> > infrastructure to mobile devices.
> >
>
> This seems to be a broad gap between committers and contributors.  Of the
> remaining 220 not included as committer, are they being considered?
>

Mu's contacting (and has been contacting) contributors to invite them; for
the proposal I went with the notion of the current active folk with write
access and having others introduce themselves. I suspect some are already
awaiting general@ moderators to pass their email through :)

>
> > The wiki proposal page is located here:
> >
> >   https://wiki.apache.org/incubator/MXNetProposal
> >
> >
> I've added it to the project proposals page.
>
>
*doh* Thank you :)


>
> > I've included the text below in case anyone wants to focus on parts of it
> > in a reply.
> >
> > Looking forward to your thoughts, and for lots of interested Apache
> members
> > to volunteer to mentor the project in addition to Sebastian and myself.
> >
> > Currently the list of committers is based on the current active coders,
> so
> > we're also very interested in hearing from anyone else who is interested
> in
> > working on the project, be they current or future contributor!
> >
> > Thanks,
> >
> > Hen
> > On behalf of the MXNet project
> >
> > ---------
> >
> > = MXNet: Apache Incubator Proposal =
> >
>
>
<snip>


> >
> > === Core Developers ===
> >
> > (with GitHub logins)
> >
> >  * Tianqi Chen (@tqchen)
> >  * Mu Li (@mli)
> >  * Junyuan Xie (@piiswrong)
> >  * Bing Xu (@antinucleon)
> >  * Chiyuan Zhang (@pluskid)
> >  * Minjie Wang (@jermainewang)
> >  * Naiyan Wang (@winstywang)
> >  * Yizhi Liu (@javelinjs)
> >  * Tong He (@hetong007)
> >  * Qiang Kou (@thirdwing)
> >  * Xingjian Shi (@sxjscience)
> >
> >
> AFAIK, we still expect email addresses, not github accounts.
>
>
The GitHubs were just as an FYI :)


>
> >
> > == Initial Source ==
> >
> > We currently use Github to maintain our source code,
> > https://github.com/MXNet
>
>
> This doesn't look right.  This github organization has a single repo that
> contains a mobile app that appears to been 4 years old.  The website points
> to https://github.com/dmlc/mxnet which looks more correct.  Which is it?
> I'll also note that if it is the latter, there are git submodules in the
> codebase.  Please include references to those modules as well.  Please also
> include the website source code, if available.
>

*DOH* thank you :)

There will be more than dmlc/mxnet as some of the other dmlc repos relate
to mxnet too.


>
> I'll point out that since the github name is taken, it may be cause to say
> that MXNet isn't a viable name.  But that can be worked out later.
>

Yup. I did a preliminary name search of the various TESSA type places and
didn't see any blocking concerns.


>
> >
> > == External Dependencies ==
> >
> >  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> > (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> > lib-zeromq), TBB
> >  * required for GPU usage: cudnn, cuda
> >  * required for python usage: Python 2/3
> >  * required for R module: R, Rcpp (GPLv2 licensing)
> >  * optional for image preparation and preprocessing: opencv
> >  * optional dependencies for additional features: torch7, numba, cython
> (in
> > NNVM branch)
> >
>
> Before we go any further, I think we need to address the incompatible
> licenses.  Knowing that R is a core component of MXNet, how will you
> replace it?
>

I don't expect R itself to be an issue. Rcpp I expect to be an issue and
there are options here - either hosting the R module separately or
switching to using Rcpp11 (MIT license).

ZeroMQ is perhaps a larger issue given it's a transitive dependency of the
core codebase. That community has a licensing exception in addition to its
LGPL which can be discussed, it has expressed a desire to move to MPL and
is so prevalent in the ecosystem that I feel it's something to be figured
out rather than avoided. Solutions would involve either approval for the
exception text, rewriting to not use ps-lite, or rewriting ps-lite to not
use zmq (given ps-lite is a dmlc project).


> >
> > == Committers and Affiliations ==
> >
> >  * Tianqi Chen (UW)
> >  * Mu Li (AWS)
> >  * Junyuan Xie (AWS)
> >  * Bing Xu (Apple)
> >  * Chiyuan Zhang (MIT)
> >  * Minjie Wang (UYU)
> >  * Naiyan Wang (Tusimple)
> >  * Yizhi Liu (Mediav)
> >  * Tong He (Simon Fraser University)
> >  * Qiang Kou (Indiana U)
> >  * Xingjian Shi (HKUST)
> >
> >
> This seems to go against the notion of reliance on salaried developers.
> You're in fact not reliant on salaries developers.
>
>
Agreed. I think a lot of the developers are paid to work on mxnet, but it's
as mxnet committers (i.e. they got into mxnet in academia) rather than
their employer directing them to mxnet. I expect more of the latter to
happen, but the core of the project are committed to the project.

Thanks for all the feedback.

Hen

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

["John D. Ament" <jo...@apache.org>]

There seem to be some discrepancies in the proposal vs what they currently
have.  That and some comments in line.

On Fri, Jan 6, 2017 at 12:12 AM Henri Yandell <ba...@apache.org> wrote:

> Hello Incubator,
>
> I'd like to propose a new incubator Apache MXNet podling.
>
> The existing MXNet project (http://mxnet.io - 1.5 years old, 15
> committers,
> 200 contributors) is very interested in joining Apache. MXNet is an
> open-source deep learning framework that allows you to define, train, and
> deploy deep neural networks on a wide array of devices, from cloud
> infrastructure to mobile devices.
>

This seems to be a broad gap between committers and contributors.  Of the
remaining 220 not included as committer, are they being considered?


>
> The wiki proposal page is located here:
>
>   https://wiki.apache.org/incubator/MXNetProposal
>
>
I've added it to the project proposals page.


> I've included the text below in case anyone wants to focus on parts of it
> in a reply.
>
> Looking forward to your thoughts, and for lots of interested Apache members
> to volunteer to mentor the project in addition to Sebastian and myself.
>
> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!
>
> Thanks,
>
> Hen
> On behalf of the MXNet project
>
> ---------
>
> = MXNet: Apache Incubator Proposal =
>
> == Abstract ==
>
> MXNet is a Flexible and Efficient Library for Deep Learning
>
> == Proposal ==
>
> MXNet is an open-source deep learning framework that allows you to define,
> train, and deploy deep neural networks on a wide array of devices, from
> cloud infrastructure to mobile devices. It is highly scalable, allowing for
> fast model training, and supports a flexible programming model and multiple
> languages. MXNet allows you to mix symbolic and imperative programming
> flavors to maximize both efficiency and productivity. MXNet is built on a
> dynamic dependency scheduler that automatically parallelizes both symbolic
> and imperative operations on the fly. A graph optimization layer on top of
> that makes symbolic execution fast and memory efficient. The MXNet library
> is portable and lightweight, and it scales to multiple GPUs and multiple
> machines.
>
> == Background ==
>
> Deep learning is a subset of Machine learning and refers to a class of
> algorithms that use a hierarchical approach with non-linearities to
> discover and learn representations within data. Deep Learning has recently
> become very popular due to its applicability and advancement of domains
> such as Computer Vision, Speech Recognition, Natural Language Understanding
> and Recommender Systems. With pervasive and cost effective cloud computing,
> large labeled datasets and continued algorithmic innovation, Deep Learning
> has become the one of the most popular classes of algorithms for machine
> learning practitioners in recent years.
>
> == Rational ==
>
> The adoption of deep learning is quickly expanding from initial deep domain
> experts rooted in academia to data scientists and developers working to
> deploy intelligent services and products. Deep learning however has many
> challenges.  These include model training time (which can take days to
> weeks), programmability (not everyone writes Python or C++ and like
> symbolic programming) and balancing production readiness (support for
> things like failover) with development flexibility (ability to program
> different ways, support for new operators and model types) and speed of
> execution (fast and scalable model training).  Other frameworks excel on
> some but not all of these aspects.
>
>
> == Initial Goals ==
>
> MXNet is a fairly established project on GitHub with its first code
> contribution in April 2015 and roughly 200 contributors. It is used by
> several large companies and some of the top research institutions on the
> planet. Initial goals would be the following:
>
>  1. Move the existing codebase(s) to Apache
>  1. Integrate with the Apache development process/sign CLAs
>  1. Ensure all dependencies are compliant with Apache License version 2.0
>  1. Incremental development and releases per Apache guidelines
>  1. Establish engineering discipline and a predictable release cadence of
> high quality releases
>  1. Expand the community beyond the current base of expert level users
>  1. Improve usability and the overall developer/user experience
>  1. Add additional functionality to address newer problem types and
> algorithms
>
>
> == Current Status ==
>
> === Meritocracy ===
>
> The MXNet project already operates on meritocratic principles. Today, MXNet
> has developers worldwide and has accepted multiple major patches from a
> diverse set of contributors within both industry and academia. We would
> like to follow ASF meritocratic principles to encourage more developers to
> contribute in this project. We know that only active and committed
> developers from a diverse set of backgrounds can make MXNet a successful
> project.  We are also improving the documentation and code to help new
> developers get started quickly.
>
> === Community ===
>
> Acceptance into the Apache foundation would bolster the growing user and
> developer community around MXNet. That community includes around 200
> contributors from academia and industry. The core developers of our project
> are listed in our contributors below and are also represented by logos on
> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
> University of Washington and Wolfram.
>
> === Core Developers ===
>
> (with GitHub logins)
>
>  * Tianqi Chen (@tqchen)
>  * Mu Li (@mli)
>  * Junyuan Xie (@piiswrong)
>  * Bing Xu (@antinucleon)
>  * Chiyuan Zhang (@pluskid)
>  * Minjie Wang (@jermainewang)
>  * Naiyan Wang (@winstywang)
>  * Yizhi Liu (@javelinjs)
>  * Tong He (@hetong007)
>  * Qiang Kou (@thirdwing)
>  * Xingjian Shi (@sxjscience)
>
>
AFAIK, we still expect email addresses, not github accounts.


> === Alignment ===
>
> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
> and Mahout, each of which targets a different application domain. MXNet,
> being a distributed platform for large-scale deep learning, focuses on
> another important domain for which there still lacks a scalable,
> programmable, flexible and super fast open-source platform. The recent
> success of deep learning models especially for vision and speech
> recognition tasks has generated interests in both applying existing deep
> learning models and in developing new ones. Thus, an open-source platform
> for deep learning backed by some of the top industry and academic players
> will be able to attract a large community of users and developers. MXNet is
> a complex system needing many iterations of design, implementation and
> testing. Apache's collaboration framework which encourages active
> contribution from developers will inevitably help improve the quality of
> the system, as shown in the success of Hadoop, Spark, etc. Equally
> important is the community of users which helps identify real-life
> applications of deep learning, and helps to evaluate the system's
> performance and ease-of-use. We hope to leverage ASF for coordinating and
> promoting both communities, and in return benefit the communities with
> another useful tool.
>
> == Known Risks ==
>
> === Orphaned products ===
>
> Given the current level of investment in MXNet and the stakeholders using
> it - the risk of the project being abandoned is minimal. Amazon, for
> example, is in active development to use MXNet in many of its services and
> many large corporations use it in their production applications.
>
> === Inexperience with Open Source ===
>
> MXNet has existed as a healthy open source project for more than a year.
> During that time, the project has attracted 200+ contributors.
>
> === Homogenous Developers ===
>
> The initial list of committers and contributors includes developers from
> several institutions and industry participants (see above).
>
> === Reliance on Salaried Developers ===
>
> Like most open source projects, MXNet receives a substantial support from
> salaried developers. A large fraction of MXNet development is supported by
> graduate students at various universities in the course of research degrees
> - this is more a “volunteer” relationship, since in most cases students
> contribute vastly more than is necessary to immediately support research.
> In addition, those working from within corporations are devoting
> significant time and effort in the project - and these come from several
> organizations.
>
> === A Excessive Fascination with the Apache Brand ===
>
> We choose Apache not for publicity. We have two purposes. First, we hope
> that Apache's known best-practices for managing a mature open source
> project can help guide us.  For example, we are feeling the growing pains
> of a successful open source project as we attempt a major refactor of the
> internals while customers are using the system in production. We seek
> guidance in communicating breaking API changes and version revisions.
> Also, as our involvement from major corporations increases, we want to
> assure our users that MXNet will stay open and not favor any particular
> platform or environment. These are some examples of the know-how and
> discipline we're hoping Apache can bring to our project.
>
> Second, we want to leverage Apache's reputation to recruit more developers
> to create a diverse community.
>
> === Relationship with Other Apache Products ===
>
> Apache Mahout and Apache Spark's MLlib are general machine learning
> systems. Deep learning algorithms can thus be implemented on these two
> platforms as well. However, in practice, the overlap will be minimal.  Deep
> learning is so computationally intensive that it often requires specialized
> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
> of GPU hardware is complex because the hardware is so fast that the
> supporting systems around it must be carefully optimized to keep the GPU
> cores busy.  Extending this capability to distributed multi-GPU and
> multi-host environments requires great care.  This is a critical
> differentiator between MXNet and existing Apache machine learning systems.
>
> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
> This is the fundamental difference to MXNet who follows the parameter
> server framework. MXNet can run synchronously or asynchronously. In
> addition, MXNet has optimizations for training a wide range of deep
> learning models using a variety of approaches (e.g., model parallelism and
> data parallelism) which makes MXNet much more efficient (near-linear
> speedup on state of the art models). MXNet also supports both imperative
> and symbolic approaches providing ease of programming for deep learning
> algorithms.
>
> Other Apache projects that are potentially complimentary:
>
> Apache Arrow - read data in Apache Arrow‘s internal format from MXNet, that
> would allow users to run ETL/preprocessing in Spark, save the results in
> Arrow’s format and then run DL algorithms on it.
>
> Apache Singa - MXNet and Singa are both deep learning projects, and can
> benefit from a larger deep learning community at Apache.
>
> == Documentation ==
>
> Documentation has recently migrated to http://mxnet.io.  We continue to
> refine and improve the documentation.
>
> == Initial Source ==
>
> We currently use Github to maintain our source code,
> https://github.com/MXNet


This doesn't look right.  This github organization has a single repo that
contains a mobile app that appears to been 4 years old.  The website points
to https://github.com/dmlc/mxnet which looks more correct.  Which is it?
I'll also note that if it is the latter, there are git submodules in the
codebase.  Please include references to those modules as well.  Please also
include the website source code, if available.

I'll point out that since the github name is taken, it may be cause to say
that MXNet isn't a viable name.  But that can be worked out later.


>
>
> == Source and Intellectual Property Submission Plan ==
>
> MXNet Code is available under Apache License, Version 2.0. We will work
> with the committers to get CLAs signed and review previous contributions.
>
> == External Dependencies ==
>
>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> lib-zeromq), TBB
>  * required for GPU usage: cudnn, cuda
>  * required for python usage: Python 2/3
>  * required for R module: R, Rcpp (GPLv2 licensing)
>  * optional for image preparation and preprocessing: opencv
>  * optional dependencies for additional features: torch7, numba, cython (in
> NNVM branch)
>

Before we go any further, I think we need to address the incompatible
licenses.  Knowing that R is a core component of MXNet, how will you
replace it?


>
> Rcpt and lib-zeromq are expected to be licensing discussions.
>
> == Cryptography ==
>
> Not Applicable
>
> == Required Resources ==
>
> === Mailing Lists ===
>
> There is currently no mailing list.
>
> === Issue Tracking ===
>
> Currently uses GitHub to track issues. Would like to continue to do so.
>
> == Committers and Affiliations ==
>
>  * Tianqi Chen (UW)
>  * Mu Li (AWS)
>  * Junyuan Xie (AWS)
>  * Bing Xu (Apple)
>  * Chiyuan Zhang (MIT)
>  * Minjie Wang (UYU)
>  * Naiyan Wang (Tusimple)
>  * Yizhi Liu (Mediav)
>  * Tong He (Simon Fraser University)
>  * Qiang Kou (Indiana U)
>  * Xingjian Shi (HKUST)
>
>
This seems to go against the notion of reliance on salaried developers.
You're in fact not reliant on salaries developers.


> == Sponsors ==
>
> === Champion ===
>
> Henri Yandell (bayard at apache.org)
>
> === Nominated Mentors ===
>
> Sebastian Schelter (ssc@apache.org)
>
>
> === Sponsoring Entity ===
>
> We are requesting the Incubator to sponsor this project.
>

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Naveen Swamy <mn...@gmail.com>]

Hello 

Please sign me up as a committer for MXNet - I've been working with Mu at work on MXNet (Amazon) and would love to get more involved in the project.

GitHub ID: nswamy

Thanks, Naveen 


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [DISCUSS] Proposing MXNet for the Apache Incubator

[Mu Li <mu...@gmail.com>]

Thanks, Henri.

The correct github link is https://github.com/dmlc/mxnet

On 2017-01-05 21:12 (-0800), Henri Yandell <ba...@apache.org> wrote: 
> Hello Incubator,
> 
> I'd like to propose a new incubator Apache MXNet podling.
> 
> The existing MXNet project (http://mxnet.io - 1.5 years old, 15 committers,
> 200 contributors) is very interested in joining Apache. MXNet is an
> open-source deep learning framework that allows you to define, train, and
> deploy deep neural networks on a wide array of devices, from cloud
> infrastructure to mobile devices.
> 
> The wiki proposal page is located here:
> 
>   https://wiki.apache.org/incubator/MXNetProposal
> 
> I've included the text below in case anyone wants to focus on parts of it
> in a reply.
> 
> Looking forward to your thoughts, and for lots of interested Apache members
> to volunteer to mentor the project in addition to Sebastian and myself.
> 
> Currently the list of committers is based on the current active coders, so
> we're also very interested in hearing from anyone else who is interested in
> working on the project, be they current or future contributor!
> 
> Thanks,
> 
> Hen
> On behalf of the MXNet project
> 
> ---------
> 
> = MXNet: Apache Incubator Proposal =
> 
> == Abstract ==
> 
> MXNet is a Flexible and Efficient Library for Deep Learning
> 
> == Proposal ==
> 
> MXNet is an open-source deep learning framework that allows you to define,
> train, and deploy deep neural networks on a wide array of devices, from
> cloud infrastructure to mobile devices. It is highly scalable, allowing for
> fast model training, and supports a flexible programming model and multiple
> languages. MXNet allows you to mix symbolic and imperative programming
> flavors to maximize both efficiency and productivity. MXNet is built on a
> dynamic dependency scheduler that automatically parallelizes both symbolic
> and imperative operations on the fly. A graph optimization layer on top of
> that makes symbolic execution fast and memory efficient. The MXNet library
> is portable and lightweight, and it scales to multiple GPUs and multiple
> machines.
> 
> == Background ==
> 
> Deep learning is a subset of Machine learning and refers to a class of
> algorithms that use a hierarchical approach with non-linearities to
> discover and learn representations within data. Deep Learning has recently
> become very popular due to its applicability and advancement of domains
> such as Computer Vision, Speech Recognition, Natural Language Understanding
> and Recommender Systems. With pervasive and cost effective cloud computing,
> large labeled datasets and continued algorithmic innovation, Deep Learning
> has become the one of the most popular classes of algorithms for machine
> learning practitioners in recent years.
> 
> == Rational ==
> 
> The adoption of deep learning is quickly expanding from initial deep domain
> experts rooted in academia to data scientists and developers working to
> deploy intelligent services and products. Deep learning however has many
> challenges.  These include model training time (which can take days to
> weeks), programmability (not everyone writes Python or C++ and like
> symbolic programming) and balancing production readiness (support for
> things like failover) with development flexibility (ability to program
> different ways, support for new operators and model types) and speed of
> execution (fast and scalable model training).  Other frameworks excel on
> some but not all of these aspects.
> 
> 
> == Initial Goals ==
> 
> MXNet is a fairly established project on GitHub with its first code
> contribution in April 2015 and roughly 200 contributors. It is used by
> several large companies and some of the top research institutions on the
> planet. Initial goals would be the following:
> 
>  1. Move the existing codebase(s) to Apache
>  1. Integrate with the Apache development process/sign CLAs
>  1. Ensure all dependencies are compliant with Apache License version 2.0
>  1. Incremental development and releases per Apache guidelines
>  1. Establish engineering discipline and a predictable release cadence of
> high quality releases
>  1. Expand the community beyond the current base of expert level users
>  1. Improve usability and the overall developer/user experience
>  1. Add additional functionality to address newer problem types and
> algorithms
> 
> 
> == Current Status ==
> 
> === Meritocracy ===
> 
> The MXNet project already operates on meritocratic principles. Today, MXNet
> has developers worldwide and has accepted multiple major patches from a
> diverse set of contributors within both industry and academia. We would
> like to follow ASF meritocratic principles to encourage more developers to
> contribute in this project. We know that only active and committed
> developers from a diverse set of backgrounds can make MXNet a successful
> project.  We are also improving the documentation and code to help new
> developers get started quickly.
> 
> === Community ===
> 
> Acceptance into the Apache foundation would bolster the growing user and
> developer community around MXNet. That community includes around 200
> contributors from academia and industry. The core developers of our project
> are listed in our contributors below and are also represented by logos on
> the mxnet.io site including Amazon, Baidu, Carnegie Mellon University,
> Turi, Intel, NYU, Nvidia, MIT, Microsoft, TuSimple, University of Alberta,
> University of Washington and Wolfram.
> 
> === Core Developers ===
> 
> (with GitHub logins)
> 
>  * Tianqi Chen (@tqchen)
>  * Mu Li (@mli)
>  * Junyuan Xie (@piiswrong)
>  * Bing Xu (@antinucleon)
>  * Chiyuan Zhang (@pluskid)
>  * Minjie Wang (@jermainewang)
>  * Naiyan Wang (@winstywang)
>  * Yizhi Liu (@javelinjs)
>  * Tong He (@hetong007)
>  * Qiang Kou (@thirdwing)
>  * Xingjian Shi (@sxjscience)
> 
> === Alignment ===
> 
> ASF is already the home of many distributed platforms, e.g., Hadoop, Spark
> and Mahout, each of which targets a different application domain. MXNet,
> being a distributed platform for large-scale deep learning, focuses on
> another important domain for which there still lacks a scalable,
> programmable, flexible and super fast open-source platform. The recent
> success of deep learning models especially for vision and speech
> recognition tasks has generated interests in both applying existing deep
> learning models and in developing new ones. Thus, an open-source platform
> for deep learning backed by some of the top industry and academic players
> will be able to attract a large community of users and developers. MXNet is
> a complex system needing many iterations of design, implementation and
> testing. Apache's collaboration framework which encourages active
> contribution from developers will inevitably help improve the quality of
> the system, as shown in the success of Hadoop, Spark, etc. Equally
> important is the community of users which helps identify real-life
> applications of deep learning, and helps to evaluate the system's
> performance and ease-of-use. We hope to leverage ASF for coordinating and
> promoting both communities, and in return benefit the communities with
> another useful tool.
> 
> == Known Risks ==
> 
> === Orphaned products ===
> 
> Given the current level of investment in MXNet and the stakeholders using
> it - the risk of the project being abandoned is minimal. Amazon, for
> example, is in active development to use MXNet in many of its services and
> many large corporations use it in their production applications.
> 
> === Inexperience with Open Source ===
> 
> MXNet has existed as a healthy open source project for more than a year.
> During that time, the project has attracted 200+ contributors.
> 
> === Homogenous Developers ===
> 
> The initial list of committers and contributors includes developers from
> several institutions and industry participants (see above).
> 
> === Reliance on Salaried Developers ===
> 
> Like most open source projects, MXNet receives a substantial support from
> salaried developers. A large fraction of MXNet development is supported by
> graduate students at various universities in the course of research degrees
> - this is more a \u201cvolunteer\u201d relationship, since in most cases students
> contribute vastly more than is necessary to immediately support research.
> In addition, those working from within corporations are devoting
> significant time and effort in the project - and these come from several
> organizations.
> 
> === A Excessive Fascination with the Apache Brand ===
> 
> We choose Apache not for publicity. We have two purposes. First, we hope
> that Apache's known best-practices for managing a mature open source
> project can help guide us.  For example, we are feeling the growing pains
> of a successful open source project as we attempt a major refactor of the
> internals while customers are using the system in production. We seek
> guidance in communicating breaking API changes and version revisions.
> Also, as our involvement from major corporations increases, we want to
> assure our users that MXNet will stay open and not favor any particular
> platform or environment. These are some examples of the know-how and
> discipline we're hoping Apache can bring to our project.
> 
> Second, we want to leverage Apache's reputation to recruit more developers
> to create a diverse community.
> 
> === Relationship with Other Apache Products ===
> 
> Apache Mahout and Apache Spark's MLlib are general machine learning
> systems. Deep learning algorithms can thus be implemented on these two
> platforms as well. However, in practice, the overlap will be minimal.  Deep
> learning is so computationally intensive that it often requires specialized
> GPU hardware to accomplish tasks of meaningful size.  Making efficient use
> of GPU hardware is complex because the hardware is so fast that the
> supporting systems around it must be carefully optimized to keep the GPU
> cores busy.  Extending this capability to distributed multi-GPU and
> multi-host environments requires great care.  This is a critical
> differentiator between MXNet and existing Apache machine learning systems.
> 
> Mahout and Spark ML-LIB follow models where their nodes run synchronously.
> This is the fundamental difference to MXNet who follows the parameter
> server framework. MXNet can run synchronously or asynchronously. In
> addition, MXNet has optimizations for training a wide range of deep
> learning models using a variety of approaches (e.g., model parallelism and
> data parallelism) which makes MXNet much more efficient (near-linear
> speedup on state of the art models). MXNet also supports both imperative
> and symbolic approaches providing ease of programming for deep learning
> algorithms.
> 
> Other Apache projects that are potentially complimentary:
> 
> Apache Arrow - read data in Apache Arrow\u2018s internal format from MXNet, that
> would allow users to run ETL/preprocessing in Spark, save the results in
> Arrow\u2019s format and then run DL algorithms on it.
> 
> Apache Singa - MXNet and Singa are both deep learning projects, and can
> benefit from a larger deep learning community at Apache.
> 
> == Documentation ==
> 
> Documentation has recently migrated to http://mxnet.io.  We continue to
> refine and improve the documentation.
> 
> == Initial Source ==
> 
> We currently use Github to maintain our source code,
> https://github.com/MXNet
> 
> == Source and Intellectual Property Submission Plan ==
> 
> MXNet Code is available under Apache License, Version 2.0. We will work
> with the committers to get CLAs signed and review previous contributions.
> 
> == External Dependencies ==
> 
>  * required by the core code base: GCC or CLOM, Clang, any BLAS library
> (ATLAS, OpenBLAS, MKL), dmlc-core, mshadow, ps-lite (which requires
> lib-zeromq), TBB
>  * required for GPU usage: cudnn, cuda
>  * required for python usage: Python 2/3
>  * required for R module: R, Rcpp (GPLv2 licensing)
>  * optional for image preparation and preprocessing: opencv
>  * optional dependencies for additional features: torch7, numba, cython (in
> NNVM branch)
> 
> Rcpt and lib-zeromq are expected to be licensing discussions.
> 
> == Cryptography ==
> 
> Not Applicable
> 
> == Required Resources ==
> 
> === Mailing Lists ===
> 
> There is currently no mailing list.
> 
> === Issue Tracking ===
> 
> Currently uses GitHub to track issues. Would like to continue to do so.
> 
> == Committers and Affiliations ==
> 
>  * Tianqi Chen (UW)
>  * Mu Li (AWS)
>  * Junyuan Xie (AWS)
>  * Bing Xu (Apple)
>  * Chiyuan Zhang (MIT)
>  * Minjie Wang (UYU)
>  * Naiyan Wang (Tusimple)
>  * Yizhi Liu (Mediav)
>  * Tong He (Simon Fraser University)
>  * Qiang Kou (Indiana U)
>  * Xingjian Shi (HKUST)
> 
> == Sponsors ==
> 
> === Champion ===
> 
> Henri Yandell (bayard at apache.org)
> 
> === Nominated Mentors ===
> 
> Sebastian Schelter (ssc@apache.org)
> 
> 
> === Sponsoring Entity ===
> 
> We are requesting the Incubator to sponsor this project.
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org