You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mxnet.apache.org by Sheng Zha <zh...@apache.org> on 2023/06/21 02:39:10 UTC
[RFC] Future of Apache MXNet
Dear MXNet community,
I would like to start a conversation about the future of Apache MXNet
and where we should head next.
# What we built
Apache MXNet is an open-source deep learning framework used to train
and deploy deep learning models developed by contributors from
multiple organizations. MXNet is known for its efficiency at scale.
MXNet supports multiple languages including Python, Scala, R, Julia,
Perl, and more, which makes it accessible to a wide variety of
developers and data scientists. Its core is written in C++ for
performance, but it provides a flexible interface that allows users to
write code in their preferred language.
Another important feature of MXNet is its support for both imperative
and symbolic programming styles. Imperative programming (like PyTorch)
is more intuitive and flexible, allowing users to write code as they
would in standard Python, whereas symbolic programming (like Theano,
TensorFlow) is more efficient in terms of runtime and memory usage and
allows for certain optimizations like graph-level optimizations and
auto-differentiation. This allows users to choose the programming
style that best suits their needs. Gluon interface further attempts at
unifying these paradigms.
As an attempt to address the legacy issues, the community started
working on the development of MXNet 2.0 in 2020. Some of the updates
include a new design for data loading in Gluon, a unified distributed
data parallel interface, parameterizable probability distributions, a
refactored MXNet np interface, and enhanced support for 3rd-party
functionality. We've also improved the development process with a new
CMake build system, a memory profiler, and more Pythonic exception
handling.
Along the journey, many people who share the enthusiasm about deep
learning joined our cause, and we managed to develop a community with
875 contributors, 87 committers and 51 PMC members. Many of the
community members continue to play important roles outside of MXNet in
the generative AI and deep learning system spaces. We graduated from
Apache Incubator in Sept 2022 to a top-level project. Our project is
the culmination of the hard work of many people over the years.
# Where we are
Since late 2022 the code development has mostly halted and community
engagement slowed. Despite the boom in generative AI and related
spaces such as deep learning frameworks and distributed training
solutions, the project's current positioning is not enough in
sustaining the growth of the project, especially in light of the
development in the open source deep learning framework space.
# What can we do
There are a few choices that we as a community can make:
1) we can continue on the current path by finding a critical mass to
continue drive maintenance
2) we can discuss alternative positions that MXNet can pivot to so
that this community can bring value beyond existing offerings.
3) we can retire as an Apache TLP into Attic.
Thanks,
Sheng
----------
This is a re-post of https://github.com/apache/mxnet/issues/21206
Re: [RFC] Future of Apache MXNet
Posted by Justin Mclean <jm...@apache.org>.
Hi,
Another month has passed and I'm still not seeing any activity or the requested actions being taken so I will be putting forward a board resolution to retire the project at the next board meeting.
Kind Regards,
Justin
Re: [RFC] Future of Apache MXNet
Posted by Rich Bowen <rb...@apache.org>.
It's been another month, and still no discussion on this thread. This seems to leave open 3 open - a move to the attic for this project. If that's the path that you're leaning towards, please do announce this intention on this list, and on the users@ list, so that this does not come as a surprise to anyone relying on this project.
--Rich
On 2023/06/21 02:39:10 Sheng Zha wrote:
> Dear MXNet community,
>
> I would like to start a conversation about the future of Apache MXNet
> and where we should head next.
>
> # What we built
>
> Apache MXNet is an open-source deep learning framework used to train
> and deploy deep learning models developed by contributors from
> multiple organizations. MXNet is known for its efficiency at scale.
> MXNet supports multiple languages including Python, Scala, R, Julia,
> Perl, and more, which makes it accessible to a wide variety of
> developers and data scientists. Its core is written in C++ for
> performance, but it provides a flexible interface that allows users to
> write code in their preferred language.
>
> Another important feature of MXNet is its support for both imperative
> and symbolic programming styles. Imperative programming (like PyTorch)
> is more intuitive and flexible, allowing users to write code as they
> would in standard Python, whereas symbolic programming (like Theano,
> TensorFlow) is more efficient in terms of runtime and memory usage and
> allows for certain optimizations like graph-level optimizations and
> auto-differentiation. This allows users to choose the programming
> style that best suits their needs. Gluon interface further attempts at
> unifying these paradigms.
>
> As an attempt to address the legacy issues, the community started
> working on the development of MXNet 2.0 in 2020. Some of the updates
> include a new design for data loading in Gluon, a unified distributed
> data parallel interface, parameterizable probability distributions, a
> refactored MXNet np interface, and enhanced support for 3rd-party
> functionality. We've also improved the development process with a new
> CMake build system, a memory profiler, and more Pythonic exception
> handling.
>
> Along the journey, many people who share the enthusiasm about deep
> learning joined our cause, and we managed to develop a community with
> 875 contributors, 87 committers and 51 PMC members. Many of the
> community members continue to play important roles outside of MXNet in
> the generative AI and deep learning system spaces. We graduated from
> Apache Incubator in Sept 2022 to a top-level project. Our project is
> the culmination of the hard work of many people over the years.
>
> # Where we are
>
> Since late 2022 the code development has mostly halted and community
> engagement slowed. Despite the boom in generative AI and related
> spaces such as deep learning frameworks and distributed training
> solutions, the project's current positioning is not enough in
> sustaining the growth of the project, especially in light of the
> development in the open source deep learning framework space.
>
> # What can we do
>
> There are a few choices that we as a community can make:
> 1) we can continue on the current path by finding a critical mass to
> continue drive maintenance
> 2) we can discuss alternative positions that MXNet can pivot to so
> that this community can bring value beyond existing offerings.
> 3) we can retire as an Apache TLP into Attic.
>
> Thanks,
> Sheng
> ----------
> This is a re-post of https://github.com/apache/mxnet/issues/21206
>
Re: [RFC] Future of Apache MXNet
Posted by Qing Lan <la...@live.com>.
I think it is just being quiet. Ack this message
Thanks,
Qing
________________________________
From: Rich Bowen <rb...@apache.org>
Sent: Wednesday, June 28, 2023 10:33
To: dev@mxnet.apache.org <de...@mxnet.apache.org>
Subject: Re: [RFC] Future of Apache MXNet
On 2023/06/21 02:39:10 Sheng Zha wrote:
> Dear MXNet community,
>
> I would like to start a conversation about the future of Apache MXNet
> and where we should head next.
I am very concerned that, a week on, there have been no responses to this message. Does that mean that the conversation is happening somewhere else? Or are there simply no project participants sufficiently engaged to express an opinion?
Re: [RFC] Future of Apache MXNet
Posted by Rich Bowen <rb...@apache.org>.
On 2023/06/21 02:39:10 Sheng Zha wrote:
> Dear MXNet community,
>
> I would like to start a conversation about the future of Apache MXNet
> and where we should head next.
I am very concerned that, a week on, there have been no responses to this message. Does that mean that the conversation is happening somewhere else? Or are there simply no project participants sufficiently engaged to express an opinion?