You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/08/10 21:17:03 UTC
[GitHub] [incubator-mxnet] mk-61 opened a new issue #18896: [RFC] Moving MXNet-AMP to core
mk-61 opened a new issue #18896:
URL: https://github.com/apache/incubator-mxnet/issues/18896
MXNet already has experimental AMP (Automatic Mixed Precision) support, exposed in mxnet.contrib package. It is used for automatic casting models to both float16 and bfloat16. This RFC covers moving it into core / making a first-class feature, as well as further development.
Here's a rough task break down for the initial move:
* Need to ensure AMP works with numpy ops - i.e., all ops are in either of the lists
* API change: make loss scale public (https://github.com/apache/incubator-mxnet/issues/17507)
* A number of issues has to be resolved to improve user experience:
1. Cannot load trainer with AMP (https://github.com/apache/incubator-mxnet/issues/16858)
2. There's a CUDA crash (IMA) in amp_multicast, happens on some models (Yolo3)
* The actual shuffling code around and updating import paths
Post move:
1. Layout optimization - upstreaming feature already existing in NVIDIA NGC container. This helps convolutions' performance by automatically casting between NCHW and NHWC layouts.
2. Explore alternatives to front end ops monkey-patching (https://github.com/apache/incubator-mxnet/issues/18697)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] sxjscience commented on issue #18896: [RFC] Moving MXNet-AMP to core
Posted by GitBox <gi...@apache.org>.
sxjscience commented on issue #18896:
URL: https://github.com/apache/incubator-mxnet/issues/18896#issuecomment-673143785
@mk-61 If you'd like to see some test cases of the new numpy API, you can also try the numpy version of GluonNLP: https://github.com/dmlc/gluon-nlp/tree/numpy . Would we connect via Slack?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] github-actions[bot] commented on issue #18896: [RFC] Moving MXNet-AMP to core
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #18896:
URL: https://github.com/apache/incubator-mxnet/issues/18896#issuecomment-671594508
Welcome to Apache MXNet (incubating)! We are on a mission to democratize AI, and we are glad that you are contributing to it by opening this issue.
Please make sure to include all the relevant context, and one of the @apache/mxnet-committers will be here shortly.
If you are interested in contributing to our project, let us know! Also, be sure to check out our guide on [contributing to MXNet](https://mxnet.apache.org/community/contribute) and our [development guides wiki](https://cwiki.apache.org/confluence/display/MXNET/Developments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on issue #18896: [RFC] Moving MXNet-AMP to core
Posted by GitBox <gi...@apache.org>.
szha commented on issue #18896:
URL: https://github.com/apache/incubator-mxnet/issues/18896#issuecomment-673009714
cc @sxjscience @leezu
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org