You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mxnet.apache.org by GitBox <gi...@apache.org> on 2021/08/09 02:47:44 UTC
[GitHub] [incubator-mxnet] NathanYyc opened a new issue #20501: [RFC] Python Array API standardization
NathanYyc opened a new issue #20501:
URL: https://github.com/apache/incubator-mxnet/issues/20501
## Problem statement
[Python API Standard](https://data-apis.org/array-api/latest/) is an standard Array API for all Python numerical computing, data science, machine learning, and deep learning frameworks. By specifying an API for the most common ways arrays are constructed and used, It aims to address the differences among those frameworks that makes developers quite difficult to write code that works with multiple (or all) of these libraries.
## Proposed solutions
By comparing the Python API Standard and APIs in MXNet.numpy, an list of APIs similar to the list in [pytorch Python Array API Compatibility Tracker](https://github.com/pytorch/pytorch/issues/58743) is formed. Further works will be done according to the list attached. The dlpack synchronization APIs in the list is specified in [Specify synchronization semantics](https://github.com/dmlc/dlpack/issues/57).
## References
- [Python API Standard](https://data-apis.org/array-api/latest/)
- [pytorch Python Array API Compatibility Tracker](https://github.com/pytorch/pytorch/issues/58743)
- [dlpack Specify synchronization semantics](https://github.com/dmlc/dlpack/issues/57)
## On-going Efforts
https://github.com/apache/incubator-mxnet/pull/20454
https://github.com/apache/incubator-mxnet/pull/20478
## Attachment
[MXNet & Python Api Standard Comparison](https://docs.google.com/spreadsheets/d/1pbIPGVXD6FrHqMbKtRqmaT5dzO1tNgb915Th8ZUCoNw/edit?usp=sharing)
- current list is base on APIs in MXNet 2.0 & Python Data API Specification(commit: cf5aa666017b8a8a2117aa1cb1770eb1cf841942)
- We classified each API into 6 categories:
- No Changes: The current PyTorch implementation already conforms to the Data API.
- New API: Need to add a new function or constant.
- Enhancement: Need enhancement to the existing function (add new argument, etc.).
- Enhancement with BC: Need to change signature of the existing function. (breaking change if we implement the API directly under torch.* namespace)
- Behavior: Need to change behavior of the existing function. (breaking change if we implement the API directly under torch.* namespace)
- N/A: The detailed specification is not determined yet in the Data API.
- There is 46 New APIs needed, 60 current APIs needed to be enhanced, 0 API needed to change behavior and 2 APIs is not specified in current Data API commit.
@barry-jin @szha
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] asmeurer commented on issue #20501: [RFC] Python Array API standardization
Posted by GitBox <gi...@apache.org>.
asmeurer commented on issue #20501:
URL: https://github.com/apache/incubator-mxnet/issues/20501#issuecomment-1014884305
Hi. Just wanted to comment here to see if there's any way we can help with the mxnet implementation of the array API.
As you may be aware, we have a test suite for array API compliance at https://github.com/data-apis/array-api-tests/. If you have any issues or questions regarding the test suite, please reach out to us by opening an issue in the array-api-tests issue tracker. In particular if you have any ways that you feel the test suite could be easier to use, we'd like to know about that so we can improve things. We want it to be as useful as possible for people implementing array API support in libraries like mxnet.
CC @honno
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] barry-jin commented on issue #20501: [RFC] Python Array API standardization
Posted by GitBox <gi...@apache.org>.
barry-jin commented on issue #20501:
URL: https://github.com/apache/incubator-mxnet/issues/20501#issuecomment-1015854400
Yes, that would be helpful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] github-actions[bot] commented on issue #20501: [RFC] Python Array API standardization
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #20501:
URL: https://github.com/apache/incubator-mxnet/issues/20501#issuecomment-894917935
Welcome to Apache MXNet (incubating)! We are on a mission to democratize AI, and we are glad that you are contributing to it by opening this issue.
Please make sure to include all the relevant context, and one of the @apache/mxnet-committers will be here shortly.
If you are interested in contributing to our project, let us know! Also, be sure to check out our guide on [contributing to MXNet](https://mxnet.apache.org/community/contribute) and our [development guides wiki](https://cwiki.apache.org/confluence/display/MXNET/Developments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] honno commented on issue #20501: [RFC] Python Array API standardization
Posted by GitBox <gi...@apache.org>.
honno commented on issue #20501:
URL: https://github.com/apache/incubator-mxnet/issues/20501#issuecomment-1015298985
I've written a basic wrapper to get things started, as the test suite requires some things like namespaced dtypes and info functions for most of the tests. I'm quite unfamiliar with MXNet so struggled to get far in making sure the test suite was working fine and it all the failures are non-compliance issues. https://gist.github.com/honno/c95eeabc71f5ca65d40e5a5105eda672
I think a native and fully-compliant [`xp.asarray()`](https://data-apis.org/array-api/latest/API_specification/creation_functions.html#asarray-obj-dtype-none-device-none-copy-none) would be a good priority if folks wanted to start using the test suite, although generally I think it's fair to ignore the test suite until you're much closer to covering the spec.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] honno edited a comment on issue #20501: [RFC] Python Array API standardization
Posted by GitBox <gi...@apache.org>.
honno edited a comment on issue #20501:
URL: https://github.com/apache/incubator-mxnet/issues/20501#issuecomment-1015298985
I've written a basic wrapper to get things started, as the test suite requires some things like namespaced dtypes and info functions for most of the tests. I'm quite unfamiliar with MXNet so struggled to get far in making sure the test suite was working fine and all the failures were expected non-compliance issues. https://gist.github.com/honno/c95eeabc71f5ca65d40e5a5105eda672
I think a native and fully-compliant [`xp.asarray()`](https://data-apis.org/array-api/latest/API_specification/creation_functions.html#asarray-obj-dtype-none-device-none-copy-none) would be a good priority if folks wanted to start using the test suite, although generally I think it's fair to ignore the test suite until you're much closer to covering the spec.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] asmeurer commented on issue #20501: [RFC] Python Array API standardization
Posted by GitBox <gi...@apache.org>.
asmeurer commented on issue #20501:
URL: https://github.com/apache/incubator-mxnet/issues/20501#issuecomment-1015842872
There have been quite a few additions and general improvements to the test suite since commit [c1dba80a196a03f880d2e0a998a272fb3867b720](https://github.com/data-apis/array-api-tests/commit/c1dba80a196a03f880d2e0a998a272fb3867b720), which is from September. Would it help you if we were to tag releases for the tests repo?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] barry-jin commented on issue #20501: [RFC] Python Array API standardization
Posted by GitBox <gi...@apache.org>.
barry-jin commented on issue #20501:
URL: https://github.com/apache/incubator-mxnet/issues/20501#issuecomment-1015572046
Thanks @asmeurer, @honno. MXNet master branch is using array-api-tests part of test suites in CI ([commit hash here](https://github.com/apache/incubator-mxnet/blob/7d84b598459985a59f7601f638cf8707389609c9/ci/docker/runtime_functions.sh#L876)). We also have [mxnet.numpy.array](https://github.com/apache/incubator-mxnet/blob/7d84b598459985a59f7601f638cf8707389609c9/python/mxnet/numpy/multiarray.py#L422-L447) as standard array API to be interoperable with other array libraries.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] AnirudhDagar commented on issue #20501: [RFC] Python Array API standardization
Posted by GitBox <gi...@apache.org>.
AnirudhDagar commented on issue #20501:
URL: https://github.com/apache/incubator-mxnet/issues/20501#issuecomment-899047539
Thanks for opening this discussion. Looking forward to MXNet's support of Array API.
cc @rgommers (for awareness)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org