[GitHub] tqchen commented on issue #11964: [Feature request] There is no converter API from mx.nd.NDArray to DLPack tensor

[GitBox <gi...@apache.org>]

tqchen commented on issue #11964: [Feature request] There is no converter API from mx.nd.NDArray to DLPack tensor
URL: https://github.com/apache/incubator-mxnet/issues/11964#issuecomment-410771164
 
 
   This is a good proposal, there is one thing that we need to be cautious about: MXNet uses asynchronous execution and things can run out of order if it is not tracked by mxnet engine.
   
   Here is an example that things can go wrong.
   ```python
   x = mxnet.nd.zeros((3,4))
   x = x + 1
   # pass to pytorch or tvm
   y = other_framework.nd.fromdlpack(x.asdlpack())
   # Y may or maynot be 1
   print(y)
   ```
   This is because when we directly call ```x.asdlpack()``` and directly convert the data structure, the execution may not yet finished.
   
   ### A Fix
   We need to also add flags to asdlpack
   
   ```python
   x = mxnet.nd.zeros((3,4))
   x = x + 1
   # pass to pytorch or tvm
   x.wait_for_read()
   y = other_framework.nd.fromdlpack(x.asdlpack)
   # Y may or maynot be 1
   print(y)
   ```
   This works for read case, similarly, if we plan to modify the content, we need to do
   ```python
   x = mxnet.nd.zeros((3,4))
   x = x + 1
   # pass to pytorch or tvm
   x.wait_for_write()
   y = other_framework.nd.fromdlpack(x.asdlpack)
   y = y + 2
   # Y should be 3
   print(y)
   ```
   
   ### API design
   To make sure things won't go wrong because the user does not understand things, I would recommend we use the following API
   
   - NDArray.asdlpack_for_read()
   - NDArray.asdlpack_for_write()
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services