[GitHub] [arrow] ezyang commented on pull request #33925: GH-33923: [Docs] Tensor canonical extension type specification

["ezyang (via GitHub)" <gi...@apache.org>]

ezyang commented on PR #33925:
URL: https://github.com/apache/arrow/pull/33925#issuecomment-1416169108

   Some recommendations from me (not officially representing PyTorch team, but just based on my experience.)
   
   * Don't encode dimension names. We tried this with named tensor, but it's too restrictive and the propagation rules are too difficult to right. The correct way to do it is first class dims (https://github.com/facebookresearch/torchdim) but this is a UX problem that shouldn't be in arrow
   * You can choose to represent strides or dimension ordering. Dimension ordering is good enough to represent memory ordering, but not good enough to represent a broadcasted input. We are internally experimenting with only recording memory ordering for our mobile runtime, since strides are too expressive and they do not support aliasing in their runtime anyway. It is a lossy conversion (distinct strides can map to the same dimension ordering), but it is not too difficult to define a "best" stride for any given dimension ordering.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org