[GitHub] [incubator-mxnet] connorgoggins commented on issue #17632: [Large Tensor] Fixed RNN op

[GitBox <gi...@apache.org>]

connorgoggins commented on issue #17632: [Large Tensor] Fixed RNN op
URL: https://github.com/apache/incubator-mxnet/pull/17632#issuecomment-593589130
 
 
   @zixuanweeei thanks for your feedback! After testing the `size_t` changes on a separate branch, I found that they resulted in a segmentation fault (even on low-dimensional input). When I ran the op on the same input with my `index_t` changes (as they exist in the current state of this PR), the op passed without errors. My `index_t` changes also allow the op to run successfully on large tensor (dimension >= 2^32) input.
   
   With these considerations in mind, we are discussing the best way to move forward.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services