[GitHub] [incubator-mxnet] dibgerge commented on issue #19722: [RFC + BUG?] Implementation of Categorical distribution

[GitBox <gi...@apache.org>]

dibgerge commented on issue #19722:
URL: https://github.com/apache/incubator-mxnet/issues/19722#issuecomment-754336962


   > Hi @dibgerge
   > The reason "num_events" is required is that we cannot call `logit.shape` or `prob.shape` in the symbolic mode, as you can see, TF would use a very complicated subroutine to get num_events in the symbol mode: https://github.com/tensorflow/probability/blob/v0.11.1/tensorflow_probability/python/distributions/categorical.py#L352
   > while torch does not have such issue : https://github.com/pytorch/pytorch/blob/master/torch/distributions/categorical.py#L57
   > 
   > You are also right about the `logit` issue, that could be a **bug**. But MXNet currently does not have a logsumexp Op, a relative Op would be logaddexp, which is still not merged yet: #15857
   > A possible solution could be applying npx.logsoftmax on the logit.
   
   Thank you for the clarification. I am not very familiar with the symbolic code. I wonder if it is worth implementing this subroutine to calculate shape if at all possible, or it may add significant overhead. 
   Also wondering, would there be performance differences between `npx.logsoftmax` and the `logaddexp` PR?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org