You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/01/07 19:45:22 UTC

[GitHub] [incubator-mxnet] szhengac opened a new issue #17240: [MXNet 2.0] [RFC] Distribution Module

szhengac opened a new issue #17240: [MXNet 2.0] [RFC] Distribution Module
URL: https://github.com/apache/incubator-mxnet/issues/17240
 
 
   ## Description
   In MxNet 2.0, we would like to provide a distribution module, analogous to Pytorch distribution. The main difference from theirs is that we use numpy op and it allows hybridization. The current project code can be seen from https://github.com/xidulu/incubator-mxnet/tree/distribution_dev/python/mxnet/gluon/probability. 
   
   The basic skeleton divides into following parts:
   
   1. Stochastic `HybridBlock` and `HybridSequential`: they build upon gluon `HybridBlock` and `HybridSequential` and allows adding extra loss to each layer.
   2. Distribution class: it implements a variety of functionalities including `prob`, `log_prob`, `sample`, `broadcast_to`, `mean`, `variance`, etc.
   3. KL divergence: `kl_divergence(p, q)` function searches over registered KL divergence functions and performs computation.
   4. Transform: transform one distribution to another invertible distribution.
   5. Independent: reinterprets some of the batch dims of a distribution as event dims.
   
   Two features that is currently either not supported or kind of broken in MxNet will be very useful to this projects: symbolic shape and control flow.
   
   At the moment, we will implement most of distribution in frontend. We will move the computation to backend when new numpy probability ops such as `chisquare`, `dirichlet` and `multivariate_normal` are introduced into MxNet.
   
   ## References
   - https://pytorch.org/docs/stable/distributions.html
   - https://docs.scipy.org/doc/numpy-1.14.1/reference/routines.random.html
   
   @xidulu @szha @leezu @haojin2 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-mxnet] xidulu edited a comment on issue #17240: [MXNet 2.0] [RFC] Distribution Module

Posted by GitBox <gi...@apache.org>.
xidulu edited a comment on issue #17240: [MXNet 2.0] [RFC] Distribution Module
URL: https://github.com/apache/incubator-mxnet/issues/17240#issuecomment-571845540
 
 
   > Two features that is currently either not supported or kind of broken in MxNet will be very useful to this projects: symbolic shape and control flow.
   
   More specifically, control flow helps perform parameter validity check: https://github.com/apache/incubator-mxnet/issues/16868
   
   `Symbolic Shape` is required for decomposing tensor shape into `batch_shape` and `event_shape`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-mxnet] xidulu commented on issue #17240: [MXNet 2.0] [RFC] Distribution Module

Posted by GitBox <gi...@apache.org>.
xidulu commented on issue #17240: [MXNet 2.0] [RFC] Distribution Module
URL: https://github.com/apache/incubator-mxnet/issues/17240#issuecomment-571845540
 
 
   > Two features that is currently either not supported or kind of broken in MxNet will be very useful to this projects: symbolic shape and control flow.
   
   More specifically, control flow helps perform parameter validity check: https://github.com/apache/incubator-mxnet/issues/16868
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-mxnet] xidulu edited a comment on issue #17240: [MXNet 2.0] [RFC] Distribution Module

Posted by GitBox <gi...@apache.org>.
xidulu edited a comment on issue #17240: [MXNet 2.0] [RFC] Distribution Module
URL: https://github.com/apache/incubator-mxnet/issues/17240#issuecomment-571845540
 
 
   > Two features that is currently either not supported or kind of broken in MxNet will be very useful to this projects: symbolic shape and control flow.
   
   More specifically, control flow helps perform parameter validity check: https://github.com/apache/incubator-mxnet/issues/16868
   
   `Symbolic Shape` is required for decomposing tensor shape into `batch_shape` and `event_shape`.
   
   ----------------------------------------------
   
   > Stochastic HybridBlock and HybridSequential: they build upon gluon HybridBlock and HybridSequential and allows adding extra loss to each layer.
   
   `Stochastic Block`, like its name, is a block that produces non-deterministic output, e.g. a bayesian neural network.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-mxnet] szha commented on issue #17240: [MXNet 2.0] [RFC] Distribution Module

Posted by GitBox <gi...@apache.org>.
szha commented on issue #17240:
URL: https://github.com/apache/incubator-mxnet/issues/17240#issuecomment-667311435


   This module has been merged and will be included as part of mxnet 2.0. Thank you for the contribution @xidulu @szhengac 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-mxnet] szha closed issue #17240: [MXNet 2.0] [RFC] Distribution Module

Posted by GitBox <gi...@apache.org>.
szha closed issue #17240:
URL: https://github.com/apache/incubator-mxnet/issues/17240


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org