[GitHub] samskalicky opened a new pull request #12091: [MXNET-792] Fix for issue #9816

[GitBox <gi...@apache.org>]

samskalicky opened a new pull request #12091: [MXNET-792] Fix for issue #9816
URL: https://github.com/apache/incubator-mxnet/pull/12091
 
 
   ## Description ##
   Fix for issue #9816 Dropout may mask values even when ratio=0.0
   
   as mentioned by @DickJC123, the problem is that on the CPU and GPU the random number generator works slightly differently. On the CPU the numbers are generated in the range [0,1) and on the GPU they are (0,1]. Therefor on the GPU a value of 1 can be generated and using the existing thresholding mshadow operator this results in an incorrect value since its only performing < operation. 
   
   This fix adds a <= comparison for dropout to fix this issue. 
   
   Another possible solution is to match the random number generator behavior between CPU and GPU, for which there is already a PR in the mshadow repo: https://github.com/dmlc/mshadow/pull/338
   
   When this PR gets merged the behavior of dropout with this fix will still be the same. So this fix works now and in the future.
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [X] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) created (except PRs with tiny changes)
   - [X] Changes are complete (i.e. I finished coding on this PR)
   - [X] All changes have test coverage:
   - Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
   - Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
   - Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
   - [X] Code is well-documented: 
   - For user-facing API changes, API doc string has been updated. 
   - For new C++ functions in header files, their functionalities and arguments are documented. 
   - For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
   - Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
   - [X] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change
   
   ### Changes ###
   added mshadow op for threshold_eq (theshold currently does <, this will do <=)
   
   modified dropout operator to use threshold_eq instead of theshold this will ensure equivalent behavior for the random numbers generated on CPU [0, 1) and GPU (0, 1]
   
   removed fixed seed for test_dropout
   
   ## Comments ##
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services