[GitHub] [incubator-mxnet] rondogency commented on a change in pull request #17547: Fix cudnn Dropout reproducibility

[GitBox <gi...@apache.org>]

rondogency commented on a change in pull request #17547:
URL: https://github.com/apache/incubator-mxnet/pull/17547#discussion_r413358071



##########
File path: src/operator/nn/dropout-inl.h
##########
@@ -255,8 +255,13 @@ class DropoutOp {
       Stream<xpu> *s = ctx.get_stream<xpu>();
 
       // set dropout state.
-      ctx.requested[0].get_cudnn_dropout_desc(&dropout_desc_, s, 1.0f - this->pkeep_, seed_);
-
+      Random<xpu, unsigned> *prnd = ctx.requested[1].get_random<xpu, unsigned>(s);
+      uint64_t rng_seed = prnd->GetSeed();
+      // reset dropout descriptor if rng seed changed.
+      bool reset = seed_ != rng_seed;
+      seed_ = rng_seed;
+      ctx.requested[0].get_cudnn_dropout_desc(&dropout_desc_, s, 1.0f - this->pkeep_,
+          seed_, reset);

Review comment:
       I believe @roywei is already working on this issue, since this regression has been caught by our benchmark last week




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org