[GitHub] zhanghang1989 opened a new issue #9648: BatchNorm Evaluation Mode Backward Fails with cudnn Enabled

[GitBox <gi...@apache.org>]

zhanghang1989 opened a new issue #9648: BatchNorm Evaluation Mode Backward Fails with cudnn Enabled
URL: https://github.com/apache/incubator-mxnet/issues/9648
 
 
   When installing MXNet with cudnn enabled ``USE_CUDNN=1``. The backward of BatchNorm operator fails
   
   ## Reproducing the error
   ```
   import mxnet as mx
   import mxnet.ndarray as F
   from mxnet import autograd
   
   B,C,H,W = 4,3,2,2
   x = mx.nd.random.poisson(1,shape=(B,C,H,W)).as_in_context(mx.gpu(0))
   gamma = mx.nd.random.normal(shape=(C)).as_in_context(mx.gpu(0))
   beta = mx.nd.random.normal(shape=(C)).as_in_context(mx.gpu(0))
   mean = mx.nd.random.normal(shape=(C)).as_in_context(mx.gpu(0))
   std = mx.nd.random.normal(shape=(C)).as_in_context(mx.gpu(0))
   x.attach_grad()
   
   with autograd.record(False):
       y = F.BatchNorm(x, gamma, beta, mean, std.square(), fix_gamma=False)     
       loss=y.square().sum()
   loss.backward(train_mode=False)
   ```
   got the error:
   ```bash
   terminate called after throwing an instance of 'dmlc::Error'
     what():  [19:23:23] src/engine/./threaded_engine.h:359: [19:23:23] src/operator/nn/./cudnn/cudnn_batch_norm-inl.h:193: Check failed: ctx.is_train && !param_.use_global_stats use global statistics is not yet supported in CuDNNBatchNorm
   ```
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services