You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/03/18 03:59:46 UTC
[GitHub] [incubator-mxnet] MonicaGu opened a new issue #17863: Inconsistent
results in gluon.nn.BatchNorm with autograd.record()
MonicaGu opened a new issue #17863: Inconsistent results in gluon.nn.BatchNorm with autograd.record()
URL: https://github.com/apache/incubator-mxnet/issues/17863
## Description
I found that the calculation result of gluon.nn.BatchNorm is different with autograd.record(). In my case, with `with autograd.record():`, the result is an all-zero array.
## To Reproduce
```
import mxnet as mx
from mxnet import gluon, autograd
import numpy as np
x = np.random.rand(100).reshape(1, 100)
x1 = mx.nd.array(x)
net = gluon.nn.BatchNorm(scale=False, epsilon=2e-5, momentum=0.0)
net.initialize()
net.hybridize()
out = net(x1)
print("mxnet output without autograd.record(): ", out)
with autograd.record():
out = net(x1)
print("mxnet output with autograd.record(): ", out)
```
The output is:
```
mxnet output without autograd.record()::
[[8.18885207e-01 2.66843528e-01 5.33376813e-01 1.83545023e-01
8.08935821e-01 8.00210759e-02 3.52186531e-01 8.47646832e-01
5.41471958e-01 5.62665751e-04 7.71035254e-01 9.04082730e-02
5.68769276e-01 2.59562898e-02 4.34581816e-01 7.15692341e-01
7.32050180e-01 5.12164235e-01 4.77175564e-01 7.24380538e-02
7.21841812e-01 5.90174019e-01 4.94065672e-01 6.09854400e-01
4.65724707e-01 8.74445856e-01 7.62582242e-01 9.46706593e-01
3.58928591e-01 2.08127305e-01 2.27749124e-01 6.64541721e-01
7.28368759e-01 8.72908592e-01 6.98456526e-01 5.15673637e-01
3.61637414e-01 6.66116893e-01 7.96750307e-01 1.76226780e-01
5.01491308e-01 8.95380676e-01 2.06285361e-02 2.11953908e-01
4.65327024e-01 4.11169454e-02 5.48946597e-02 6.57186732e-02
3.97666693e-01 8.16613257e-01 2.83932853e-02 3.12020212e-01
9.10354972e-01 1.99801296e-01 6.88620925e-01 8.13198566e-01
6.23547018e-01 2.73092359e-01 3.04562356e-02 2.86002994e-01
1.88241839e-01 5.69275916e-01 7.89262831e-01 6.32545769e-01
8.09272565e-03 6.22344255e-01 4.10679430e-02 5.36309004e-01
9.36858892e-01 2.38068365e-02 5.42313457e-01 6.75516129e-01
5.42027175e-01 3.49770606e-01 3.54465336e-01 6.52725697e-01
3.10871273e-01 4.37797189e-01 8.00835729e-01 9.35012043e-01
4.64217126e-01 1.08712226e-01 2.83921272e-01 8.04676950e-01
1.54787794e-01 2.97454774e-01 7.89665878e-01 5.84875762e-01
3.08593541e-01 7.66469896e-01 8.56471717e-01 6.50003374e-01
9.76841450e-01 8.51188004e-01 7.20394015e-01 9.61582899e-01
8.43558788e-01 8.13576020e-03 2.78116226e-01 1.57843783e-01]]
<NDArray 1x100 @cpu(0)>
mxnet output with autograd.record()::
[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
0. 0. 0. 0.]]
<NDArray 1x100 @cpu(0)>
```
### Steps to reproduce
Just run the code above and compare the results.
## Environment
```
----------Python Info----------
Version : 3.6.9
Compiler : GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)
Build : ('default', 'Sep 16 2019 21:06:28')
Arch : ('64bit', '')
------------Pip Info-----------
Version : 18.1
Directory : /usr/local/var/pyenv/versions/3.6.9/lib/python3.6/site-packages/pip
----------MXNet Info-----------
Version : 1.6.0
Directory : /usr/local/var/pyenv/versions/3.6.9/lib/python3.6/site-packages/mxnet
Num GPUs : 0
Commit Hash : 6eec9da55c5096079355d1f1a5fa58dcf35d6752
----------System Info----------
Platform : Darwin-19.0.0-x86_64-i386-64bit
system : Darwin
node : gudiandiandeMacBook.local
release : 19.0.0
version : Darwin Kernel Version 19.0.0: Wed Sep 25 20:18:50 PDT 2019; root:xnu-6153.11.26~2/RELEASE_X86_64
----------Hardware Info----------
machine : x86_64
processor : i386
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] MonicaGu commented on issue #17863: Inconsistent
results in gluon.nn.BatchNorm with autograd.record()
Posted by GitBox <gi...@apache.org>.
MonicaGu commented on issue #17863: Inconsistent results in gluon.nn.BatchNorm with autograd.record()
URL: https://github.com/apache/incubator-mxnet/issues/17863#issuecomment-602976658
OK I see. Thank you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] MonicaGu closed issue #17863: Inconsistent
results in gluon.nn.BatchNorm with autograd.record()
Posted by GitBox <gi...@apache.org>.
MonicaGu closed issue #17863: Inconsistent results in gluon.nn.BatchNorm with autograd.record()
URL: https://github.com/apache/incubator-mxnet/issues/17863
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] nicklhy commented on issue #17863: Inconsistent
results in gluon.nn.BatchNorm with autograd.record()
Posted by GitBox <gi...@apache.org>.
nicklhy commented on issue #17863: Inconsistent results in gluon.nn.BatchNorm with autograd.record()
URL: https://github.com/apache/incubator-mxnet/issues/17863#issuecomment-602652408
I guess this is the expected result. In training mode, BN calculates the mean and variance of batch data first. And since you set momentum as 0, the data_mean is exactly the input data. Thus, you will get an output of all zeros here.
You can refer the [doc](https://mxnet.incubator.apache.org/api/python/docs/api/ndarray/ndarray.html#mxnet.ndarray.BatchNorm) of BN for details.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services