You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/12/04 03:44:15 UTC
[GitHub] [incubator-mxnet] feevos commented on issue #19609: mxnet 2.0 - GroupNorm does not work with as_nd_ndarray() upon activation of mx.npx.set_np()
feevos commented on issue #19609:
URL: https://github.com/apache/incubator-mxnet/issues/19609#issuecomment-738546600
Workaround that allows operations:
```python
net = gluon.nn.GroupNorm(num_groups=4)
net.initialize()
xx = mx.np.random.rand(3,32,512,512)
yy = xx.as_nd_ndarray()
mx.npx.reset_np() # <====== This fixes the situation by resetting numpy behaviour (I think!)
net.summary(yy)
mx.npx.set_np() # <======== This restores numpy behaviour.
```
Output:
```python
--------------------------------------------------------------------------------
Layer (type) Output Shape Param #
================================================================================
Input (3, 32, 512, 512) 0
GroupNorm-1 (3, 32, 512, 512) 64
================================================================================
Parameters in forward computation graph, duplicate included
Total params: 64
Trainable params: 64
Non-trainable params: 0
Shared params in forward computation graph: 0
Unique parameters in model: 64
--------------------------------------------------------------------------------
```
This is a small replacement hack for GroupNorm:
```python
import mxnet as mx
from mxnet import gluon
class GroupNormHack(gluon.nn.HybridBlock):
"""
This is a partial fix for issue #19609
see https://github.com/apache/incubator-mxnet/issues/19609
"""
def __init__(self, num_groups, **kwards):
super().__init__(**kwards)
self.norm = gluon.nn.GroupNorm(num_groups=num_groups,**kwards)
def forward(self, input):
tinput = input.as_nd_ndarray() if mx.npx.is_np_array() else input
mx.npx.reset_np()
out = self.norm(tinput)
mx.npx.set_np()
out = out.as_np_ndarray()
return out
````
toy run:
```python
In [11]: net = GroupNormHack(num_groups=4)
...: net.initialize()
...: xx = mx.np.random.rand(3,32,512,512)
...: net.summary(xx)
--------------------------------------------------------------------------------
Layer (type) Output Shape Param #
================================================================================
Input (3, 32, 512, 512) 0
GroupNorm-1 (3, 32, 512, 512) 64
GroupNormHack-2 (3, 32, 512, 512) 0
================================================================================
Parameters in forward computation graph, duplicate included
Total params: 64
Trainable params: 64
Non-trainable params: 0
Shared params in forward computation graph: 0
Unique parameters in model: 64
--------------------------------------------------------------------------------
In [12]: yy = xx.as_nd_ndarray()
In [13]: net.summary(yy)
--------------------------------------------------------------------------------
Layer (type) Output Shape Param #
================================================================================
Input (3, 32, 512, 512) 0
GroupNorm-1 (3, 32, 512, 512) 64
GroupNormHack-2 (3, 32, 512, 512) 0
================================================================================
Parameters in forward computation graph, duplicate included
Total params: 64
Trainable params: 64
Non-trainable params: 0
Shared params in forward computation graph: 0
Unique parameters in model: 64
--------------------------------------------------------------------------------
In [14]: net.hybridize()
In [15]: out = net(xx)
In [16]: out = net(yy)
In [17]:
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org