You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2018/01/04 02:53:51 UTC
[GitHub] feevos commented on issue #9288: Get HybridBlock layer shape on runtime
feevos commented on issue #9288: Get HybridBlock layer shape on runtime
URL: https://github.com/apache/incubator-mxnet/issues/9288#issuecomment-355186696
Hi @lupesko , thank you very much for your reply. Perhaps this is something I am not understanding very well (the way to properly use hybridize call in a unified way both with Symbol and NDArray). I've just started learning Gluon + mxnet.
Your solution works (I actually have this solution to my code) when the ```_input``` variable is an NDArray type. But then I want a unified way of finding the shape at runtime both if I feed in an NDArray or a Symbol. As I understand it, the whole point of HybridBlock is to be able to use it both with Symbol and NDArray (and it's the former I have trouble with).
This modification of your example:
```Python
xx = nd.random_uniform(shape=[5,5,16,16])
mynet = runtime_shape()
mynet.hybridize() # This is the modification line
mynet.hybrid_forward(nd,xx)
```
Results the following error:
```Python
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-13-dd8cecdcffd1> in <module>()
----> 1 temp = mynet(xx)
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in __call__(self, *args)
302 def __call__(self, *args):
303 """Calls forward. Only accepts positional arguments."""
--> 304 return self.forward(*args)
305
306 def forward(self, *args):
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in forward(self, x, *args)
504 try:
505 if self._active:
--> 506 return self._call_cached_op(x, *args)
507 params = {i: j.data(ctx) for i, j in self._reg_params.items()}
508 except DeferredInitializationError:
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in _call_cached_op(self, *args)
412 def _call_cached_op(self, *args):
413 if self._cached_op is None:
--> 414 self._build_cache(*args)
415
416 args, fmt = _flatten(args)
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in _build_cache(self, *args)
377
378 def _build_cache(self, *args):
--> 379 inputs, out = self._get_graph(*args)
380 input_idx = {var.name: i for i, var in enumerate(inputs)}
381 self._cached_op = ndarray.CachedOp(out)
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in _get_graph(self, *args)
369 params = {i: j.var() for i, j in self._reg_params.items()}
370 with self.name_scope():
--> 371 out = self.hybrid_forward(symbol, *grouped_inputs, **params) # pylint: disable=no-value-for-parameter
372 out, self._out_format = _flatten(out)
373
<ipython-input-10-451e3d4c189d> in hybrid_forward(self, F, _input)
9
10 def hybrid_forward(self,F,_input):
---> 11 print('input shape: {}'.format(_input.shape))
12 return _input
13
AttributeError: 'Symbol' object has no attribute 'shape'
```
Another example of the same error can be seen in this Gluon example (from [here](http://gluon.mxnet.io/chapter07_distributed-learning/hybridize.html?highlight=hybridize))
```Python
import mxnet as mx
import mxnet.ndarray as nd
from mxnet import gluon
class Net(gluon.HybridBlock):
def __init__(self, **kwargs):
super(Net, self).__init__(**kwargs)
with self.name_scope():
self.fc1 = gluon.nn.Dense(256)
self.fc2 = gluon.nn.Dense(128)
self.fc3 = gluon.nn.Dense(2)
def hybrid_forward(self, F, x):
# F is a function space that depends on the type of x
# If x's type is NDArray, then F will be mxnet.nd
# If x's type is Symbol, then F will be mxnet.sym
print('type(x): {}, F: {}'.format(
type(x).__name__, F.__name__))
# ----------------------------------------------------------------------------------------------
# This is the line that I added, to infer the shape, based on your proposal
print('input shape: {}'.format(x.shape))
# ----------------------------------------------------------------------------------------------
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
return self.fc3(x)
# First run with NDArray works:
net = Net()
net.collect_params().initialize()
x = nd.random_normal(shape=(1, 512))
print('=== 1st forward ===')
y = net(x)
print('=== 2nd forward ===')
y = net(x)
```
### Output:
```Python
=== 1st forward ===
type(x): NDArray, F: mxnet.ndarray
input shape: (1L, 512L)
=== 2nd forward ===
type(x): NDArray, F: mxnet.ndarray
input shape: (1L, 512L)
```
But if you call hybridize, you get an error:
```Python
net.hybridize()
print('=== 1st forward ===')
y = net(x)
print('=== 2nd forward ===')
y = net(x)
```
## Error:
```Python
=== 1st forward ===
type(x): Symbol, F: mxnet.symbol
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-6-984f1343eb63> in <module>()
1 net.hybridize()
2 print('=== 1st forward ===')
----> 3 y = net(x)
4 print('=== 2nd forward ===')
5 y = net(x)
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in __call__(self, *args)
302 def __call__(self, *args):
303 """Calls forward. Only accepts positional arguments."""
--> 304 return self.forward(*args)
305
306 def forward(self, *args):
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in forward(self, x, *args)
504 try:
505 if self._active:
--> 506 return self._call_cached_op(x, *args)
507 params = {i: j.data(ctx) for i, j in self._reg_params.items()}
508 except DeferredInitializationError:
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in _call_cached_op(self, *args)
412 def _call_cached_op(self, *args):
413 if self._cached_op is None:
--> 414 self._build_cache(*args)
415
416 args, fmt = _flatten(args)
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in _build_cache(self, *args)
377
378 def _build_cache(self, *args):
--> 379 inputs, out = self._get_graph(*args)
380 input_idx = {var.name: i for i, var in enumerate(inputs)}
381 self._cached_op = ndarray.CachedOp(out)
/home/dia021/anaconda2/lib/python2.7/site-packages/mxnet/gluon/block.pyc in _get_graph(self, *args)
369 params = {i: j.var() for i, j in self._reg_params.items()}
370 with self.name_scope():
--> 371 out = self.hybrid_forward(symbol, *grouped_inputs, **params) # pylint: disable=no-value-for-parameter
372 out, self._out_format = _flatten(out)
373
<ipython-input-3-53434f883f09> in hybrid_forward(self, F, x)
20
21
---> 22 print('input shape: {}'.format(x.shape))
23
24 x = F.relu(self.fc1(x))
AttributeError: 'Symbol' object has no attribute 'shape'
```
And to make the idea of what I need more concrete, I want to implement the pyramid scene parsing pooling (from [here](https://hszhao.github.io/projects/pspnet/)) as a custom layer (inheritance from HybridBlock) in which I need to apply pooling in 4 scales. And to my understanding, I need to know the input layer size, so I can infer correct proposals for kernel/stride sizes in the ```F.Pooling``` operator. The point is, I want to test/debug with NDArray and then call hybridize for efficient computations.
Again thank you very much for your answer.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services