You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2018/01/22 00:54:38 UTC
[GitHub] masa-ito-fj commented on issue #9374: Conversion from Gluon block to MXNet module failed
masa-ito-fj commented on issue #9374: Conversion from Gluon block to MXNet module failed
URL: https://github.com/apache/incubator-mxnet/issues/9374#issuecomment-359297970
Thanks to the advice of [@merrymercy](https://github.com/merrymercy) in [the NNVM issue #297](https://github.com/dmlc/nnvm/issues/297), the problem has been solved.
The trick was to add one line to the function `block2symbol(block)` and to correct some lines about which I made mistakes about the usage of basic python modules.
<pre>
m-ito@dgx1:~/TestbedNNVM/nnvm1223_mxnet100/mxnet100$ git diff example/gluon/from_gluon_model_zoo_to_mxnet_module.py
diff --git a/nnvm1223_mxnet100/mxnet100/example/gluon/from_gluon_model_zoo_to_mxnet_module.py b/nnvm1223_mxnet100/mxnet100/example/gluon/from_gluon_model_zoo_to_mxnet_module.py
index 0a07e01..88631cb 100644
--- a/nnvm1223_mxnet100/mxnet100/example/gluon/from_gluon_model_zoo_to_mxnet_module.py
+++ b/nnvm1223_mxnet100/mxnet100/example/gluon/from_gluon_model_zoo_to_mxnet_module.py
@@ -42,6 +42,7 @@ def block2symbol(block):
auxs = {}
for k, v in block.collect_params().items():
args[k] = mx.nd.array(v.data().asnumpy())
+ auxs[k] = mx.nd.array(v.data().asnumpy())
return sym, args, auxs
mx_sym, args, auxs = block2symbol(block)
@@ -70,10 +71,12 @@ model.set_params( args, auxs)
######################################################################
# Time measurement for MXNet
######################################################################
+import time
+
print("Time measurement starts for mxnet itself..")
start_time = time.time()
-mx_output = model.predict( eval_iter).asnumpy()
+mx_output = model.predict( eval_iter)
elapsed_time = time.time() - start_time
print("elapsed time:{0}".format( elapsed_time) + "[sec]")
</pre>
I've confirmed that the corrected script worked well.
<pre>
$ python mxnet100/example/gluon/from_gluon_model_zoo_to_mxnet_module.py
/home/m-ito/anaconda3/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py:46: DeprecationWarning: OpenSSL.rand is deprecated - you should use os.urandom instead
import OpenSSL.SSL
x (1, 3, 224, 224)
[09:32:52] src/operator/././cudnn_algoreg-inl.h:112: Running performance tests to find the best convolution algorithm, this can take a while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
Time measurement starts for mxnet itself..
elapsed time:0.0020139217376708984[sec]
MXNet prediction top-1: 281 tabby, tabby cat
</pre>
Now I'm happily closing this issue.
Thanks,
Masa
## The corrected script
`example/gluon/from_gluon_model_zoo_to_mxnet_module.py`
```python
# some standard imports
import mxnet as mx
import numpy as np
######################################################################
# Download Resnet18 model from Gluon Model Zoo
######################################################################
from mxnet.gluon.model_zoo.vision import get_model
from mxnet.gluon.utils import download
from PIL import Image
from matplotlib import pyplot as plt
block = get_model('resnet18_v1', pretrained=True)
img_name = 'cat.jpg'
synset_url = ''.join(['https://gist.githubusercontent.com/zhreshold/',
'4d0b62f3d01426887599d4f7ede23ee5/raw/',
'596b27d23537e5a1b5751d2b0481ef172f58b539/',
'imagenet1000_clsid_to_human.txt'])
synset_name = 'synset.txt'
download('https://github.com/dmlc/mxnet.js/blob/master/data/cat.png?raw=true', img_name)
download(synset_url, synset_name)
with open(synset_name) as f:
synset = eval(f.read())
image = Image.open(img_name).resize((224, 224))
def transform_image(image):
image = np.array(image) - np.array([123., 117., 104.])
image /= np.array([58.395, 57.12, 57.375])
image = image.transpose((2, 0, 1))
image = image[np.newaxis, :]
return image
x = transform_image(image)
print('x', x.shape)
######################################################################
# Convert Gluon Block to MXNet Symbol
######################################################################
def block2symbol(block):
data = mx.sym.Variable('data')
sym = block(data)
args = {}
auxs = {}
for k, v in block.collect_params().items():
args[k] = mx.nd.array(v.data().asnumpy())
auxs[k] = mx.nd.array(v.data().asnumpy())
return sym, args, auxs
mx_sym, args, auxs = block2symbol(block)
######################################################################
# Prepare MXNet module
######################################################################
mx_sym = mx.sym.SoftmaxOutput( data=mx_sym, name='softmax')
model = mx.mod.Module( symbol = mx_sym, context = mx.gpu(),
label_names = ['softmax_label'])
eval_data = x
eval_label = np.array([0.0])
batch_size = 1
eval_iter = mx.io.NDArrayIter( eval_data, eval_label,
batch_size, shuffle=False)
model.bind( data_shapes = eval_iter.provide_data,
label_shapes = eval_iter.provide_label )
model.set_params( args, auxs)
######################################################################
# Time measurement for MXNet
######################################################################
import time
print("Time measurement starts for mxnet itself..")
start_time = time.time()
mx_output = model.predict( eval_iter)
elapsed_time = time.time() - start_time
print("elapsed time:{0}".format( elapsed_time) + "[sec]")
top1 = np.argmax(mx_output.asnumpy())
print('MXNet prediction top-1:', top1, synset[top1])
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services