[GitHub] lanking520 opened a new issue #13332: MXNet Python inference crash when copy from CPU to GPU

[GitBox <gi...@apache.org>]

lanking520 opened a new issue #13332: MXNet Python inference crash when copy from CPU to GPU
URL: https://github.com/apache/incubator-mxnet/issues/13332
 
 
   ## Description
   MXNet Python GPU would crash if pass cpu-context image to GPU context model
   model download script:
   https://github.com/apache/incubator-mxnet/blob/master/scala-package/examples/scripts/infer/imageclassifier/get_resnet_data.sh
   
   If the user defines the image as GPU-contexted, problem solved.
   
   @piyushghai @andrewfayres 
   
   Package used:
   Python
   
   Build config:
   ```
   pip install mxnet-cu90
   ```
   Build from source also not solve the problem
   
   ## Error Message:
   ```
   [06:43:27] src/nnvm/legacy_json_util.cc:209: Loading symbol saved by previous version v0.8.0. Attempting to upgrade...
   [06:43:27] src/nnvm/legacy_json_util.cc:217: Symbol successfully upgraded!
   Traceback (most recent call last):
     File "ssd_minimum.py", line 50, in <module>
       predict()
     File "ssd_minimum.py", line 35, in predict
       print (prob)
     File "/home/ubuntu/venv/local/lib/python2.7/site-packages/mxnet/ndarray/ndarray.py", line 189, in __repr__
       return '\n%s\n<%s %s @%s>' % (str(self.asnumpy()),
     File "/home/ubuntu/venv/local/lib/python2.7/site-packages/mxnet/ndarray/ndarray.py", line 1972, in asnumpy
       ctypes.c_size_t(data.size)))
     File "/home/ubuntu/venv/local/lib/python2.7/site-packages/mxnet/base.py", line 252, in check_call
       raise MXNetError(py_str(_LIB.MXGetLastError()))
   mxnet.base.MXNetError: [06:43:31] src/ndarray/ndarray_function.cu:45: Check failed: to->type_flag_ == from.type_flag_ (0 vs. 3) Source and target must have the same data type when copying across devices.
   
   Stack trace returned 10 entries:
   [bt] (0) /home/ubuntu/venv/local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x382d4a) [0x7fd2cce78d4a]
   [bt] (1) /home/ubuntu/venv/local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x383381) [0x7fd2cce79381]
   [bt] (2) /home/ubuntu/venv/local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x4df96e8) [0x7fd2d18ef6e8]
   [bt] (3) /home/ubuntu/venv/local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2ce0546) [0x7fd2cf7d6546]
   [bt] (4) /home/ubuntu/venv/local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2cf635a) [0x7fd2cf7ec35a]
   [bt] (5) /home/ubuntu/venv/local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2cf648b) [0x7fd2cf7ec48b]
   [bt] (6) /home/ubuntu/venv/local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2af2a24) [0x7fd2cf5e8a24]
   [bt] (7) /home/ubuntu/venv/local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2af9aa3) [0x7fd2cf5efaa3]
   [bt] (8) /home/ubuntu/venv/local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2af9cf6) [0x7fd2cf5efcf6]
   [bt] (9) /home/ubuntu/venv/local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2af3134) [0x7fd2cf5e9134]
   ```
   
   ## Minimum reproducible example
   ```
   import mxnet as mx
   from mxnet import ndarray as nd
   
   image_path = "dog.jpg"
   synset_path = "synset.txt"
   model_prefix = "resnet-152"
   
   ctx = mx.gpu()
   sym, arg_params, aux_params = mx.model.load_checkpoint(model_prefix, 0)
   mod = mx.mod.Module(symbol=sym, context=ctx, label_names=None)
   mod.bind(for_training=False, data_shapes=[('data', (1,3,224,224))],
            label_shapes=mod._label_shapes)
   mod.set_params(arg_params, aux_params, allow_missing=True)
   
   with open(synset_path, 'r') as f:
       labels = [l.rstrip() for l in f]
   
   def pre_process_image(path):
       img = mx.image.imread(path)
       if img is None:
               return None
       img = mx.image.imresize(img, 224, 224) # resize
       img = img.transpose((2, 0, 1)) # Channel first
       img = img.expand_dims(axis=0) # batchify
       a = nd.concat(img, dim = 0)
       return a
   
   
   def predict():
       # compute the predict probabilities
       img = pre_process_image(image_path)
       data_iter = mx.io.NDArrayIter([img], None, 1)
       op = mod.predict(data_iter)
       prob = op[0]
       print (prob)
   
   predict()
   ```
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services