[GitHub] NRauschmayr opened a new issue #13113: MXNet TensorRT generates wrong results

[GitBox <gi...@apache.org>]

NRauschmayr opened a new issue #13113: MXNet TensorRT generates wrong results
URL: https://github.com/apache/incubator-mxnet/issues/13113
 
 
   Following up from this thread: https://discuss.mxnet.io/t/mxnet-tensorrt-result-different/2139
   
   I tested it myself and it seems that MXNet TensorRT seems to work fine for VGG16 and other models but not for any of the Resnet models from the Gluon model zoo. 
   
   I used the mxnet/tensorrt Docker image.
   
   Here an example to reproduce the problem:
   
   ```import mxnet as mx
   from mxnet.gluon.model_zoo import vision
   import time
   import os
   import mxnet as mx
   import numpy as np
   from collections import namedtuple
   
   batch_shape = (1, 3, 224, 224)
   def get_image(url, show=False):
       fname = mx.test_utils.download(url, fname=url.split('/')[-1].split('?')[0])
       img = mx.image.imread(fname)
       img = mx.image.imresize(img, 224, 224) # resize
       img = img.transpose((2, 0, 1)) # Channel first
       img = img.expand_dims(axis=0) # batchify
       return img/255.0
   url= 'https://github.com/dmlc/web-data/blob/master/mxnet/doc/tutorials/python/predict_image/cat.jpg?raw=true'
   
   input_data = get_image(url, show=True)
   #resnet18 = vision.vgg16(pretrained=True)
   resnet18 = vision.resnet18_v2(pretrained=True)
   resnet18.hybridize()
   resnet18.forward(mx.nd.zeros(batch_shape))
   resnet18.export('resnet18_v2')
   sym, arg_params, aux_params = mx.model.load_checkpoint('resnet18_v2', 0)
   
   # Execute with MXNet
   os.environ['MXNET_USE_TENSORRT'] = '0'
   executor = sym.simple_bind(ctx=mx.gpu(0), data=batch_shape, grad_req='null', force_rebind=True)
   executor.copy_params_from(arg_params, aux_params)
   input_data = input_data.as_in_context(mx.gpu())
   y= executor.forward(is_train=False, data=input_data)
   print (y[0].asnumpy())
   
   # Execute with TensorRT
   os.environ['MXNET_USE_TENSORRT'] = '1'
   arg_params.update(aux_params)
   all_params = dict([(k, v.as_in_context(mx.gpu(0))) for k, v in arg_params.items()])
   executor = mx.contrib.tensorrt.tensorrt_bind(sym, ctx=mx.gpu(0), all_params=all_params,
                                                data=batch_shape, grad_req='null', force_rebind=True)
   
   y = executor.forward(is_train=False, data=input_data)
   print (y[0].asnumpy())
   ```
   The TensorRT version either delivers NaN or values in the order or ^+30. 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services