[GitHub] mkolod opened a new pull request #11325: Added TensorRT runtime integration

[GitBox <gi...@apache.org>]

mkolod opened a new pull request #11325: Added TensorRT runtime integration
URL: https://github.com/apache/incubator-mxnet/pull/11325
 
 
   ## Description ##
   This PR introduces runtime integration of TensorRT into MxNet, in order to accelerate inference. 
   
   ## Checklist ##
   ### Essentials ###
   Please feel free to remove inapplicable items for your PR.
   - [x] Test coverage:
     - the end-to-end application of the graph partitioner, NNVM-to-ONNX converer, and TensorRT inference is covered by a test (LeNet-5 on MNIST). The intermediate methods aren't tested, but end-to-end is. 
   - [ ] CI build updates
     - There exists a Dockerfile that reproduces the build ([link](https://github.com/mkolod/incubator-mxnet/blob/tensorrt_integration/ci/docker/Dockerfile.build.ubuntu_gpu_tensorrt)), but it's not yet enabled for running with current CI. This will come in an update to this PR. For now I'm soliciting feedback regarding the existing code, I can update the CI component in the meantime
   - [ ] Code is well-documented: 
   - Code is fairly documented, but more comments will be added when the first PR is under code review.
   - A [README](https://github.com/mkolod/incubator-mxnet/blob/tensorrt_integration/docs/api/python/contrib/tensorrt.md) discusses usage
   
   ### Changes ###
   - [x] NNVM graph partitioner for TensorRT graph compatibility
   - [x] NNVM-to-ONNX converter (for common layers such as convolution, batchnorm, pooling, fully-connected, etc.)
   - [x] Git submodule to support the [ONNX-to-TensorRT](https://github.com/onnx/onnx-tensorrt) graph converter  
   - [x] Makefile updates to support linking against ONNX, Protobuf (for ONNX) and TensorRT
   - [x] Code guards to only enable the TensorRT-inclusive build when USE_TENSORRT flag is added to config.mk
   - [x] End-to-end unit test for LeNet-5 on MNIST
   - [x] `example/image_classification/tensorrt` example with ResNet-50 on CIFAR-10
   
   ## Comments ##
   - The TensorRT 3.0 package only works with CUDA up to 9.0. TensorRT 4.0 is in RC and will soon be GA, and it will support CUDA 9.1 onwards. The build was tested with both TensorRT 3.0 GA and 4.0 RC, but won't be updated to TensorRT 4.0 until it becomes GA.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services