You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/04/29 05:17:38 UTC
[GitHub] [tvm] apivovarov opened a new pull request, #11178: [TRT] Add check to use setBindingDimensions if TRT>=6.0.1
apivovarov opened a new pull request, #11178:
URL: https://github.com/apache/tvm/pull/11178
TRT Dynamic batching with `use_implicit_batch=False` uses `nvinfer1::IExecutionContext::setBindingDimensions()` which is not available before TRT 6.0.1.
This PR adds TRT version check around the code block which uses `setBindingDimensions()`.
Related PR: https://github.com/apache/tvm/pull/8461
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] apivovarov commented on a diff in pull request #11178: [TRT] Add check to use setBindingDimensions if TRT>=6.0.1
Posted by GitBox <gi...@apache.org>.
apivovarov commented on code in PR #11178:
URL: https://github.com/apache/tvm/pull/11178#discussion_r863050743
##########
src/runtime/contrib/tensorrt/tensorrt_runtime.cc:
##########
@@ -163,12 +163,14 @@ class TensorRTRuntime : public JSONRuntimeBase {
const std::string name = nodes_[nid].GetOpName() + "_" + std::to_string(j);
int binding_index = engine->getBindingIndex(name.c_str());
ICHECK_NE(binding_index, -1);
+#if TRT_VERSION_GE(6, 0, 1)
Review Comment:
Below this code on [line 215](https://github.com/apache/tvm/blob/ceda892f081ceb39457394916a18f6e7045a1a8e/src/runtime/contrib/tensorrt/tensorrt_runtime.cc#L215) we just ignore `use_implicit_batch_` flag if TRT <6.0.1.
This PR applies the same logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] apivovarov commented on pull request #11178: [TRT] Add check to use setBindingDimensions if TRT>=6.0.1
Posted by GitBox <gi...@apache.org>.
apivovarov commented on PR #11178:
URL: https://github.com/apache/tvm/pull/11178#issuecomment-1115168476
Below this code on [line 215](https://github.com/apache/tvm/blob/ceda892f081ceb39457394916a18f6e7045a1a8e/src/runtime/contrib/tensorrt/tensorrt_runtime.cc#L215) we just ignore `use_implicit_batch_` flag if TRT <6.0.1.
This PR applies the same logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on pull request #11178: [TRT] Add check to use setBindingDimensions if TRT>=6.0.1
Posted by GitBox <gi...@apache.org>.
comaniac commented on PR #11178:
URL: https://github.com/apache/tvm/pull/11178#issuecomment-1115214117
Thanks @apivovarov
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] apivovarov commented on pull request #11178: [TRT] Add check to use setBindingDimensions if TRT>=6.0.1
Posted by GitBox <gi...@apache.org>.
apivovarov commented on PR #11178:
URL: https://github.com/apache/tvm/pull/11178#issuecomment-1115146456
@comaniac @ziyu-guo This fix it is similar to #11154. Can you have a look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] apivovarov commented on a diff in pull request #11178: [TRT] Add check to use setBindingDimensions if TRT>=6.0.1
Posted by GitBox <gi...@apache.org>.
apivovarov commented on code in PR #11178:
URL: https://github.com/apache/tvm/pull/11178#discussion_r863050743
##########
src/runtime/contrib/tensorrt/tensorrt_runtime.cc:
##########
@@ -163,12 +163,14 @@ class TensorRTRuntime : public JSONRuntimeBase {
const std::string name = nodes_[nid].GetOpName() + "_" + std::to_string(j);
int binding_index = engine->getBindingIndex(name.c_str());
ICHECK_NE(binding_index, -1);
+#if TRT_VERSION_GE(6, 0, 1)
Review Comment:
Below this code on [line 215](https://github.com/apache/tvm/blob/ceda892f081ceb39457394916a18f6e7045a1a8e/src/runtime/contrib/tensorrt/tensorrt_runtime.cc#L215) we just ignore use_implicit_batch_ flag if TRT <6.0.1.
This PR applies the same logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac merged pull request #11178: [TRT] Add check to use setBindingDimensions if TRT>=6.0.1
Posted by GitBox <gi...@apache.org>.
comaniac merged PR #11178:
URL: https://github.com/apache/tvm/pull/11178
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on a diff in pull request #11178: [TRT] Add check to use setBindingDimensions if TRT>=6.0.1
Posted by GitBox <gi...@apache.org>.
comaniac commented on code in PR #11178:
URL: https://github.com/apache/tvm/pull/11178#discussion_r863044761
##########
src/runtime/contrib/tensorrt/tensorrt_runtime.cc:
##########
@@ -163,12 +163,14 @@ class TensorRTRuntime : public JSONRuntimeBase {
const std::string name = nodes_[nid].GetOpName() + "_" + std::to_string(j);
int binding_index = engine->getBindingIndex(name.c_str());
ICHECK_NE(binding_index, -1);
+#if TRT_VERSION_GE(6, 0, 1)
Review Comment:
Do we need a warning or something when TRT < 6.0.1 and `use_implicit_batch_ == False`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on a diff in pull request #11178: [TRT] Add check to use setBindingDimensions if TRT>=6.0.1
Posted by GitBox <gi...@apache.org>.
comaniac commented on code in PR #11178:
URL: https://github.com/apache/tvm/pull/11178#discussion_r863075323
##########
src/runtime/contrib/tensorrt/tensorrt_runtime.cc:
##########
@@ -163,12 +163,14 @@ class TensorRTRuntime : public JSONRuntimeBase {
const std::string name = nodes_[nid].GetOpName() + "_" + std::to_string(j);
int binding_index = engine->getBindingIndex(name.c_str());
ICHECK_NE(binding_index, -1);
+#if TRT_VERSION_GE(6, 0, 1)
Review Comment:
I see. Make sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org