You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "jikechao (via GitHub)" <gi...@apache.org> on 2023/04/13 17:16:16 UTC
[GitHub] [tvm] jikechao opened a new issue, #14615: [Bug] cannot allocate memory symbolic tensor shape for T.Any()
jikechao opened a new issue, #14615:
URL: https://github.com/apache/tvm/issues/14615
The ONNX model can be imported into Relay well but crashed when compiled. It throws the following crash messages:
```
Traceback (most recent call last):
....
irmod, params = relay.frontend.from_onnx(model, {'x': (1, 1, 5, 5)}, freeze_params=True)
File "/workplace/software/tvm/tvm/python/tvm/relay/frontend/onnx.py", line 7346, in from_onnx
mod, params = g.from_onnx(graph, opset)
File "/workplace/software/tvm/tvm/python/tvm/relay/frontend/onnx.py", line 6963, in from_onnx
self._construct_nodes(graph)
File "/workplace/software/tvm/tvm/python/tvm/relay/frontend/onnx.py", line 7120, in _construct_nodes
len(node_output), outputs_num, op_name
AssertionError: Number of output mismatch 2 vs 1 in MaxPool.
(tf2) root@R730-3:/share_host/TVMFT/BorrowTests/ONNX/bugs# python 6_sorted.py
Traceback (most recent call last):
File "6_sorted.py", line 11, in <module>
graph, lib, params = relay.build_module.build(irmod, target='llvm -mcpu=core-avx2', params=params)
File "/workplace/software/tvm/tvm/python/tvm/relay/build_module.py", line 372, in build
mod_name=mod_name,
File "/workplace/software/tvm/tvm/python/tvm/relay/build_module.py", line 169, in build
mod_name,
File "/workplace/software/tvm/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 237, in __call__
raise get_last_ffi_error()
tvm._ffi.base.TVMError: Traceback (most recent call last):
25: TVMFuncCall
24: tvm::relay::backend::RelayBuildModule::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#3}::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
23: tvm::relay::backend::RelayBuildModule::Build(tvm::IRModule, tvm::runtime::Array<tvm::Target, void> const&, tvm::Target const&, tvm::relay::Executor const&, tvm::relay::Runtime const&, tvm::WorkspaceMemoryPools const&, tvm::ConstantMemoryPools const&, tvm::runtime::String)
22: tvm::relay::backend::RelayBuildModule::BuildRelay(tvm::IRModule, tvm::runtime::String const&)
21: tvm::relay::backend::ExecutorCodegen::Codegen(tvm::IRModule, tvm::relay::Function const&, tvm::runtime::String)
20: void tvm::relay::backend::ExecutorCodegen::CallFunc<tvm::IRModule, tvm::relay::Function, tvm::runtime::String>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::IRModule, tvm::relay::Function, tvm::runtime::String)
19: tvm::relay::backend::GraphExecutorCodegenModule::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#2}::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const
18: tvm::relay::backend::GraphExecutorCodegen::Codegen(tvm::IRModule, tvm::relay::Function, tvm::runtime::String)
17: tvm::relay::GraphPlanMemory(tvm::relay::Function const&)
16: tvm::relay::StorageAllocator::Plan(tvm::relay::Function const&)
15: tvm::relay::ExprVisitor::VisitExpr(tvm::RelayExpr const&)
14: tvm::relay::transform::DeviceAwareExprVisitor::VisitExpr_(tvm::relay::FunctionNode const*)
13: tvm::relay::StorageAllocaBaseVisitor::DeviceAwareVisitExpr_(tvm::relay::FunctionNode const*)
12: tvm::relay::StorageAllocaBaseVisitor::GetToken(tvm::RelayExpr const&)
11: tvm::relay::ExprVisitor::VisitExpr(tvm::RelayExpr const&)
10: tvm::relay::StorageAllocaBaseVisitor::VisitExpr_(tvm::relay::TupleNode const*)
9: tvm::relay::StorageAllocaBaseVisitor::GetToken(tvm::RelayExpr const&)
8: tvm::relay::ExprVisitor::VisitExpr(tvm::RelayExpr const&)
7: tvm::relay::transform::DeviceAwareExprVisitor::VisitExpr_(tvm::relay::CallNode const*)
6: tvm::relay::StorageAllocator::DeviceAwareVisitExpr_(tvm::relay::CallNode const*)
5: tvm::relay::StorageAllocaBaseVisitor::CreateToken(tvm::RelayExprNode const*, bool)
4: tvm::relay::StorageAllocator::CreateTokenOnDevice(tvm::RelayExprNode const*, tvm::VirtualDevice const&, bool)
3: tvm::relay::StorageAllocator::TokenAllocator::Request(tvm::relay::StorageToken*)
2: tvm::relay::TokenAllocator1D::Request(tvm::relay::StorageToken*)
1: tvm::relay::TokenAllocator1D::GetMemorySize(tvm::relay::StorageToken*)
0: _ZN3tvm7runtime6detail
File "/workplace/software/tvm/tvm/src/relay/backend/token_allocator.cc", line 41
TVMError:
---------------------------------------------------------------
An error occurred during the execution of TVM.
For more information, please see: https://tvm.apache.org/docs/errors.html
---------------------------------------------------------------
Check failed: (pval != nullptr) is false: Cannot allocate memory symbolic tensor shape [T.Any()]
```
Although the shape is `T.Any()`, it's a static graph rather than. Thus this crash is a bug, we should support the T.any() correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo commented on issue #14615: [Bug] cannot allocate memory symbolic tensor shape for T.Any()
Posted by "AndrewZhaoLuo (via GitHub)" <gi...@apache.org>.
AndrewZhaoLuo commented on issue #14615:
URL: https://github.com/apache/tvm/issues/14615#issuecomment-1511794778
With modification the code works for me, let me know if you have any additional issues. If so, feel free to open this issue again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo commented on issue #14615: [Bug] cannot allocate memory symbolic tensor shape for T.Any()
Posted by "AndrewZhaoLuo (via GitHub)" <gi...@apache.org>.
AndrewZhaoLuo commented on issue #14615:
URL: https://github.com/apache/tvm/issues/14615#issuecomment-1511793932
Hey @jikechao , @echuraev is correct in that you need the VM to support dynamic shape.
His code is actually correct except instead of
`vmc = relay.vm.compile(model, target="llvm", params=params)`
you should do
`vmc = relay.vm.compile(irmod, target="llvm", params=params)`
The error here is `model` is the ONNX Serialized model, while the `irmod` is the TVM serialized model (in relay). Compile takes in the relay model instead of onnx model (which is what the error is).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jikechao commented on issue #14615: [Bug] cannot allocate memory symbolic tensor shape for T.Any()
Posted by "jikechao (via GitHub)" <gi...@apache.org>.
jikechao commented on issue #14615:
URL: https://github.com/apache/tvm/issues/14615#issuecomment-1509513559
@echuraev thanks for your comments. Following your suggestion, I tried to locate the TypeError crash but failed due to my limitation of domain knowledge.
@AndrewZhaoLuo Could you give me some suggestions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] cannot allocate memory symbolic tensor shape for T.Any() [tvm]
Posted by "MrJungle1 (via GitHub)" <gi...@apache.org>.
MrJungle1 commented on issue #14615:
URL: https://github.com/apache/tvm/issues/14615#issuecomment-1988749914
@AndrewZhaoLuo @echuraev Hello，After obtaining the vmc, what should I do if I want to generate a .so? It turns out that I can use lib.export_library("model.so", ndk.create_shared)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] echuraev commented on issue #14615: [Bug] cannot allocate memory symbolic tensor shape for T.Any()
Posted by "echuraev (via GitHub)" <gi...@apache.org>.
echuraev commented on issue #14615:
URL: https://github.com/apache/tvm/issues/14615#issuecomment-1508036638
Hi @jikechao! You are trying to compile your model for graph executor, but it doesn't support dynamic shapes. I suppose, you have to use VirtualMachine because it supports dynamic shapes:
```python
import onnx
import tvm
from tvm import relay
from tvm.runtime.vm import VirtualMachine
onnx_model_path = "model_unique_op.onnx"
model = onnx.load(onnx_model_path)
irmod, params = relay.frontend.from_onnx(model, {'X': (6,)}, freeze_params=True)
print(irmod)
with tvm.transform.PassContext(opt_level=3):
vmc = relay.vm.compile(model, target="llvm", params=params)
```
Unfortunately, this code also will fail with the following error:
```
Traceback (most recent call last):
File "script.py", line 12, in <module>
vmc = relay.vm.compile(model, target="llvm", params=params)
File "/home/echuraev/Workspace/OctoML/tvm/python/tvm/relay/backend/vm.py", line 67, in compile
compiler.lower(mod, target, target_host)
File "/home/echuraev/Workspace/OctoML/tvm/python/tvm/relay/backend/vm.py", line 127, in lower
self._lower(mod, raw_targets)
File "/home/echuraev/Workspace/OctoML/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 223, in __call__
values, tcodes, num_args = _make_tvm_args(args, temp_args)
File "/home/echuraev/Workspace/OctoML/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 188, in _make_tvm_args
raise TypeError("Don't know how to handle type %s" % type(arg))
TypeError: Don't know how to handle type <class 'onnx.onnx_ml_pb2.ModelProto'>
```
Probably there is a problem in ONNX importer. I'm not sure. Maybe @AndrewZhaoLuo may know more about that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jikechao commented on issue #14615: [Bug] cannot allocate memory symbolic tensor shape for T.Any()
Posted by "jikechao (via GitHub)" <gi...@apache.org>.
jikechao commented on issue #14615:
URL: https://github.com/apache/tvm/issues/14615#issuecomment-1507811186
@mbrookhart @echuraev Could you help me check this crash?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] echuraev commented on issue #14615: [Bug] cannot allocate memory symbolic tensor shape for T.Any()
Posted by "echuraev (via GitHub)" <gi...@apache.org>.
echuraev commented on issue #14615:
URL: https://github.com/apache/tvm/issues/14615#issuecomment-1511814992
@AndrewZhaoLuo sorry, it was a copy-paste mistake :) Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo closed issue #14615: [Bug] cannot allocate memory symbolic tensor shape for T.Any()
Posted by "AndrewZhaoLuo (via GitHub)" <gi...@apache.org>.
AndrewZhaoLuo closed issue #14615: [Bug] cannot allocate memory symbolic tensor shape for T.Any()
URL: https://github.com/apache/tvm/issues/14615
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org