You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/02/08 20:46:35 UTC
[GitHub] [incubator-tvm] kumasento opened a new pull request #4847: Use
dummy func when no lowered_funcs exists in Relay mod
kumasento opened a new pull request #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847
This PR implements the dummy function idea as mentioned in #4748 - when the whole Relay module is optimized to empty, we can insert a dummy operator that allows TVM to still produce a library.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-591705619
Yes, I will do that very soon, was trapped at a conference. Sorry for the
delay.
On Wed, 26 Feb 2020 at 05:00, Zhao Wu <no...@github.com> wrote:
> @kumasento <https://github.com/kumasento> @FrozenGene
> <https://github.com/FrozenGene> can you followup on this PR?
>
> Sure. I am waiting @kumasento <https://github.com/kumasento> 's new
> implementation as comments said.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/incubator-tvm/pull/4847?email_source=notifications&email_token=ACC42R7WHOV6YFUXC32V4QDREXZPNA5CNFSM4KR43YZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM6ZTZA#issuecomment-591239652>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ACC42R6SLTCAU4RARSUMEOLREXZPNANCNFSM4KR43YZQ>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-591235513
@kumasento @FrozenGene can you followup on this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on a change in pull request
#4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on a change in pull request #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r386604910
##########
File path: src/target/llvm/llvm_module.cc
##########
@@ -356,6 +356,40 @@ TVM_REGISTER_GLOBAL("codegen.build_llvm")
*rv = runtime::Module(n);
});
+TVM_REGISTER_GLOBAL("codegen.LLVMModuleCreate")
+.set_body([](TVMArgs args, TVMRetValue *rv) {
+ auto n = make_object<LLVMModuleNode>();
+
+ // parse target triple from the first argument
+ auto target = args[0].operator std::string();
+ std::string triple, mcpu, mattr;
+ llvm::TargetOptions opt;
+ ParseLLVMTargetOptions(target, &triple, &mcpu, &mattr, &opt);
+
+ // create a default data layout
+ auto tm = GetLLVMTargetMachine(target);
+ llvm::DataLayout layout(tm->createDataLayout());
+
+ // initialize an IR code snippet from a simple template
+ std::string ir_str;
+ ir_str += "target triple = \"" + triple + "\"\n";
+ ir_str += "target datalayout = \"" + layout.getStringRepresentation() + "\"";
+
+ // use parseIR to create a LLVM Module.
+ auto ctx = std::make_shared<llvm::LLVMContext>();
+ llvm::SMDiagnostic err;
+ auto mem_buf = llvm::MemoryBuffer::getMemBuffer(ir_str);
+ auto module = llvm::parseIR(mem_buf->getMemBufferRef(), err, *ctx);
Review comment:
@FrozenGene Thank you very much! It looks much better. I've updated my PR accordingly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] zhiics commented on a change in pull request #4847:
Use dummy func when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
zhiics commented on a change in pull request #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r377923131
##########
File path: src/relay/backend/build_module.cc
##########
@@ -438,13 +439,14 @@ class RelayBuildModule : public runtime::ModuleNode {
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
if (lowered_funcs.size() == 0) {
- LOG(WARNING) << "no lowered funcs exist in the compiled module";
+ ret_.mod = tvm::codegen::CSourceModuleCreate(std::string(), std::string());
Review comment:
just `codegen::CSourceModuleCreate("", "")` ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-585462552
The reason that LLVMModule simplifies compilation is because it remembers the correct target triple. We can try to enhance the code by introducing an Empty LLVM module(with the correct target) when it is available(so that Q2 can be resolved) and fallback to the CSourceModule.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #4847: Use dummy func
when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-584305730
cc @zhiics @FrozenGene
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on a change in pull request
#4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on a change in pull request #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r380682757
##########
File path: src/relay/backend/build_module.cc
##########
@@ -437,28 +441,51 @@ class RelayBuildModule : public runtime::ModuleNode {
ret_.params = graph_codegen_->GetParams();
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
+
+ // When there is no lowered_funcs due to reasons such as optimization,
+ // we first try to generate a dummy one if the target host is "llvm".
if (lowered_funcs.size() == 0) {
- LOG(WARNING) << "no lowered funcs exist in the compiled module";
+ // Decide first the target host
+ Target target_host_val = target_host_;
+ if (!target_host_.defined()) {
+ for (const auto &it : targets_) {
+ if (it.second->device_type == kDLCPU) {
+ target_host_val = it.second;
+ break;
+ }
+ }
+ }
+
+ // If no target_host has been set, we choose a default one, which is
+ // llvm if "codegen.build_llvm" is accessible.
+ const runtime::PackedFunc* pf = runtime::Registry::Get("codegen.build_llvm");
+ if (!target_host_val.defined())
+ target_host_val = (pf != nullptr) ? target::llvm() : target::stackvm();
+
+ if (target_host_val.defined() && target_host_val->target_name == "llvm")
+ lowered_funcs.Set(
+ target_host_val->str(),
+ Array<LoweredFunc>({
+ MakeAPI(EvaluateNode::make(0), "__dummy__", Array<ObjectRef>(), 0, false) }));
Review comment:
This sounds like a better way to go. I can try this out later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene edited a comment on issue #4847: Return
empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
FrozenGene edited a comment on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-585527191
> I did try just commenting out the assert and it seems to work. However, I then ran into a new problem... Because what you produce is ostensibly a 'c source' file (even if it's empty), all the external modules are also exported as c files. This results in having to do a proper compile when you try and merge all the modules together to produce a shared library. If one of your external modules happens to be a binary artifact that's 150MB (my case), this compile uses an incredible amount of RAM and takes a long time to complete. I didn't get this problem with your first solution as it used an LLVMModule (I think?) which results in the external modules becoming object files so all you need to do is link, not perform a full compilation.
This is because our new `ModulePackImportsToLLVM` simplifies compilation and generate object file directly, it is designed for large artifact like your case.
> The reason that LLVMModule simplifies compilation is because it remembers the correct target triple. We can try to enhance the code by introducing an Empty LLVM module(with the correct target) when it is available(so that Q2 can be resolved) and fallback to the CSourceModule.
cc @FrozenGene who created the mechanism for the binary exporting
I agree we could create empty llvm module if we have llvm target support, then we could fallback to CSourceModule. We could refer logic here:
```cpp
runtime::Module build(const Map<Target, Array<LoweredFunc>>& inputs,
const Target& target_host,
const BuildConfig& config)
Target target_host_val = target_host;
if (!target_host.defined()) {
for (const auto& it : inputs) {
if (it.first->device_type == kDLCPU) {
target_host_val = it.first;
break;
}
}
}
if (!target_host_val.defined()) {
target_host_val = DefaultTargetHost(target_host_val);
}
```
Here, we have logic to detect and extract the correct target host. According to this, we could know whether we have LLVM (with correct target).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-600251067
Thanks @trevor-m
@FrozenGene sorry for bothering you but does this ring a bell?
I feel it is weird that `GetFunction` won't return an invalid/empty value directly when the function name cannot be found. I'm wondering why `LazyJIT` should be called in this scenario?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] mbarrett97 commented on a change in pull request
#4847: Use dummy func when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
mbarrett97 commented on a change in pull request #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r377290273
##########
File path: src/relay/backend/build_module.cc
##########
@@ -438,13 +442,19 @@ class RelayBuildModule : public runtime::ModuleNode {
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
if (lowered_funcs.size() == 0) {
- LOG(WARNING) << "no lowered funcs exist in the compiled module";
- } else {
- ret_.mod = tvm::build(
- lowered_funcs,
- target_host_,
- BuildConfig::Current());
+ LOG(WARNING) << "No lowered funcs exist in the compiled module, "
Review comment:
Do we need to retain this warning? With external codegen, having no lowered funcs can be a perfectly normal mode of operation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] zhiics commented on a change in pull request #4847:
Use dummy func when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
zhiics commented on a change in pull request #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r377926439
##########
File path: src/relay/backend/build_module.cc
##########
@@ -438,13 +439,14 @@ class RelayBuildModule : public runtime::ModuleNode {
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
if (lowered_funcs.size() == 0) {
- LOG(WARNING) << "no lowered funcs exist in the compiled module";
+ ret_.mod = tvm::codegen::CSourceModuleCreate("", "");
Review comment:
You can force push so that your previous CI could be terminated earlier.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on a change in pull request
#4847: Use dummy func when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on a change in pull request #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r377927901
##########
File path: src/relay/backend/build_module.cc
##########
@@ -438,13 +439,14 @@ class RelayBuildModule : public runtime::ModuleNode {
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
if (lowered_funcs.size() == 0) {
- LOG(WARNING) << "no lowered funcs exist in the compiled module";
+ ret_.mod = tvm::codegen::CSourceModuleCreate("", "");
Review comment:
Sure thing, just added that
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-591239652
> @kumasento @FrozenGene can you followup on this PR?
Sure. I am waiting @kumasento 's new implementation as comments said.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] zhiics edited a comment on issue #4847: Return
empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
zhiics edited a comment on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-585464161
For Q2, a simpler way would be feeding the first argument of CSourceModuleCreate with ";" instead of empty string. But LLVM module should be faster for compilation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-600394618
> Thanks @trevor-m
>
> @FrozenGene sorry for bothering you but does this ring a bell?
>
> I feel it is weird that `GetFunction` won't return an invalid/empty value directly when the function name cannot be found. I'm wondering why `LazyJIT` should be called in this scenario?
Please see: https://github.com/apache/incubator-tvm/pull/4847/files#diff-8baddb83a9684e8373691bb48a946900R469-R474 When entire program is offloaded, I find previous logic will
```cpp
// Execute the whole module using external runtime.
ret_.mod = ext_mods[0];
```
However, current logic we will
```cpp
// Import all external runtime modules.
for (const auto& it : ext_mods)
ret_.mod.Import(it);
```
Could you double check the logic is the same as previous? Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-585454809
@mbarrett97 Thank you for your detailed explanation! I kind of understand what's going here.
For Q1, I have updated the logic involving external module importing. It now always import external modules into `ret_.mod` since it is always there.
For Q2, I'm afraid that I don't have a better idea for that. Maybe @tqchen @zhiics @FrozenGene can have an input on this? Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] mbarrett97 commented on issue #4847: Use dummy func
when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
mbarrett97 commented on issue #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-585180105
I don't think the empty CSourceModule method works. There's a check in source_module.cc that fails when you try and create one with an empty string.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on issue #4847: Use dummy func
when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on issue #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-585197212
> I don't think the empty CSourceModule method works. There's a check in source_module.cc that fails when you try and create one with an empty string.
Hi @mbarrett97 thanks for your comment. Currently I haven't met such an issue while testing. Would you mind letting me know which assertion you were referring to?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] trevor-m edited a comment on issue #4847: Return
empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
trevor-m edited a comment on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-600221459
HI @kumasento the graph runtime is trying to get my external codegen functions from the empty LLVM module.
Stack trace shows the segfault is coming from LLVMModuleNode, so I added a print statement after [this line](https://github.com/apache/incubator-tvm/blob/master/src/target/llvm/llvm_module.cc#L59) which showed that GetFunction() was called on the LLVM module with name=`tensorrt_0` which should be going to my external codegen module instead.
Stack trace:
```
[18:07:04] /data/neo-ai-tvm/src/target/llvm/llvm_module.cc:60: LLVMModuleNode::GetFunction() func name: tensorrt_29
[bt] (0) /home/ubuntu/anaconda3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2e6b140) [0x7f3b510c0140]
[bt] (1) /lib/x86_64-linux-gnu/libc.so.6(+0x354b0) [0x7f3bdd7134b0]
[bt] (2) /lib/x86_64-linux-gnu/libc.so.6(strlen+0x26) [0x7f3bdd769746]
[bt] (3) /data/neo-ai-tvm/build/libtvm.so(tvm::codegen::LLVMModuleNode::LazyInitJIT()+0x894) [0x7f3bcca32ad4]
[bt] (4) /data/neo-ai-tvm/build/libtvm.so(tvm::codegen::LLVMModuleNode::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)+0x488) [0x7f3bcca334b8]
[bt] (5) /data/neo-ai-tvm/build/libtvm.so(tvm::runtime::ModuleNode::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)+0x45) [0x7f3bcca757f5]
[bt] (6) /data/neo-ai-tvm/build/libtvm.so(tvm::runtime::GraphRuntime::CreateTVMOp(tvm::runtime::TVMOpParam const&, std::vector<DLTensor, std::allocator<DLTensor> > const&, unsigned long)+0x4d5) [0x7f3bccad8e05]
[bt] (7) /data/neo-ai-tvm/build/libtvm.so(tvm::runtime::GraphRuntime::SetupOpExecs()+0x661) [0x7f3bccadb0d1]
[bt] (8) /data/neo-ai-tvm/build/libtvm.so(tvm::runtime::GraphRuntime::Init(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::Module, std::vector<DLContext, std::allocator<DLContext> > const&)+0x260) [0x7f3bccadcf60]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-585438528
Hi @mbarrett97
Sorry I have got some questions for the case you've mentioned.
1. As you suggested, the assertion for `lowered_funcs.size()` in the branch assuming that external modules exist should be raised in the current implementation, but what is the expected behaviour when we actually use `CSourceModule` with an empty string? Should we simply remove this assertion since `ret_.mod` exists anyway and won't affect external module importing?
2. This line of code:
https://github.com/apache/incubator-tvm/blob/a5661611472c8e92b20bbe4d074333b8183f2878/src/target/source/source_module.cc#L101
is located in a `SaveToFile` function, which is not called frequently, and the single use case I've noticed:
https://github.com/apache/incubator-tvm/blob/fc1a1d83691f7229e25a37befdcd3ad9c3d430b3/tests/cpp/utvm_runtime_standalone_test.cc#L100
only exists in test and doesn't trigger that assertsion since `fmt` is `"o"`. Even if `fmt` is `"cc"`, assuming no such assertion and the string is empty, there should not be any exception caused (at the code level, I'm not sure whether functionally it is expected). So do you think we could simply remove that empty string checker?
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-602022364
> Hi @FrozenGene
>
> Thanks for your explanation. I do have a question about this part, hope you won't mind:
>
> 1. Before this PR, we replace `ret_.mod` with `ext_mods[0]` when there is no `lowered_funcs`, which is sensible since no `ret_.mod` is available when `lowered_funcs` does not exist.
> 2. Now we will always create a `ret_.mod`. What would the logic be if there is only one external module (the condition for `ret_.mod` replacement)? Should we do replacement or import?
>
> Also @trevor-m would you mind sending me a minimal workable example? I would like to do the tracing myself as well.
>
> Thanks
I think we should be `ret_.mod` replacement as previous pr. @zhiics do the external codegen part. He could answer it more authority. @zhiics Could you help to answer this question?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-597588778
@mbaret @zhiics @tqchen could take a look another round.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-602020720
Hi @FrozenGene
Thanks for your explanation. I do have a question about this part, hope you won't mind:
1. Before this PR, we replace `ret_.mod` with `ext_mods[0]` when there is no `lowered_funcs`, which is sensible since no `ret_.mod` is available when `lowered_funcs` does not exist.
2. Now we will always create a `ret_.mod`. What would the logic be if there is only one external module (the condition for `ret_.mod` replacement)? Should we do replacement or import?
Also @trevor-m would you mind sending me a minimal workable example? I would like to do the tracing myself as well.
Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] trevor-m edited a comment on issue #4847: Return
empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
trevor-m edited a comment on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-600221459
HI @kumasento the graph runtime is trying to get my external codegen functions from the empty LLVM module.
Stack trace shows the segfault is coming from LLVMModuleNode, so I added a print statement after [this line](https://github.com/apache/incubator-tvm/blob/master/src/target/llvm/llvm_module.cc#L59) which showed that GetFunction() was called on the LLVM module with name=`tensorrt_29` which should be going to my external codegen module instead.
Stack trace:
```
[18:07:04] /data/neo-ai-tvm/src/target/llvm/llvm_module.cc:60: LLVMModuleNode::GetFunction() func name: tensorrt_29
[bt] (0) /home/ubuntu/anaconda3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2e6b140) [0x7f3b510c0140]
[bt] (1) /lib/x86_64-linux-gnu/libc.so.6(+0x354b0) [0x7f3bdd7134b0]
[bt] (2) /lib/x86_64-linux-gnu/libc.so.6(strlen+0x26) [0x7f3bdd769746]
[bt] (3) /data/neo-ai-tvm/build/libtvm.so(tvm::codegen::LLVMModuleNode::LazyInitJIT()+0x894) [0x7f3bcca32ad4]
[bt] (4) /data/neo-ai-tvm/build/libtvm.so(tvm::codegen::LLVMModuleNode::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)+0x488) [0x7f3bcca334b8]
[bt] (5) /data/neo-ai-tvm/build/libtvm.so(tvm::runtime::ModuleNode::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)+0x45) [0x7f3bcca757f5]
[bt] (6) /data/neo-ai-tvm/build/libtvm.so(tvm::runtime::GraphRuntime::CreateTVMOp(tvm::runtime::TVMOpParam const&, std::vector<DLTensor, std::allocator<DLTensor> > const&, unsigned long)+0x4d5) [0x7f3bccad8e05]
[bt] (7) /data/neo-ai-tvm/build/libtvm.so(tvm::runtime::GraphRuntime::SetupOpExecs()+0x661) [0x7f3bccadb0d1]
[bt] (8) /data/neo-ai-tvm/build/libtvm.so(tvm::runtime::GraphRuntime::Init(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::Module, std::vector<DLContext, std::allocator<DLContext> > const&)+0x260) [0x7f3bccadcf60]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] zhiics commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
zhiics commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-585464161
For Q2, a simpler would be feeding it with ";" instead of empty string. But LLVM module should be faster for compilation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on a change in pull request
#4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on a change in pull request #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r386479510
##########
File path: src/target/llvm/llvm_module.cc
##########
@@ -356,6 +356,40 @@ TVM_REGISTER_GLOBAL("codegen.build_llvm")
*rv = runtime::Module(n);
});
+TVM_REGISTER_GLOBAL("codegen.LLVMModuleCreate")
+.set_body([](TVMArgs args, TVMRetValue *rv) {
+ auto n = make_object<LLVMModuleNode>();
+
+ // parse target triple from the first argument
+ auto target = args[0].operator std::string();
+ std::string triple, mcpu, mattr;
+ llvm::TargetOptions opt;
+ ParseLLVMTargetOptions(target, &triple, &mcpu, &mattr, &opt);
+
+ // create a default data layout
+ auto tm = GetLLVMTargetMachine(target);
+ llvm::DataLayout layout(tm->createDataLayout());
+
+ // initialize an IR code snippet from a simple template
+ std::string ir_str;
+ ir_str += "target triple = \"" + triple + "\"\n";
+ ir_str += "target datalayout = \"" + layout.getStringRepresentation() + "\"";
+
+ // use parseIR to create a LLVM Module.
+ auto ctx = std::make_shared<llvm::LLVMContext>();
+ llvm::SMDiagnostic err;
+ auto mem_buf = llvm::MemoryBuffer::getMemBuffer(ir_str);
+ auto module = llvm::parseIR(mem_buf->getMemBufferRef(), err, *ctx);
Review comment:
I think it is not one elegant solution to create LLVM module using ir string concatenation. You could use the api like this:
```cpp
auto triple = tm->getTargetTriple();
auto ctx = std::make_shared<llvm::LLVMContext>();
std::string module_name = "dummy";
std::unique_ptr<llvm::Module> module(new llvm::Module(module_name, *ctx));
module->setTargetTriple(triple.str());
module->setDataLayout(tm->createDataLayout());
```
Does this work well too? You could refer more detail in my previous implementation of codegen_blob.cc
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on a change in pull request
#4847: Use dummy func when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on a change in pull request #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r377444157
##########
File path: src/relay/backend/build_module.cc
##########
@@ -438,13 +442,19 @@ class RelayBuildModule : public runtime::ModuleNode {
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
if (lowered_funcs.size() == 0) {
- LOG(WARNING) << "no lowered funcs exist in the compiled module";
- } else {
- ret_.mod = tvm::build(
- lowered_funcs,
- target_host_,
- BuildConfig::Current());
+ LOG(WARNING) << "No lowered funcs exist in the compiled module, "
+ << "a dummy function \"__dummy__\" will be created.";
+ Stmt body = EvaluateNode::make(0);
+ Array<ObjectRef> api_args;
+ auto dummy_func = MakeAPI(body, "__dummy__", api_args, 0, false);
+ lowered_funcs.Set("llvm", Array<LoweredFunc>({dummy_func}));
Review comment:
I think should set `target_host_`. Even we have LLVM support, it is not correct too, imagine our target host is ARM.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] zhiics commented on a change in pull request #4847:
Use dummy func when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
zhiics commented on a change in pull request #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r377926067
##########
File path: src/relay/backend/build_module.cc
##########
@@ -438,13 +439,14 @@ class RelayBuildModule : public runtime::ModuleNode {
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
if (lowered_funcs.size() == 0) {
- LOG(WARNING) << "no lowered funcs exist in the compiled module";
+ ret_.mod = tvm::codegen::CSourceModuleCreate("", "");
Review comment:
Sorry for the back and forth. Could you please add a comment here so that ppl would know what we are doing here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-586005848
Thank you for your valuable suggestions @tqchen @zhiics @FrozenGene !
I now changed the logic to try `LLVMModule`, and if the `target_host` is not `llvm`, we fallback to `CSourceModule`. I use the code snippet that @FrozenGene suggested to decide the target host. But I still need to bring back the dummy function idea, otherwise, I cannot find a way to generate a `LLVMModule`, which requires `lowered_funcs` to be non-empty.
@mbarrett97 Please see whether this version suits your scenario. Any other suggestions are also welcomed!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on a change in pull request
#4847: Use dummy func when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on a change in pull request #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r377919071
##########
File path: src/relay/backend/build_module.cc
##########
@@ -438,13 +442,19 @@ class RelayBuildModule : public runtime::ModuleNode {
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
if (lowered_funcs.size() == 0) {
- LOG(WARNING) << "no lowered funcs exist in the compiled module";
- } else {
- ret_.mod = tvm::build(
- lowered_funcs,
- target_host_,
- BuildConfig::Current());
+ LOG(WARNING) << "No lowered funcs exist in the compiled module, "
+ << "a dummy function \"__dummy__\" will be created.";
+ Stmt body = EvaluateNode::make(0);
+ Array<ObjectRef> api_args;
+ auto dummy_func = MakeAPI(body, "__dummy__", api_args, 0, false);
+ lowered_funcs.Set("llvm", Array<LoweredFunc>({dummy_func}));
Review comment:
Thank you guys for your kind comments. We don't need to set target in the latest commit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-603793596
Hi @trevor-m
Thanks for this information. I ran the test and reproduced the bug. I've located that the segfault should be raised from the `create` function of `llvm::EngineBuilder` (this [line](https://github.com/apache/incubator-tvm/blob/master/src/target/llvm/llvm_module.cc#L316)).
Now I'm looking at the internal logic in LLVM to find out what the actual cause is. Please bear with me for 1-2 days.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on a change in pull request
#4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on a change in pull request #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r380467063
##########
File path: src/relay/backend/build_module.cc
##########
@@ -437,28 +441,51 @@ class RelayBuildModule : public runtime::ModuleNode {
ret_.params = graph_codegen_->GetParams();
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
+
+ // When there is no lowered_funcs due to reasons such as optimization,
+ // we first try to generate a dummy one if the target host is "llvm".
if (lowered_funcs.size() == 0) {
- LOG(WARNING) << "no lowered funcs exist in the compiled module";
+ // Decide first the target host
+ Target target_host_val = target_host_;
+ if (!target_host_.defined()) {
+ for (const auto &it : targets_) {
+ if (it.second->device_type == kDLCPU) {
+ target_host_val = it.second;
+ break;
+ }
+ }
+ }
+
+ // If no target_host has been set, we choose a default one, which is
+ // llvm if "codegen.build_llvm" is accessible.
+ const runtime::PackedFunc* pf = runtime::Registry::Get("codegen.build_llvm");
+ if (!target_host_val.defined())
+ target_host_val = (pf != nullptr) ? target::llvm() : target::stackvm();
+
+ if (target_host_val.defined() && target_host_val->target_name == "llvm")
+ lowered_funcs.Set(
+ target_host_val->str(),
+ Array<LoweredFunc>({
+ MakeAPI(EvaluateNode::make(0), "__dummy__", Array<ObjectRef>(), 0, false) }));
Review comment:
Add one `__dummy__` is not the most elegant solution IMO. If we have this LLVM IR:
```
target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-apple-macosx10.14.0"
```
Does this satisfy your need?
If it is, I think we could create one packed function in `llvm_module.cc`. We could create this kind of empty llvm module. Then we could have`ret_.mod = tvm::codegen::LLVMModuleCreate...` like CSourceModuleCreate.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] zhiics edited a comment on issue #4847: Return
empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
zhiics edited a comment on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-585464161
For Q2, a simpler way would be feeding it with ";" instead of empty string. But LLVM module should be faster for compilation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-603859603
@trevor-m a tentative fix has been posted in #5146
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on a change in pull request
#4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on a change in pull request #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r379544814
##########
File path: src/relay/backend/build_module.cc
##########
@@ -437,28 +441,50 @@ class RelayBuildModule : public runtime::ModuleNode {
ret_.params = graph_codegen_->GetParams();
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
+
+ // When there is no lowered_funcs due to reasons such as optimization,
+ // we first try to generate a dummy one if the target host is "llvm".
+ if (lowered_funcs.size() == 0) {
+ // Decide first the target host
+ Target target_host_val = target_host_;
+ if (!target_host_.defined()) {
+ for (const auto &it: targets_) {
+ if (it.second->device_type == kDLCPU) {
+ target_host_val = it.second;
+ break;
+ }
+ }
+ }
+
+ // If no target_host has been set, we choose a default one, which is
+ // llvm if "codegen.build_llvm" is accessible.
+ const runtime::PackedFunc* pf = runtime::Registry::Get("codegen.build_llvm");
+ if (!target_host_val.defined())
+ target_host_val = (pf != nullptr) ? target::llvm() : target::stackvm();
+
+ if (target_host_val.defined() && target_host_val->str() == "llvm")
Review comment:
Thanks @FrozenGene, this is indeed problematic. I changed the condition to `target_host_val->target_name == "llvm"`. Hope that's OK.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-596871412
ping @FrozenGene please followup
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] trevor-m commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
trevor-m commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-603369453
> I think changing it to a llvm module and import all submodules is okay. Now if you only have an external module. You will need to create a llvm module first and them import the external module to it.
>
> Stepping into llvm module to find the symbol is not wrong because we will always try to find the symbol from the host module first. If it is not found, we will then try to check each imported module. See the code here:
>
> https://github.com/apache/incubator-tvm/blob/050f2bde2c694af9b5569ca954ca041c3767787b/src/runtime/module.cc#L65
>
> A minimal example to reproduce this and track the root cause would be more helpful.
You can reproduce this by running `test_extern_dnnl()` after commenting out this line: https://github.com/apache/incubator-tvm/blob/master/tests/python/relay/test_pass_partition_graph.py#L203
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-585243629
@mbarrett97 Thanks, I just noticed that the base of my PR is not the latest commit. I will update it soon.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] trevor-m commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
trevor-m commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-600200202
This commit is causing segfaults when my entire relay program is offloaded to an external codegen.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] trevor-m commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
trevor-m commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-600221459
HI @kumasento the graph runtime is trying to get my external codegen functions from the empty LLVM module.
Stack trace shows the segfault is coming from LLVMModuleNode, so I added a print statement after [this line](https://github.com/apache/incubator-tvm/blob/master/src/target/llvm/llvm_module.cc#L59) which showed that GetFunction() was called on the LLVM module for my function `tensorrt_0` which should be going to my external codegen module instead.
Stack trace:
```
[bt] (0) /home/ubuntu/anaconda3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2e6b140) [0x7f3b510c0140]
[bt] (1) /lib/x86_64-linux-gnu/libc.so.6(+0x354b0) [0x7f3bdd7134b0]
[bt] (2) /lib/x86_64-linux-gnu/libc.so.6(strlen+0x26) [0x7f3bdd769746]
[bt] (3) /data/neo-ai-tvm/build/libtvm.so(tvm::codegen::LLVMModuleNode::LazyInitJIT()+0x894) [0x7f3bcca32ad4]
[bt] (4) /data/neo-ai-tvm/build/libtvm.so(tvm::codegen::LLVMModuleNode::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::ObjectPtr<tvm::runtime::Object> const&)+0x488) [0x7f3bcca334b8]
[bt] (5) /data/neo-ai-tvm/build/libtvm.so(tvm::runtime::ModuleNode::GetFunction(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)+0x45) [0x7f3bcca757f5]
[bt] (6) /data/neo-ai-tvm/build/libtvm.so(tvm::runtime::GraphRuntime::CreateTVMOp(tvm::runtime::TVMOpParam const&, std::vector<DLTensor, std::allocator<DLTensor> > const&, unsigned long)+0x4d5) [0x7f3bccad8e05]
[bt] (7) /data/neo-ai-tvm/build/libtvm.so(tvm::runtime::GraphRuntime::SetupOpExecs()+0x661) [0x7f3bccadb0d1]
[bt] (8) /data/neo-ai-tvm/build/libtvm.so(tvm::runtime::GraphRuntime::Init(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tvm::runtime::Module, std::vector<DLContext, std::allocator<DLContext> > const&)+0x260) [0x7f3bccadcf60]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #4847: Use dummy func
when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-584384721
I agree that perhaps an empty module provides useful middle ground. The closest thing so far might be CSourceModule with an empty string https://github.com/apache/incubator-tvm/blob/master/src/target/source/source_module.cc#L190
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on a change in pull request
#4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on a change in pull request #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r380467063
##########
File path: src/relay/backend/build_module.cc
##########
@@ -437,28 +441,51 @@ class RelayBuildModule : public runtime::ModuleNode {
ret_.params = graph_codegen_->GetParams();
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
+
+ // When there is no lowered_funcs due to reasons such as optimization,
+ // we first try to generate a dummy one if the target host is "llvm".
if (lowered_funcs.size() == 0) {
- LOG(WARNING) << "no lowered funcs exist in the compiled module";
+ // Decide first the target host
+ Target target_host_val = target_host_;
+ if (!target_host_.defined()) {
+ for (const auto &it : targets_) {
+ if (it.second->device_type == kDLCPU) {
+ target_host_val = it.second;
+ break;
+ }
+ }
+ }
+
+ // If no target_host has been set, we choose a default one, which is
+ // llvm if "codegen.build_llvm" is accessible.
+ const runtime::PackedFunc* pf = runtime::Registry::Get("codegen.build_llvm");
+ if (!target_host_val.defined())
+ target_host_val = (pf != nullptr) ? target::llvm() : target::stackvm();
+
+ if (target_host_val.defined() && target_host_val->target_name == "llvm")
+ lowered_funcs.Set(
+ target_host_val->str(),
+ Array<LoweredFunc>({
+ MakeAPI(EvaluateNode::make(0), "__dummy__", Array<ObjectRef>(), 0, false) }));
Review comment:
Add one `__dummy__` is not elegant solution IMO. If we have this LLVM IR:
```
target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-apple-macosx10.14.0"
```
Does this satisfy your need?
If it is, I think we could create one packed function in `llvm_module.cc`. We could create this kind of empty llvm module. Then we could have`ret_.mod = tvm::codegen::LLVMModuleCreate...` like CSourceModuleCreate.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] mbarrett97 commented on a change in pull request
#4847: Use dummy func when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
mbarrett97 commented on a change in pull request #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r377289530
##########
File path: src/relay/backend/build_module.cc
##########
@@ -438,13 +442,19 @@ class RelayBuildModule : public runtime::ModuleNode {
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
if (lowered_funcs.size() == 0) {
- LOG(WARNING) << "no lowered funcs exist in the compiled module";
- } else {
- ret_.mod = tvm::build(
- lowered_funcs,
- target_host_,
- BuildConfig::Current());
+ LOG(WARNING) << "No lowered funcs exist in the compiled module, "
+ << "a dummy function \"__dummy__\" will be created.";
+ Stmt body = EvaluateNode::make(0);
+ Array<ObjectRef> api_args;
+ auto dummy_func = MakeAPI(body, "__dummy__", api_args, 0, false);
+ lowered_funcs.Set("llvm", Array<LoweredFunc>({dummy_func}));
Review comment:
Is defaulting the LLVM the correct behaviour here (eg. will this fall over if we build without LLVM support)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-585527191
> I did try just commenting out the assert and it seems to work. However, I then ran into a new problem... Because what you produce is ostensibly a 'c source' file (even if it's empty), all the external modules are also exported as c files. This results in having to do a proper compile when you try and merge all the modules together to produce a shared library. If one of your external modules happens to be a binary artifact that's 150MB (my case), this compile uses an incredible amount of RAM and takes a long time to complete. I didn't get this problem with your first solution as it used an LLVMModule (I think?) which results in the external modules becoming object files so all you need to do is link, not perform a full compilation.
This is because our new `ModulePackImportsToLLVM` simplifies complication and generate object file directly, it is designed for large artifact like your case.
> The reason that LLVMModule simplifies compilation is because it remembers the correct target triple. We can try to enhance the code by introducing an Empty LLVM module(with the correct target) when it is available(so that Q2 can be resolved) and fallback to the CSourceModule.
cc @FrozenGene who created the mechanism for the binary exporting
I agree we could create empty llvm module if we have llvm target support, then we could fallback to CSourceModule. We could refer logic here:
```cpp
runtime::Module build(const Map<Target, Array<LoweredFunc>>& inputs,
const Target& target_host,
const BuildConfig& config)
Target target_host_val = target_host;
if (!target_host.defined()) {
for (const auto& it : inputs) {
if (it.first->device_type == kDLCPU) {
target_host_val = it.first;
break;
}
}
}
if (!target_host_val.defined()) {
target_host_val = DefaultTargetHost(target_host_val);
}
```
Here, we have logic to detect and extract the correct target host. According to this, we could know whether we have LLVM (with correct target).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-600211279
Hi Trevor,
Thanks for the info. Can you provide more information about the bug you've
got? Thanks.
On Tue, 17 Mar 2020 at 17:29, Trevor Morris <no...@github.com>
wrote:
> This commit is causing segfaults when my entire relay program is offloaded
> to an external codegen.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/incubator-tvm/pull/4847#issuecomment-600200202>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ACC42R5SF23UYPEF6M5ABT3RH6XPNANCNFSM4KR43YZQ>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] zhiics commented on issue #4847: Use dummy func
when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
zhiics commented on issue #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-584462639
CSourceModule with an empty string looks to me as well. @kumasento could you do that instead of creating a dummy llvm module? Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-599730355
Thanks @kumasento @FrozenGene @mbaret @zhiics This PR is now merged
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on a change in pull request
#4847: Use dummy func when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on a change in pull request #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r377919235
##########
File path: src/relay/backend/build_module.cc
##########
@@ -438,13 +442,19 @@ class RelayBuildModule : public runtime::ModuleNode {
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
if (lowered_funcs.size() == 0) {
- LOG(WARNING) << "no lowered funcs exist in the compiled module";
- } else {
- ret_.mod = tvm::build(
- lowered_funcs,
- target_host_,
- BuildConfig::Current());
+ LOG(WARNING) << "No lowered funcs exist in the compiled module, "
Review comment:
Thanks @mbarrett97 , I've removed that log.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on a change in pull request
#4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on a change in pull request #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r379218842
##########
File path: src/relay/backend/build_module.cc
##########
@@ -437,28 +441,50 @@ class RelayBuildModule : public runtime::ModuleNode {
ret_.params = graph_codegen_->GetParams();
auto lowered_funcs = graph_codegen_->GetLoweredFunc();
+
+ // When there is no lowered_funcs due to reasons such as optimization,
+ // we first try to generate a dummy one if the target host is "llvm".
+ if (lowered_funcs.size() == 0) {
+ // Decide first the target host
+ Target target_host_val = target_host_;
+ if (!target_host_.defined()) {
+ for (const auto &it: targets_) {
+ if (it.second->device_type == kDLCPU) {
+ target_host_val = it.second;
+ break;
+ }
+ }
+ }
+
+ // If no target_host has been set, we choose a default one, which is
+ // llvm if "codegen.build_llvm" is accessible.
+ const runtime::PackedFunc* pf = runtime::Registry::Get("codegen.build_llvm");
+ if (!target_host_val.defined())
+ target_host_val = (pf != nullptr) ? target::llvm() : target::stackvm();
+
+ if (target_host_val.defined() && target_host_val->str() == "llvm")
Review comment:
Please add one test "llvm -device=arm_cpu -mattr=+neon -target=aarch64-linux-gnu" to make sure we get correct llvm target. I doubt your `target_host_val->str() == "llvm"` is not correct logic.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on issue #4847: Use dummy func
when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on issue #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-584869309
Thank you guys for all your kind reviews! @mbarrett97 @FrozenGene @tqchen @zhiics
I've updated this PR to fulfill the following revisions:
1. Using `CSourceModule` with an empty string as the returned module object `ret_.mod`, instead of creating a dummy lowered function.
2. Removed the warning based on @mbarrett97 review.
Now the generated module looks clean and tidy. No redundant dummy function generated and no extra design decisions should be made.
Please let me know if there is anything else you feel should be done. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] mbarrett97 commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
mbarrett97 commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-585449674
I should probably explain my 'test case' :) Apologies if you know this all already. I'm using the external codegen feature which is relatively new. Essentially it allows you to take parts of the Relay graph and run them through a non-TVM compiler. In some cases, we take the entire Relay graph and put it through one of these external compilers resulting in the case where there are no lowered TVM functions. What does get produced though is 'external' runtime modules which correspond to the outputs from these external compilers.
You can see this happening in the BuildRelay function. We call 'GetExternalModules' to get those modules that have been produced by external compilers. But to be able to compile these modules into a shared library, we need a 'host' TVM module. This host module then 'imports' in all the external modules, as you can see here:
https://github.com/apache/incubator-tvm/blob/a5661611472c8e92b20bbe4d074333b8183f2878/src/relay/backend/build_module.cc#L458
However, if there is no host TVM module (which will happen when there are no lowered funcs), then you can't import the external modules into a host module and accordingly we fail to compile the result to a shared library. So this fix is very helpful by making sure that host module always gets created, even if it's empty, so we can always compile to a shared library.
Now addressing your questions. For 1, yes I think we just remove this branch entirely and have it always import the external modules. For 2, I hit the assert when trying to call export_library on the 'lib' output of the build function. I believe SaveToFile is called when doing this export, in particular this line:
https://github.com/apache/incubator-tvm/blob/79cfab00d1a5ae55b3324dae803d5e1389e12bbe/python/tvm/runtime/module.py#L302
I did try just commenting out the assert and it seems to work. However, I then ran into a new problem... Because what you produce is ostensibly a 'c source' file (even if it's empty), all the external modules are also exported as c files. This results in having to do a proper compile when you try and merge all the modules together to produce a shared library. If one of your external modules happens to be a binary artifact that's 150MB (my case), this compile uses an incredible amount of RAM and takes a long time to complete. I didn't get this problem with your first solution as it used an LLVMModule (I think?) which results in the external modules becoming object files so all you need to do is link, not perform a full compilation.
Sorry for the wall of text, if you have any further questions I'll happily answer them. I realise I may not have perfectly explained everything here :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] zhiics commented on a change in pull request #4847:
Use dummy func when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
zhiics commented on a change in pull request #4847: Use dummy func when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r377921458
##########
File path: src/relay/backend/build_module.cc
##########
@@ -28,15 +28,20 @@
#include <tvm/relay/expr.h>
#include <tvm/relay/transform.h>
#include <tvm/relay/qnn/transform.h>
+#include <tvm/tir/ir_pass.h>
Review comment:
remove?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] zhiics commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
zhiics commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-602072692
I think changing it to a llvm module and import all submodules is okay. Now if you only have an external module. You will need to create a llvm module first and them import the external module to it.
Stepping into llvm module to find the symbol is not wrong because we will always try to find the symbol from the host module first. If it is not found, we will then try to check each imported module. See the code here:
https://github.com/apache/incubator-tvm/blob/050f2bde2c694af9b5569ca954ca041c3767787b/src/runtime/module.cc#L65
A minimal example to reproduce this and track the root cause would be more helpful.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen edited a comment on issue #4847: Return
empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
tqchen edited a comment on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-585462552
The reason that LLVMModule simplifies compilation is because it remembers the correct target triple. We can try to enhance the code by introducing an Empty LLVM module(with the correct target) when it is available(so that Q2 can be resolved) and fallback to the CSourceModule.
cc @FrozenGene who created the mechanism for the binary exporting
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on a change in pull request
#4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on a change in pull request #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r390886368
##########
File path: src/target/llvm/llvm_module.cc
##########
@@ -356,6 +356,26 @@ TVM_REGISTER_GLOBAL("codegen.build_llvm")
*rv = runtime::Module(n);
});
+TVM_REGISTER_GLOBAL("codegen.LLVMModuleCreate")
+.set_body([](TVMArgs args, TVMRetValue *rv) {
+ auto n = make_object<LLVMModuleNode>();
+ auto target = args[0].operator std::string();
+ auto module_name = args[1].operator std::string();
+
+ // create a default data layout
Review comment:
Thanks @FrozenGene I've updated the comments based on your comments :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] mbaret commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
mbaret commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-587005445
I've tested the most recent commit with my use-case (aarch64 with offloading to external codegen) and it looks to work well. It's still not a particularly elegant solution but I can't propose a better one.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen merged pull request #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
tqchen merged pull request #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] FrozenGene commented on a change in pull request
#4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on a change in pull request #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#discussion_r390724783
##########
File path: src/target/llvm/llvm_module.cc
##########
@@ -356,6 +356,26 @@ TVM_REGISTER_GLOBAL("codegen.build_llvm")
*rv = runtime::Module(n);
});
+TVM_REGISTER_GLOBAL("codegen.LLVMModuleCreate")
+.set_body([](TVMArgs args, TVMRetValue *rv) {
+ auto n = make_object<LLVMModuleNode>();
+ auto target = args[0].operator std::string();
+ auto module_name = args[1].operator std::string();
+
+ // create a default data layout
Review comment:
Sorry for later response. Minor comment: The logic here doesn't only create default data layout but also create default target triple. We should update the comment.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] kumasento commented on issue #4847: Return empty
CSourceModule when no lowered_funcs exists in Relay mod
Posted by GitBox <gi...@apache.org>.
kumasento commented on issue #4847: Return empty CSourceModule when no lowered_funcs exists in Relay mod
URL: https://github.com/apache/incubator-tvm/pull/4847#issuecomment-592741362
@FrozenGene @tqchen @zhiics @mbaret Would you mind giving another read on this? Now the approach is creating an empty LLVM module instead of using a dummy function. There is also a `LLVMModuleCreate` packed function implemented.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services