[GitHub] [tvm] gromero commented on issue #9423: [Bug] microTVM run problem on mimxrt1060_evk (NXP) board

[GitBox <gi...@apache.org>]

gromero commented on issue #9423:
URL: https://github.com/apache/tvm/issues/9423#issuecomment-975998894


   @NataliaTabirca Hi. Thanks for reporting the issue and for creating a simple reproducer.
   
   The issue boils down to a lack of RAM at runtime due to the amount of memory required by your model (mobilenet), as we suspected.
   
   More specifically, the trace you've pasted above is due to `TVMPlatformMemoryAllocate()` failing to allocate memory on Zephyr's heap (i.e. allocation via `k_heap_alloc()` - `TVMPlatformMemoryAllocate()` is wrapper around `k_heap_alloc()`) upon receiving a `tvm::runtime::RPCCode::kDevAllocDataWithScope` packet from the host instructing to allocate 196608 bytes on the heap.
   
   On increasing the memory heap size in `K_HEAP_DEFINE` in `apps/microtvm/zephyr/template_project/src/host_driven/main.c` I was able to go a little bit ahead and fail only on the fourth call to `k_heap_alloc()`. But since I'm using a mx1050 board and you're using a mx1060 (with more memory) could you please try to increase the heap size and try again on your board:
   
   ```
   diff --git a/apps/microtvm/zephyr/template_project/src/host_driven/main.c b/apps/microtvm/zephyr/template_project/src/host_driven/main.c
   index 44d656028..64cd10fed 100644
   --- a/apps/microtvm/zephyr/template_project/src/host_driven/main.c
   +++ b/apps/microtvm/zephyr/template_project/src/host_driven/main.c
   @@ -130,7 +130,7 @@ tvm_crt_error_t TVMPlatformGenerateRandom(uint8_t* buffer, size_t num_bytes) {
    }
    
    // Heap for use by TVMPlatformMemoryAllocate.
   -K_HEAP_DEFINE(tvm_heap, 216 * 1024);
   +K_HEAP_DEFINE(tvm_heap, (216+768) * 1024);
    
    // Called by TVM to allocate memory.
    tvm_crt_error_t TVMPlatformMemoryAllocate(size_t num_bytes, DLDevice dev, void** out_ptr) {
   ``` 
   
   You need to run `make` inside TVM's build dir again after that change to update the template source dirs in `<build_dir>/microtvm_template_projects` accordingly.
   
   Also, now that `[TVMC][Relay] Introduce executor and runtime parameters (#9352)` is merged you need to adapt your reproducer and pass `runtime=RUNTIME`  to `relay.build()`, for example:
   
   ```
   ######################################################################
   # Now, compile the model for the target:
   from tvm.relay.backend import Runtime
   
   RUNTIME = Runtime("crt", {"system-lib": True})
   TARGET = tvm.target.target.micro("imxrt10xx")
   with tvm.transform.PassContext(
       opt_level=3, config={"tir.disable_vectorize": True}, disabled_pass=["AlterOpLayout"]
   ):
       module = relay.build(mod, target=TARGET, runtime=RUNTIME, params=params)
   ```
   
   That said, if you're looking for some performance numbers, I think you should try to use a `aot` executor instead of the `graph` executor.  I think also that `mobilenet`  on `aot` executor will have much smaller memory footprint due to some optimizations on handling tensor data between the operations in the workspace memory.
   
   Also, it's necessary to fix that useless error message: `RPCError: Error caught from RPC call:` by setting the `g_last_error` correctly  when allocation fails. So the user at least know there is a lack of memory at runtime. I'll send a patch for it soon.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org