You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "coffezhou (via GitHub)" <gi...@apache.org> on 2025/07/11 08:18:24 UTC

[I] [Bug] InternalError: Check failed: (offset + needed_size <= this->buffer.size) is false: storage allocation failure [tvm]

coffezhou opened a new issue, #18132:
URL: https://github.com/apache/tvm/issues/18132

   
   ### Expected behavior
   
   TVM should run the model correctly.
   
   ### Actual behavior
   For the following model,
   
   <img width="269" height="460" alt="Image" src="https://github.com/user-attachments/assets/ba94f29e-b7c9-4c77-b222-ad028c7cc03d" />
   
   it can be executed by onnxruntime, the results are as follows:
   ```c
   ONNXRuntime:
    [array([[[[False, False, False, False],
            [False, False, False, False],
            [False, False, False, False],
            [False, False, False, False]]]])]
   ```
   However, when compiling and running the model using TVM, TVM crashes:
   ```c
     File "/home/carla/Documents/tvm/python/tvm/runtime/vm.py", line 295, in invoke_stateful
       self._invoke_stateful(func_name)
     File "tvm/ffi/cython/./function.pxi", line 228, in tvm.ffi.core.Function.__call__
   tvm.error.InternalError: Check failed: (offset + needed_size <= this->buffer.size) is false: storage allocation failure, attempted to allocate 18446744073709551553 at offset 0 in region that is 0bytes
   ```
   
   ### Environment
   
   OS: Ubuntu 20.04
   TVM: 0.22.dev0 (c6969d723)
   onnxruntime: 1.21.0
   
   ### Steps to reproduce
   
   This bug can be reproduced by the following code with the model in the attachment. As shown in the code, the model can be executed by onnxruntime. However, TVM crashes when calling the invoke_stateful function.
   ```python
   import sys
   
   import numpy as np
   import onnx
   import onnxruntime
   
   import tvm
   from tvm import relax
   from tvm.relax.frontend.onnx import from_onnx
   
   import pickle
   
               
   def main():
       onnx_model = onnx.load("111.onnx")
       
       with open("inputs.pkl", "rb") as fp:
           inputs = pickle.load(fp)
       print(inputs)
       try:
           ort_session = onnxruntime.InferenceSession(
               onnx_model.SerializeToString(), providers=["CPUExecutionProvider"]
           )
           ort_output = ort_session.run([], inputs)
       except Exception as e:
           print(e)
           sys.exit(1)
           
       print("ONNXRuntime:\n", ort_output)   
   
       # Convert the onnx model into relax through the onnx importer.
       tvm_model = from_onnx(onnx_model, keep_params_in_input=True)
       # Convert operators for inference mode.
       tvm_model = relax.transform.DecomposeOpsForInference()(tvm_model)
       # Legalize any relax ops into tensorir.
       tvm_model = relax.transform.LegalizeOps()(tvm_model)
   
       # Separate model from parameters.
       tvm_model, params = relax.frontend.detach_params(tvm_model)
       
       # Prepare inputs.
       input_list = [
           inputs[key.name_hint] for key in tvm_model["main"].params if key.name_hint in inputs
       ]
       if params:
           input_list += params["main"]
           
       # Compile the relax graph into a VM then run.
       #----------------------cpu-----------------------
       with tvm.transform.PassContext(opt_level=0):
           target = tvm.target.Target("llvm", host="llvm")
           relax_pipeline = relax.pipeline.get_default_pipeline(target)
           
           ex = relax.build(tvm_model, target="llvm", relax_pipeline=relax_pipeline)
           vm = relax.VirtualMachine(ex, tvm.cpu())
       
           # Run model and check outputs.
           vm.set_input("main", *input_list)
           vm.invoke_stateful("main")
           tvm_cpu_output = vm.get_outputs("main")
   
       
   if __name__ == "__main__":
       
       main()
   
   ```
   
   [testcase.zip](https://github.com/user-attachments/files/21178457/testcase.zip)
   
   ### Triage
   
   Please refer to the list of label tags [here](https://github.com/apache/tvm/wiki/Issue-Triage-Labels) to find the relevant tags and add them below in a bullet format (example below).
   
   * needs-triage
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] InternalError: Check failed: (offset + needed_size <= this->buffer.size) is false: storage allocation failure [tvm]

Posted by "coffezhou (via GitHub)" <gi...@apache.org>.
coffezhou commented on issue #18132:
URL: https://github.com/apache/tvm/issues/18132#issuecomment-3066715401

   > The error occurs because the `expand_shape` parameter you passed to the main function is `tvm.nd.array([1, 1, 4, 4])`, but TVM's _**Exapnd**_ node requires that the `expand_shape` passed in is the size of the target tensor. Therefore, `tvm.nd.array([1, 3, 4, 4])` should be passed in here:
   > 
   > if params:
   >     input_list += params["main"]
   >     input_list[-1] = tvm.nd.array([1, 3, 4, 4])
   
   Thanks! This issue could be resolved by adding the code "input_list[-1] = tvm.nd.array([1, 3, 4, 4])".
   However, according to the[ onnx specification of Expand](https://onnx.ai/onnx/operators/onnx__Expand.html), the Expand operator supports the broadcast rule. The Expand operator in TVM is not compatible with that of onnx. Maybe some documents of this should be added.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] InternalError: Check failed: (offset + needed_size <= this->buffer.size) is false: storage allocation failure [tvm]

Posted by "ConvolutedDog (via GitHub)" <gi...@apache.org>.
ConvolutedDog commented on issue #18132:
URL: https://github.com/apache/tvm/issues/18132#issuecomment-3066738642

   Yes this is a good suggestion, it needs to be better documented. Maybe there should be an operator documentation similar to ONNX or PyTorch.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] InternalError: Check failed: (offset + needed_size <= this->buffer.size) is false: storage allocation failure [tvm]

Posted by "ConvolutedDog (via GitHub)" <gi...@apache.org>.
ConvolutedDog commented on issue #18132:
URL: https://github.com/apache/tvm/issues/18132#issuecomment-3065996308

   The error occurs because the `expand_shape` parameter you passed to the main function is `tvm.nd.array([1, 1, 4, 4])`, but TVM's ***Exapnd*** node requires that the `expand_shape` passed in is the size of the target tensor. Therefore, `tvm.nd.array([1, 3, 4, 4])` should be passed in here:
   
   ```bash
   if params:
       input_list += params["main"]
       input_list[-1] = tvm.nd.array([1, 3, 4, 4])
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org