[GitHub] [tvm] hlacikd opened a new issue, #15770: [Bug]

["hlacikd (via GitHub)" <gi...@apache.org>]

hlacikd opened a new issue, #15770:
URL: https://github.com/apache/tvm/issues/15770

   ### Actual behavior
   
   When feeding data via *module.set_input* , inference time is **4fps**, without it it reaches over **100fps**!
   
   Example
   ```
   target = tvm.target.mali(model="rk3588")
   loaded_lib = tvm.runtime.load_module("lpr_model_0910_autotvm.tar")
   dev = tvm.device(str(target), 0)
   module = graph_executor.GraphModule(lib["default"](dev))
   ```
   this gives over 100fps :
   ```
   import time
   
   tic = time.time()
   for i in range(100):
       module.run()
   fps = 100 / (time.time() - tic)
   print(f"FPS: {fps:.2f}")
   ```
   
   adding set_input, slowes it down to 4.67fps!
   
   ```
   import time
   
   tic = time.time()
   for i in range(100):
       module.set_input(input_name, tvm.nd.array(img_data))
       module.run()
   fps = 100 / (time.time() - tic)
   print(f"FPS: {fps:.2f}")
   ```
   
   img_data is just normalized image
   ```
   import cv2
   
   def load_image(image_path="./lp.jpg"):
       image = cv2.imread(image_path)
       image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
       image = cv2.resize(image, (94, 24))
       return image
   
   img_data = load_image().astype(np.float32)
   img_data /= 255.0
   img_data = img_data.transpose((2, 0, 1))
   img_data = img_data[np.newaxis, :]
   ```
   
   how can i make it faster? is really feeding mali gpu taking so long?
   this makes model run faster (~30fps on cpu itself)
   
   ### Environment
   
   ## OpenCL
   root@notebook-68c585ffc-c2fx4:/app/notebook# clinfo
   arm_release_ver: g13p0-01eac0, rk_so_ver: 3
   Number of platforms                               1
     Platform Name                                   ARM Platform
     Platform Vendor                                 ARM
     Platform Version                                OpenCL 3.0 v1.g13p0-01eac0.a8b6f0c7e1f83c654c60d1775112dbe4
     Platform Profile                                FULL_PROFILE
   
   ## OS
   Welcome to Armbian 23.8.1 Jammy with Linux 5.10.160-legacy-rk35xx
   
   ## HW
   Radxa Rock 5B (RK3588]
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] masahi commented on issue #15770: [Bug]

["masahi (via GitHub)" <gi...@apache.org>]

masahi commented on issue #15770:
URL: https://github.com/apache/tvm/issues/15770#issuecomment-1725253873

   ```
   import time
   
   tic = time.time()
   for i in range(100):
       module.run()
   fps = 100 / (time.time() - tic)
   print(f"FPS: {fps:.2f}")
   ```
   
   This code is not correctly measuring the elapsed time. `.run()` is asynchronous, so it hasn't been finished when it returns.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] masahi closed issue #15770: [Bug]

["masahi (via GitHub)" <gi...@apache.org>]

masahi closed issue #15770: [Bug] 
URL: https://github.com/apache/tvm/issues/15770


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org