You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "tqchen (via GitHub)" <gi...@apache.org> on 2025/08/19 23:04:13 UTC
[PR] [FFI] AudoDLPack compatible with torch stream context [tvm]
tqchen opened a new pull request, #18217:
URL: https://github.com/apache/tvm/pull/18217
This PR updates the autodlpack path to automatically update the env stream to be consistent with torch stream context.
The change would help to make FFI functions to be
compatible in stream based executions.
We leverage torch cpp_extension load_inline to create an efficient query function, the first time loading might take more time to build the jit module and
things should be fast after the torch jit module is cached.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FFI] AudoDLPack compatible with torch stream context [tvm]
Posted by "yongwww (via GitHub)" <gi...@apache.org>.
yongwww merged PR #18217:
URL: https://github.com/apache/tvm/pull/18217
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FFI] AudoDLPack compatible with torch stream context [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on PR #18217:
URL: https://github.com/apache/tvm/pull/18217#issuecomment-3202759387
ffi overhead benchmark `AMD Ryzen 9 7950X`
```
-----------------------------
Benchmark f(x, y, z) overhead
-----------------------------
numpy.add 2.0837783813476562e-07 sec/call
torch.add[cpu] 5.690574645996094e-07 sec/call
torch.add[cuda] 2.2510528564453123e-06 sec/call
tvm.ffi.nop 2.9222965240478516e-07 sec/call
tvm.ffi.nop+from_dlpack(torch) 3.5573482513427735e-06 sec/call
tvm.ffi.nop+from_dlpack(numpy) 1.001763343811035e-06 sec/call
tvm.ffi.nop+from_dlpack(tvm) 1.0982036590576173e-06 sec/call
tvm.ffi.nop+from_dlpack(torch.utils) 2.9434442520141603e-06 sec/call
tvm.ffi.nop.autodlpack(torch[cpu]) 3.265666961669922e-06 sec/call
tvm.ffi.nop.autodlpack(torch[cuda]) 3.4897327423095704e-06 sec/call
tvm.ffi.nop.autodlpack(torch[cuda][stream]) 3.4964323043823244e-06 sec/call
tvm.ffi.nop.autodlpack(numpy) 1.4113664627075195e-06 sec/call
-------------------------------
Benchmark x.__dlpack__ overhead
-------------------------------
torch.utils.dlpack.to_dlpack 3.6129951477050783e-07 sec/call
torch.__dlpack__ 8.010625839233399e-07 sec/call
numpy.__dlpack__ 6.115436553955078e-08 sec/call
tvm.__dlpack__ 9.13858413696289e-08 sec/call
---------------------------------------------------
Benchmark x.__dlpack__(max_version=(1,1)) overhead
---------------------------------------------------
torch.__dlpack__(max_version=(1,1)) Tensor.__dlpack__() got an unexpected keyword argument 'max_version'
numpy.__dlpack__(max_version=(1,1)) 7.741451263427734e-08 sec/call
tvm.__dlpack__(max_version=(1,1)) 1.41143798828125e-07 sec/call
---------------------------------------------------
Benchmark torch.get_cuda_stream[default stream]
---------------------------------------------------
torch.cuda.current_stream[cpp-extension] 9.298324584960938e-08 sec/call
torch.cuda.current_stream[python] 8.587837219238281e-07 sec/call
---------------------------------------------------
Benchmark torch.get_cuda_stream[non-default stream]
---------------------------------------------------
torch.cuda.current_stream[cpp-extension] 9.508132934570312e-08 sec/call
torch.cuda.current_stream[python] 8.99958610534668e-07 sec/call
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org