You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "MasterJH5574 (via GitHub)" <gi...@apache.org> on 2024/02/09 21:53:25 UTC
[PR] [KVCache] Fix attention kernel for ROCm [tvm]
MasterJH5574 opened a new pull request, #16551:
URL: https://github.com/apache/tvm/pull/16551
When compiling a TensorIR function to ROCm backends, we need to be careful to store and load local registers in a consistent approach (either using vector access or scalar access), or otherwise there will be correctness issue.
This PR fixes the attention kernel for ROCm.
Meanwhile, it adds tests for float32 dtype and head dim other than 128.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [KVCache] Fix attention kernel for ROCm [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen merged PR #16551:
URL: https://github.com/apache/tvm/pull/16551
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org