You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/02/22 00:06:24 UTC
[GitHub] [tvm] yzh119 opened a new pull request #10339: [WIP][PTX] Support mma.sp to use Sparse Tensor Cores
yzh119 opened a new pull request #10339:
URL: https://github.com/apache/tvm/pull/10339
Sparse Tensor Cores(STC) was first introduced in the Ampere architecture of NVIDIA GPUs (see [whitepaper](https://www.megware.com/fileadmin/user_upload/LandingPage%20NVIDIA/nvidia-ampere-architecture-whitepaper.pdf)). Up to now, developers can only activate STC by calling wrapped APIs such as `cuSparseLt` or writing PTX assembly code.
Following #9909, this PR support `ptx_mma_sp` intrinsic so as to expose the interface of STC at the TIR level.
Remaining TODO items:
- [ ] Write more tests.
- [ ] More documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yzh119 edited a comment on pull request #10339: [PTX] Support mma.sp to use Sparse Tensor Cores and refactor mma codegen
Posted by GitBox <gi...@apache.org>.
yzh119 edited a comment on pull request #10339:
URL: https://github.com/apache/tvm/pull/10339#issuecomment-1050483346
@vinx13 , refactor of MMA codegen is also finished.
I deleted some unittests such as `s8u832`, `u8s8s32`, `s4u432`, `u4s4s32` because they do not conform to the standard described [here](https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#warp-level-matrix-data-types), which requires multiplicands to have the same data type. I wonder are there cases we want these `s8u8s32` in quantization?
p.s. I can add these tests back if necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yzh119 commented on pull request #10339: [PTX] Support mma.sp to use Sparse Tensor Cores and refactor mma codegen
Posted by GitBox <gi...@apache.org>.
yzh119 commented on pull request #10339:
URL: https://github.com/apache/tvm/pull/10339#issuecomment-1050483346
@vinx13 , refactor of MMA codegen is also finished.
I deleted some unittests such as `s8u832`, `u8s8s32`, `s4u432`, `u4s4s32` because they do not conform to the standard described [here](https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#warp-level-matrix-data-types), which requires multiplicands to have the same data type. I wonder are there cases we want these `s8u8s32` in quantization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yzh119 edited a comment on pull request #10339: [PTX] Support mma.sp to use Sparse Tensor Cores and refactor mma codegen
Posted by GitBox <gi...@apache.org>.
yzh119 edited a comment on pull request #10339:
URL: https://github.com/apache/tvm/pull/10339#issuecomment-1050538263
okay, it turns out they mean elements inside the two multiplicands must have the same data type, but the two multiplicands can have different data types.
[cutlass](https://github.com/NVIDIA/cutlass/blob/e96f00586c92c2e77575ea4d3d906027595c8d9c/include/cutlass/arch/mma_sm75.h) also support u8s8s32 mma's.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vinx13 commented on pull request #10339: [WIP][PTX] Support mma.sp to use Sparse Tensor Cores
Posted by GitBox <gi...@apache.org>.
vinx13 commented on pull request #10339:
URL: https://github.com/apache/tvm/pull/10339#issuecomment-1049176654
If metadata is stored in a larger buffer, and any some elements of it are passed to mma.sp each time, offset is needed.
#9727 introduced some breaking changes in the semantic of `T.allocate`, we might want to wait until that PR is merged to prevent conflicts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yzh119 commented on pull request #10339: [WIP][PTX] Support mma.sp to use Sparse Tensor Cores
Posted by GitBox <gi...@apache.org>.
yzh119 commented on pull request #10339:
URL: https://github.com/apache/tvm/pull/10339#issuecomment-1049159837
@vinx13 one thing I'm not sure is if we need offset for metadata (which stores the indices information and is always 32bit).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yzh119 edited a comment on pull request #10339: [PTX] Support mma.sp to use Sparse Tensor Cores and refactor mma codegen
Posted by GitBox <gi...@apache.org>.
yzh119 edited a comment on pull request #10339:
URL: https://github.com/apache/tvm/pull/10339#issuecomment-1050538263
okay, it turns out they mean elements inside the two multiplicands must have the same data type, but the two multiplicands can have different data types.
[cutlass](https://github.com/NVIDIA/cutlass/blob/e96f00586c92c2e77575ea4d3d906027595c8d9c/include/cutlass/arch/mma_sm75.h) also support u8s8s32 mma's.
I have added these tests back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yzh119 commented on pull request #10339: [PTX] Support mma.sp to use Sparse Tensor Cores and refactor mma codegen
Posted by GitBox <gi...@apache.org>.
yzh119 commented on pull request #10339:
URL: https://github.com/apache/tvm/pull/10339#issuecomment-1060154678
@vinx13 refactored according to the change in #9727 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vinx13 merged pull request #10339: [PTX] Support mma.sp to use Sparse Tensor Cores and refactor mma codegen
Posted by GitBox <gi...@apache.org>.
vinx13 merged pull request #10339:
URL: https://github.com/apache/tvm/pull/10339
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yzh119 commented on pull request #10339: [PTX] Support mma.sp to use Sparse Tensor Cores and refactor mma codegen
Posted by GitBox <gi...@apache.org>.
yzh119 commented on pull request #10339:
URL: https://github.com/apache/tvm/pull/10339#issuecomment-1050538263
okay, it turns out they mean elements inside the two multiplicands must have the same data type, but the two multiplicands can have different data types.
[cutlass](https://github.com/NVIDIA/cutlass/blob/e96f00586c92c2e77575ea4d3d906027595c8d9c/include/cutlass/arch/mma_sm75.h) also support u8s8s32 mma's.
I'll add these tests back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org