You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "ibsidorenko (via GitHub)" <gi...@apache.org> on 2023/11/07 12:20:51 UTC
[PR] [Unity][Dlight] Enhance matmul tensorizer with Int8 support [tvm]
ibsidorenko opened a new pull request, #16084:
URL: https://github.com/apache/tvm/pull/16084
Main goal of this commit was not to provide highly optimized schedule, but to enable tensor core tensorization for i8i8i32 matmul. Now it supports fp16 matmul only.
Anyway, speedup is a case compared to fp16 (fp16fp16->fp16 vs i8i8->i32):
N | M, N, K* | Shape type | fp16, us | int8, us | speedup |
--|----------------------|----------------|----------|----------|---------|
1 | 256, 256, 256 | static shapes | 15.03 | 12.55 | 1.198x |
2 | 512, 512, 4096 | static shapes | 173.25 | 129.06 | 1.342x |
3 | N (=260), 4096, 4096 | dynamic shapes | 350.44 | 298.29 | 1.175x |
where K* - is a reduction axis.
Implementation of Matmul int8 tensorizer was derived from Matmul tensorizer with minimal changes. It is possible to join these implementations (fp16 and int8), but I am not sure this is a good idea. Since it can slow down development of fp16 part.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Unity][Dlight] Enhance matmul tensorizer with Int8 support [tvm]
Posted by "Hzfengsy (via GitHub)" <gi...@apache.org>.
Hzfengsy commented on PR #16084:
URL: https://github.com/apache/tvm/pull/16084#issuecomment-1798415734
Awesome work! Thanks very much, @ibsidorenko! cc @vinx13 @masahi for review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Unity][Dlight] Enhance matmul tensorizer with Int8 support [tvm]
Posted by "vinx13 (via GitHub)" <gi...@apache.org>.
vinx13 merged PR #16084:
URL: https://github.com/apache/tvm/pull/16084
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org