You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/02/25 18:03:18 UTC
[GitHub] [tvm] mei-ye opened a new pull request #10391: [vulkan] Add integer dot product (4xint8, 4xuint8) tensorization for the vulkan SPIR-V target.
mei-ye opened a new pull request #10391:
URL: https://github.com/apache/tvm/pull/10391
@masahi
[vulkan] Add integer dot product (4xint8, 4xuint8) tensorization for the vulkan SPIR-V target. Currently only autotvm path is supported.
Prerequisites for compilation: (1) Use VulkanSDK 1.2.198 release with SPIR-V integer dot product suppport (2) set(USE_SPIRV_KHR_INTEGER_DOT_PRODUCT ON) in config.cmake and build (3) Use a driver that supports VK_KHR_shader_integer_dot_product extension.
The compiled binary can only be run on a hardware that supports relevant ISA. This work is tested on AMD RDNA2 famillies (e.g., Rembrandt and RX6800).
To compile on a device that supports this extension, use target: vulkan -from_device=0
To compile on a device that supports int8 but does not support this extension, add "-supports_integer_dot_product=1" or "-mattr=+dotprod" to the target string.
To support pre-released vulkan and SPIR-V extensions, we need SPIR-V tool and header file from Khronos github, use the option: USE_KHRONOS_SPIRV in config.cmake.
Example to use this feature can be found in: gallery/how_to/deploy_models/deploy_prequantized.py and gallery/how_to/deploy_models/deploy_prequantized_tflite.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #10391: [vulkan] Add integer dot product (4xint8, 4xuint8) tensorization for the vulkan SPIR-V target.
Posted by GitBox <gi...@apache.org>.
masahi commented on pull request #10391:
URL: https://github.com/apache/tvm/pull/10391#issuecomment-1061041348
Ok @mei-ye the CI is green now. The changes I made are:
* Remove the `storage_rewrite.cc` change that was originally in @mei-ye's commit, but it turned out it is no longer necessary after @Lunderberg's PR https://github.com/apache/tvm/pull/9727 was merged.
* Check tensorization condition in `batch_matmul` https://github.com/apache/tvm/pull/10391/commits/aa7328f09a8f0f30d4fbb1cfb8d0455748bb9547
* Do not run vk tests on int8 `batch_matmul` and `conv2d`. In particular, for some reason running int8 `batch_matmul` test on vk, disabled in https://github.com/apache/tvm/pull/10391/commits/839d2338a3a750278d77ece3de57c94559015420, was apparently the cause of the strange CI error problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #10391: [vulkan] Add integer dot product (4xint8, 4xuint8) tensorization for the vulkan SPIR-V target.
Posted by GitBox <gi...@apache.org>.
masahi commented on pull request #10391:
URL: https://github.com/apache/tvm/pull/10391#issuecomment-1051179563
@mei-ye Can you clean up the commit history? (reset to `main` and apply [7eea90b](https://github.com/apache/tvm/pull/10391/commits/7eea90bae0031d0dd6db668314f0013e95fa6f76))
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #10391: [vulkan] Add integer dot product (4xint8, 4xuint8) tensorization for the vulkan SPIR-V target.
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #10391:
URL: https://github.com/apache/tvm/pull/10391
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #10391: [vulkan] Add integer dot product (4xint8, 4xuint8) tensorization for the vulkan SPIR-V target.
Posted by GitBox <gi...@apache.org>.
masahi commented on pull request #10391:
URL: https://github.com/apache/tvm/pull/10391#issuecomment-1051191212
@mei-ye Please make
```
if (
"vulkan" not in target.keys
or "+dotprod" in target.mattr
or target.supports_integer_dot_product
):
```
more readable. For now this should only be True for cuda and vk with dot product support. But later I'll add opencl and rocm support too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #10391: [vulkan] Add integer dot product (4xint8, 4xuint8) tensorization for the vulkan SPIR-V target.
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #10391:
URL: https://github.com/apache/tvm/pull/10391#discussion_r815088137
##########
File path: python/tvm/relay/op/strategy/cuda.py
##########
@@ -372,7 +376,7 @@ def conv2d_strategy_cuda(attrs, inputs, out_type, target):
ic_chunk = in_channels // 4
if (
- target.kind.name == "cuda"
+ (target.kind.name == "cuda" or target.kind.name == "vulkan")
Review comment:
`target.kind.name in ["cuda", "vulkan"]`. Three places.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mei-ye commented on a change in pull request #10391: [vulkan] Add integer dot product (4xint8, 4xuint8) tensorization for the vulkan SPIR-V target.
Posted by GitBox <gi...@apache.org>.
mei-ye commented on a change in pull request #10391:
URL: https://github.com/apache/tvm/pull/10391#discussion_r815199397
##########
File path: python/tvm/topi/cuda/conv2d_alter_op.py
##########
@@ -83,14 +83,19 @@ def _alter_conv2d_layout(attrs, inputs, tinfos, out_type):
cfg = dispatch_ctx.query(target, workload)
if cfg.is_fallback: # if is fallback, clear query cache and return None
autotvm.task.clear_fallback_cache(target, workload)
- return None
+ if (
+ "vulkan" not in target.keys
+ or "+dotprod" not in target.mattr
+ or not target.supports_integer_dot_product
+ ):
+ return None
Review comment:
If it is fall back and if dot product is to be used, make the flow to proceed to do the NCHW => NCHW4c layout transformation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #10391: [vulkan] Add integer dot product (4xint8, 4xuint8) tensorization for the vulkan SPIR-V target.
Posted by GitBox <gi...@apache.org>.
masahi commented on pull request #10391:
URL: https://github.com/apache/tvm/pull/10391#issuecomment-1051182935
Please remove the change in `deploy_prequantized.py` and `deploy_prequantized_tflite.py`. They run on CI and we don't intend to run e2e models on vulkan target there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #10391: [vulkan] Add integer dot product (4xint8, 4xuint8) tensorization for the vulkan SPIR-V target.
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #10391:
URL: https://github.com/apache/tvm/pull/10391#discussion_r815088947
##########
File path: python/tvm/topi/cuda/conv2d_alter_op.py
##########
@@ -83,14 +83,19 @@ def _alter_conv2d_layout(attrs, inputs, tinfos, out_type):
cfg = dispatch_ctx.query(target, workload)
if cfg.is_fallback: # if is fallback, clear query cache and return None
autotvm.task.clear_fallback_cache(target, workload)
- return None
+ if (
+ "vulkan" not in target.keys
+ or "+dotprod" not in target.mattr
+ or not target.supports_integer_dot_product
+ ):
+ return None
Review comment:
What is this change for?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org