You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/02/11 23:06:04 UTC
[GitHub] [incubator-tvm] wpan11nv opened a new pull request #4867:
[TOPI][CUDA] Enable vectorization on fp16 type
wpan11nv opened a new pull request #4867: [TOPI][CUDA] Enable vectorization on fp16 type
URL: https://github.com/apache/incubator-tvm/pull/4867
- This allows to better utilize the memory bandwidth
- Note that not all cases are vectorized for fp16 datatype. For
instance, when the size is not a multiple of 1024, the inner loop
may be an expression that cannot be vectorized. In this case, a
small inner loop is still benefical for latency hidding.
Signed-off-by: Wei Pan <we...@nvidia.com>
Thanks for contributing to TVM! Please refer to guideline https://docs.tvm.ai/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @ them in the pull request thread.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] wpan11nv commented on issue #4867: [TOPI][CUDA]
Enable vectorization on fp16 type
Posted by GitBox <gi...@apache.org>.
wpan11nv commented on issue #4867: [TOPI][CUDA] Enable vectorization on fp16 type
URL: https://github.com/apache/incubator-tvm/pull/4867#issuecomment-585497119
Thanks all for the suggestions! Tests updated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] Laurawly commented on a change in pull request
#4867: [TOPI][CUDA] Enable vectorization on fp16 type
Posted by GitBox <gi...@apache.org>.
Laurawly commented on a change in pull request #4867: [TOPI][CUDA] Enable vectorization on fp16 type
URL: https://github.com/apache/incubator-tvm/pull/4867#discussion_r378516074
##########
File path: topi/tests/python/test_topi_relu.py
##########
@@ -87,12 +87,12 @@ def _prelu_numpy(x, W):
tvm.testing.assert_allclose(b.asnumpy(), out_np, rtol=1e-5)
def test_relu():
- verify_relu(10, 128)
+ verify_relu(128, 128, "float32")
Review comment:
Can you keep a test case as before where `m` and `n` have different values?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen merged pull request #4867: [TOPI][CUDA]
Enable vectorization on fp16 type
Posted by GitBox <gi...@apache.org>.
tqchen merged pull request #4867: [TOPI][CUDA] Enable vectorization on fp16 type
URL: https://github.com/apache/incubator-tvm/pull/4867
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #4867: [TOPI][CUDA]
Enable vectorization on fp16 type
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #4867: [TOPI][CUDA] Enable vectorization on fp16 type
URL: https://github.com/apache/incubator-tvm/pull/4867#issuecomment-585395086
@vinx13 @Laurawly please help to review the PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #4867: [TOPI][CUDA]
Enable vectorization on fp16 type
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #4867: [TOPI][CUDA] Enable vectorization on fp16 type
URL: https://github.com/apache/incubator-tvm/pull/4867#issuecomment-586091707
Thanks @vinx13 @wpan11nv !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] vinx13 commented on a change in pull request #4867:
[TOPI][CUDA] Enable vectorization on fp16 type
Posted by GitBox <gi...@apache.org>.
vinx13 commented on a change in pull request #4867: [TOPI][CUDA] Enable vectorization on fp16 type
URL: https://github.com/apache/incubator-tvm/pull/4867#discussion_r378985967
##########
File path: topi/tests/python/test_topi_relu.py
##########
@@ -20,11 +20,20 @@
import tvm
import topi
from topi.util import get_const_tuple
-
+from tvm.contrib.nvcc import parse_compute_version
from common import get_all_backend
-def verify_relu(m, n):
- A = tvm.placeholder((m, n), name='A')
+def skipTest(dtype, device):
Review comment:
nit: we prefer `skip_test` style naming
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #4867: [TOPI][CUDA]
Enable vectorization on fp16 type
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #4867: [TOPI][CUDA] Enable vectorization on fp16 type
URL: https://github.com/apache/incubator-tvm/pull/4867#issuecomment-584951383
Please request reviews from reviewers
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] wpan11nv commented on issue #4867: [TOPI][CUDA]
Enable vectorization on fp16 type
Posted by GitBox <gi...@apache.org>.
wpan11nv commented on issue #4867: [TOPI][CUDA] Enable vectorization on fp16 type
URL: https://github.com/apache/incubator-tvm/pull/4867#issuecomment-585355592
Kindly ping. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] Laurawly commented on a change in pull request
#4867: [TOPI][CUDA] Enable vectorization on fp16 type
Posted by GitBox <gi...@apache.org>.
Laurawly commented on a change in pull request #4867: [TOPI][CUDA] Enable vectorization on fp16 type
URL: https://github.com/apache/incubator-tvm/pull/4867#discussion_r378516859
##########
File path: topi/tests/python/test_topi_tensor.py
##########
@@ -84,18 +84,41 @@ def check_device(device):
for device in ["llvm"]:
check_device(device)
+def verify_vectorization(n, m, dtype):
+ def check_device(device):
+ if not tvm.runtime.enabled(device):
+ print("Skip because %s is not enabled" % device)
+ return
+ with tvm.target.create(device):
+ ctx = tvm.context(device, 0)
+ A = tvm.placeholder((n, m), name='A', dtype=dtype)
+ B = tvm.compute((n, m), lambda i, j:
+ A[i, j] + tvm.const(1, A.dtype), name='B')
+ S = topi.generic.schedule_elemwise(B)
+
+ fun = tvm.build(S, [A, B], device)
+ np_A = tvm.nd.empty((n, m), A.dtype, ctx).copyfrom(
+ np.random.uniform(size=(n, m)))
+ np_B = tvm.nd.empty((n, m), B.dtype, ctx)
+ fun(np_A, np_B)
+ tvm.testing.assert_allclose(np_B.asnumpy(), np_A.asnumpy() + 1, rtol=1e-5)
+
+ for device in ["cuda"]:
+ check_device(device)
+
+def test_vectorization():
+ verify_vectorization(128, 128, "float16")
Review comment:
Same here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] vinx13 commented on a change in pull request #4867:
[TOPI][CUDA] Enable vectorization on fp16 type
Posted by GitBox <gi...@apache.org>.
vinx13 commented on a change in pull request #4867: [TOPI][CUDA] Enable vectorization on fp16 type
URL: https://github.com/apache/incubator-tvm/pull/4867#discussion_r378503237
##########
File path: topi/tests/python/test_topi_tensor.py
##########
@@ -84,18 +84,41 @@ def check_device(device):
for device in ["llvm"]:
check_device(device)
+def verify_vectorization(n, m, dtype):
+ def check_device(device):
+ if not tvm.runtime.enabled(device):
+ print("Skip because %s is not enabled" % device)
+ return
+ with tvm.target.create(device):
Review comment:
add a check whether fp16 is supported here and also `verify_relu`
https://github.com/apache/incubator-tvm/blob/aaf62e47e64d592be770e915a7aa59d41eddb729/topi/tests/python/test_topi_transform.py#L57-L59
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] wpan11nv commented on a change in pull request
#4867: [TOPI][CUDA] Enable vectorization on fp16 type
Posted by GitBox <gi...@apache.org>.
wpan11nv commented on a change in pull request #4867: [TOPI][CUDA] Enable vectorization on fp16 type
URL: https://github.com/apache/incubator-tvm/pull/4867#discussion_r379014124
##########
File path: topi/tests/python/test_topi_relu.py
##########
@@ -20,11 +20,20 @@
import tvm
import topi
from topi.util import get_const_tuple
-
+from tvm.contrib.nvcc import parse_compute_version
from common import get_all_backend
-def verify_relu(m, n):
- A = tvm.placeholder((m, n), name='A')
+def skipTest(dtype, device):
Review comment:
Fixed. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services