You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/03/30 16:24:52 UTC
[GitHub] [tvm] mbaret opened a new pull request #10826: [CUBLAS] Add support for nn.dense and nn.batch_matmul
mbaret opened a new pull request #10826:
URL: https://github.com/apache/tvm/pull/10826
Currently on top of https://github.com/apache/tvm/pull/10820 so contains those changes too. Will be rebased when that PR is merged.
Add support for nn.dense and nn.batch_matmul using cuBLAS. This commit includes a fix for cublas.batch_matmul when mixed precision is being used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mbs-octoml commented on a change in pull request #10826: [CUBLAS] Add support for nn.dense and nn.batch_matmul
Posted by GitBox <gi...@apache.org>.
mbs-octoml commented on a change in pull request #10826:
URL: https://github.com/apache/tvm/pull/10826#discussion_r838913408
##########
File path: python/tvm/relay/op/contrib/cublas.py
##########
@@ -156,3 +165,21 @@ def _lower_matmul(op: relay.Call, inputs: List[te.Tensor]) -> te.Tensor:
transb=op.attrs["transpose_b"],
dtype=op.checked_type.dtype,
)
+
+
+@_lower_composite("cublas.batch_matmul")
+def _lower_batch_matmul(op: relay.Call, inputs: List[te.Tensor]) -> te.Tensor:
+ """Lower a batch_matmul using cuBLAS."""
+ return cublas.batch_matmul(
+ inputs[0],
+ inputs[1],
+ transa=op.attrs["transpose_a"],
+ transb=op.attrs["transpose_b"],
+ dtype=op.checked_type.dtype,
+ )
+
+
+@_lower_composite("cublas.dense")
+def _lower_dense(op: relay.Call, inputs: List[te.Tensor]) -> te.Tensor:
+ """Lower a dense using cuBLAS."""
+ return cublas.matmul(inputs[0], inputs[1], False, True, dtype=op.checked_type.dtype)
Review comment:
nit: use the trans{a,b} kwd args
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #10826: [CUBLAS] Add support for nn.dense and nn.batch_matmul
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #10826:
URL: https://github.com/apache/tvm/pull/10826
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mbaret commented on pull request #10826: [CUBLAS] Add support for nn.dense and nn.batch_matmul
Posted by GitBox <gi...@apache.org>.
mbaret commented on pull request #10826:
URL: https://github.com/apache/tvm/pull/10826#issuecomment-1083476419
cc @mbs-octoml @mikepapadim
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org