You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "yelite (via GitHub)" <gi...@apache.org> on 2023/03/09 18:04:34 UTC
[GitHub] [tvm] yelite opened a new pull request, #14252: [Unity][BYOC] Add conv2d and residual block patterns for Relax cutlass BYOC
yelite opened a new pull request, #14252:
URL: https://github.com/apache/tvm/pull/14252
This PR adds cutlass BYOC pattern for conv2d and conv2d residual block.
In additional to the new patterns in the cutlass pattern table, this PR:
1. Remove the assumption on fused function arg order from the conv2d codegen.
2. Restructure how the cutlass pattern table is constructed, making it more readable.
The check logic at https://github.com/apache/tvm/blob/main/python/tvm/relay/op/contrib/cutlass.py#L200 isn't implemented in this PR yet because it requires non-trivial change to the interface of FuseOpsByPattern. I will send a separate PR to do this.
cc @vinx13 @masahi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a diff in pull request #14252: [Unity][BYOC] Add conv2d and residual block patterns for Relax cutlass BYOC
Posted by "masahi (via GitHub)" <gi...@apache.org>.
masahi commented on code in PR #14252:
URL: https://github.com/apache/tvm/pull/14252#discussion_r1131602196
##########
python/tvm/relax/backend/contrib/cutlass.py:
##########
@@ -51,21 +52,59 @@ def _is_supported_dtype(lhs_dtype, rhs_dtype):
)
-def _check_matmul(
- match_result: Mapping[DFPattern, Expr],
- _: Expr,
-) -> bool:
- matmul_call: Call = None
+def _find_call(op_name: str, match_result: Mapping[DFPattern, Expr]) -> Optional[Expr]:
+ result = None
+
for pattern, expr in match_result.items():
if (
isinstance(expr, Call)
and isinstance(pattern, CallPattern)
and isinstance(expr.op, tvm.ir.Op)
- and expr.op.name == "relax.matmul"
+ and expr.op.name == op_name
):
- matmul_call = expr
+ if result is not None:
+ raise ValueError(f"Found multiple matched call node for {op_name}")
+ result = expr
+
+ return result
+
+
+def _check_conv2d(
+ match_result: Mapping[DFPattern, Expr],
+ _: Expr,
+):
+ """Check if the given conv2d workload can be offloaded to CUTLASS."""
+
+ conv2d_call = _find_call("relax.nn.conv2d", match_result)
+ if conv2d_call is None:
+ return False
+
+ data_layout = conv2d_call.attrs.data_layout
+ kernel_layout = conv2d_call.attrs.kernel_layout
+ data, weight, *_ = conv2d_call.args
+ if (
+ data_layout != "NHWC"
+ or kernel_layout != "OHWI"
+ or not _is_supported_dtype(data.struct_info.dtype, weight.struct_info.dtype)
+ ):
+ return False
+
+ # pylint: disable=invalid-name
+ IC = data.struct_info.shape.values[3]
+ OC = weight.struct_info.shape.values[0]
+ # not depthwise conv2d
Review Comment:
We do support depthwise now cc @vinx13
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tvm-bot commented on pull request #14252: [Unity][BYOC] Add conv2d and residual block patterns for Relax cutlass BYOC
Posted by "tvm-bot (via GitHub)" <gi...@apache.org>.
tvm-bot commented on PR #14252:
URL: https://github.com/apache/tvm/pull/14252#issuecomment-1462529427
<!---bot-comment-->
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
<!--bot-comment-ccs-start-->
* cc @billishyahao, @quic-sanirudh <sub>See [#10317](https://github.com/apache/tvm/issues/10317) for details</sub><!--bot-comment-ccs-end-->
<sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #14252: [Unity][BYOC] Add conv2d and residual block patterns for Relax cutlass BYOC
Posted by "masahi (via GitHub)" <gi...@apache.org>.
masahi merged PR #14252:
URL: https://github.com/apache/tvm/pull/14252
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vinx13 commented on a diff in pull request #14252: [Unity][BYOC] Add conv2d and residual block patterns for Relax cutlass BYOC
Posted by "vinx13 (via GitHub)" <gi...@apache.org>.
vinx13 commented on code in PR #14252:
URL: https://github.com/apache/tvm/pull/14252#discussion_r1131767137
##########
python/tvm/relax/backend/contrib/cutlass.py:
##########
@@ -51,21 +52,59 @@ def _is_supported_dtype(lhs_dtype, rhs_dtype):
)
-def _check_matmul(
- match_result: Mapping[DFPattern, Expr],
- _: Expr,
-) -> bool:
- matmul_call: Call = None
+def _find_call(op_name: str, match_result: Mapping[DFPattern, Expr]) -> Optional[Expr]:
+ result = None
+
for pattern, expr in match_result.items():
if (
isinstance(expr, Call)
and isinstance(pattern, CallPattern)
and isinstance(expr.op, tvm.ir.Op)
- and expr.op.name == "relax.matmul"
+ and expr.op.name == op_name
):
- matmul_call = expr
+ if result is not None:
+ raise ValueError(f"Found multiple matched call node for {op_name}")
+ result = expr
+
+ return result
+
+
+def _check_conv2d(
+ match_result: Mapping[DFPattern, Expr],
+ _: Expr,
+):
+ """Check if the given conv2d workload can be offloaded to CUTLASS."""
+
+ conv2d_call = _find_call("relax.nn.conv2d", match_result)
+ if conv2d_call is None:
+ return False
+
+ data_layout = conv2d_call.attrs.data_layout
+ kernel_layout = conv2d_call.attrs.kernel_layout
+ data, weight, *_ = conv2d_call.args
+ if (
+ data_layout != "NHWC"
+ or kernel_layout != "OHWI"
+ or not _is_supported_dtype(data.struct_info.dtype, weight.struct_info.dtype)
+ ):
+ return False
+
+ # pylint: disable=invalid-name
+ IC = data.struct_info.shape.values[3]
+ OC = weight.struct_info.shape.values[0]
+ # not depthwise conv2d
Review Comment:
the codegen part for depthwise is still missing, we should update this when it's done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vinx13 commented on a diff in pull request #14252: [Unity][BYOC] Add conv2d and residual block patterns for Relax cutlass BYOC
Posted by "vinx13 (via GitHub)" <gi...@apache.org>.
vinx13 commented on code in PR #14252:
URL: https://github.com/apache/tvm/pull/14252#discussion_r1131767137
##########
python/tvm/relax/backend/contrib/cutlass.py:
##########
@@ -51,21 +52,59 @@ def _is_supported_dtype(lhs_dtype, rhs_dtype):
)
-def _check_matmul(
- match_result: Mapping[DFPattern, Expr],
- _: Expr,
-) -> bool:
- matmul_call: Call = None
+def _find_call(op_name: str, match_result: Mapping[DFPattern, Expr]) -> Optional[Expr]:
+ result = None
+
for pattern, expr in match_result.items():
if (
isinstance(expr, Call)
and isinstance(pattern, CallPattern)
and isinstance(expr.op, tvm.ir.Op)
- and expr.op.name == "relax.matmul"
+ and expr.op.name == op_name
):
- matmul_call = expr
+ if result is not None:
+ raise ValueError(f"Found multiple matched call node for {op_name}")
+ result = expr
+
+ return result
+
+
+def _check_conv2d(
+ match_result: Mapping[DFPattern, Expr],
+ _: Expr,
+):
+ """Check if the given conv2d workload can be offloaded to CUTLASS."""
+
+ conv2d_call = _find_call("relax.nn.conv2d", match_result)
+ if conv2d_call is None:
+ return False
+
+ data_layout = conv2d_call.attrs.data_layout
+ kernel_layout = conv2d_call.attrs.kernel_layout
+ data, weight, *_ = conv2d_call.args
+ if (
+ data_layout != "NHWC"
+ or kernel_layout != "OHWI"
+ or not _is_supported_dtype(data.struct_info.dtype, weight.struct_info.dtype)
+ ):
+ return False
+
+ # pylint: disable=invalid-name
+ IC = data.struct_info.shape.values[3]
+ OC = weight.struct_info.shape.values[0]
+ # not depthwise conv2d
Review Comment:
the codegen part for depthwise is still missing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org