You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "Anndrey24 (via GitHub)" <gi...@apache.org> on 2023/11/10 16:28:08 UTC
[PR] [TOPI][Relay] Add conv2d NHWC fp32 hybrid schedule for `arm_cpu` [tvm]
Anndrey24 opened a new pull request, #16106:
URL: https://github.com/apache/tvm/pull/16106
Implemented an `arm_cpu` conv2d NHWC schedule for fp32 using a hybrid GeMM approach, effectively breaking down the matrix multiplication into a macro-kernel (partitioning into fixed-sized, tile-level subproblems) and a micro-kernel (independently dealing with each subproblem). After the im2col transformation, the input matrix is handled natively (not interleaved), while the weights matrix is tiled and interleaved at compile time.
The micro-kernel uses 16 registers to accumulate the results of each 4x16 output tile, cycling through the operands needed to compute them (from the input and weight matrices) in the remaining registers.
There are now two ways to transform the weights matrix for conv2d, which are detailed in `convolution.cc`:
* for fp32: tile, interleave
* for int8: tile, interleave, transpose
To maintain naming consistency across both of these implementations (transposed vs not transposed), all mentions of `tile_rows_B` or `tile_cols_B` have been changed to `tile_N` and `tile_K` respectively to denote the tiling size along each axis of the flattened B matrix. As usual, `N = out_channels` and `K = kernel_width * kernel_height * in_channels`.
I have also added a new conv2d NHWC fp32 test for both the `conv2d_nhwc_spatial_pack` and `conv2d_NHWC_fp32_hybrid` schedules.
cc @ekalda @lhutton1 @neildhickey @leandron
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC fp32 hybrid schedule for `arm_cpu` [tvm]
Posted by "Anndrey24 (via GitHub)" <gi...@apache.org>.
Anndrey24 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1392922308
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -242,7 +242,13 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
),
name="conv2d_NHWC_quantized_interleaved.arm_cpu",
)
- if (not is_aarch64) or (data.dtype not in ["int8", "uint8"]):
+ if is_aarch64 and data.dtype not in ["int8", "uint8"]:
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.arm_cpu.compute_conv2d_NHWC_fp32_hybrid),
+ wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_fp32_hybrid),
+ name="conv2d_NHWC_fp32_hybrid.arm_cpu",
Review Comment:
Done :thumbsup:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "ekalda (via GitHub)" <gi...@apache.org>.
ekalda commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1400918955
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -232,7 +232,7 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_quantized_native),
name="conv2d_NHWC_quantized_native.arm_cpu",
)
- if is_aarch64 and has_asimd:
+ if has_asimd:
Review Comment:
I'd favour making it explicit in the if else block which combination of architecture extensions results in which set of schedules added to the strategy, even though it would create more lines of code. So essentially recreating that bullet point table there, with both schedules added when both i8mm and dotprod are available, in the right priority order. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 merged PR #16106:
URL: https://github.com/apache/tvm/pull/16106
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1400910045
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -232,7 +232,7 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_quantized_native),
name="conv2d_NHWC_quantized_native.arm_cpu",
)
- if is_aarch64 and has_asimd:
+ if has_asimd:
Review Comment:
Perhaps we can do some trickery with priority level instead? e.g. `interleaved_plevel = 10 if has_matmul_i8 else 5`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC fp32 hybrid schedule for `arm_cpu` [tvm]
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1392777716
##########
tests/python/integration/test_arm_aprofile.py:
##########
@@ -49,6 +49,7 @@ def test_conv2d(dtype):
invar,
weight,
kernel_size=kernel_size,
+ channels=2,
Review Comment:
I see thanks. It looks like `Conv2DGemmRel` doesn't deduce the channels while `Conv2dRel` does, hence the reason for the error. Since `Conv2DGemmRel` is used elsewhere and the channels issue hasn't been brought up before, I don't think this needs fixing here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1399229059
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -242,7 +242,14 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
),
name="conv2d_NHWC_quantized_interleaved.arm_cpu",
)
- if (not is_aarch64) or (data.dtype not in ["int8", "uint8"]):
+ # Non-quantized cases
+ if is_aarch64 and data.dtype in ["float32", "float16"]:
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.arm_cpu.compute_conv2d_NHWC_hybrid),
+ wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_hybrid),
+ name="conv2d_NHWC_hybrid.arm_cpu",
+ )
+ if (not is_aarch64) or (data.dtype not in ["int8", "uint8", "float32", "float16"]):
Review Comment:
Apologies for missing this before, it might actually be better to keep the original condition here: `if not is_aarch64 or data.dtype not in ["int8", "uint8"]` and use `add_implementation`'s `plevel` argument to prioritise the hybrid schedule. This way if autoTVM is being used, both schedule options are exposed for selection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "Anndrey24 (via GitHub)" <gi...@apache.org>.
Anndrey24 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1400913110
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -250,6 +257,7 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
wrap_compute_conv2d(topi.arm_cpu.conv2d_nhwc_spatial_pack),
wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_nhwc_spatial_pack),
name="conv2d_nhwc_spatial_pack.arm_cpu",
+ plevel=5,
Review Comment:
The default priority level of an implementation is 10. When two or more implementations are added to the strategy, in the untuned case, the one with the highest priority is chosen. Also, if two implementations have the same plevel, the one added to the strategy first is selected.
So actually, the change doesn't change the strategy selection behaviour at all, since `spatial_pack` is added last in all cases and every schedule has the default priority level, so it would already only get used when no other implementation was added.
I'm not sure if having an explicit plevel makes this more or less understandable. I am happy to remove it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "ekalda (via GitHub)" <gi...@apache.org>.
ekalda commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1400924786
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -250,6 +257,7 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
wrap_compute_conv2d(topi.arm_cpu.conv2d_nhwc_spatial_pack),
wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_nhwc_spatial_pack),
name="conv2d_nhwc_spatial_pack.arm_cpu",
+ plevel=5,
Review Comment:
Thanks for the explanation, I think it makes sense to keep the change, to prevent that schedule accidentally becoming priority schedule in some future refactor of that incredibly confusing monster if else 😅
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -250,6 +257,7 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
wrap_compute_conv2d(topi.arm_cpu.conv2d_nhwc_spatial_pack),
wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_nhwc_spatial_pack),
name="conv2d_nhwc_spatial_pack.arm_cpu",
+ plevel=5,
Review Comment:
Thanks for the explanation, I think it makes sense to keep the change, to prevent that schedule accidentally becoming priority schedule in some future refactor of that incredibly confusing monster if else 😅
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on PR #16106:
URL: https://github.com/apache/tvm/pull/16106#issuecomment-1822903618
@tvm-bot rerun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC fp32 hybrid schedule for `arm_cpu` [tvm]
Posted by "Anndrey24 (via GitHub)" <gi...@apache.org>.
Anndrey24 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1391284385
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -242,7 +242,13 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
),
name="conv2d_NHWC_quantized_interleaved.arm_cpu",
)
- if (not is_aarch64) or (data.dtype not in ["int8", "uint8"]):
+ if is_aarch64 and data.dtype not in ["int8", "uint8"]:
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.arm_cpu.compute_conv2d_NHWC_fp32_hybrid),
+ wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_fp32_hybrid),
+ name="conv2d_NHWC_fp32_hybrid.arm_cpu",
Review Comment:
The schedule should indeed be applicable to other floating point data types too, so I see how this name could be misleading. Would something more general like `"conv2d_NHWC_float_hybrid.arm_cpu"` be preferable in this case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "ekalda (via GitHub)" <gi...@apache.org>.
ekalda commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1400707029
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -232,7 +232,7 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_quantized_native),
name="conv2d_NHWC_quantized_native.arm_cpu",
)
- if is_aarch64 and has_asimd:
+ if has_asimd:
Review Comment:
This condition looks redundant since `has_asimd` is defined as `has_asimd = is_aarch64 || simd_flag` and we're adding the same schedule to the strategy?
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -250,6 +257,7 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
wrap_compute_conv2d(topi.arm_cpu.conv2d_nhwc_spatial_pack),
wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_nhwc_spatial_pack),
name="conv2d_nhwc_spatial_pack.arm_cpu",
+ plevel=5,
Review Comment:
For enlightenment, what is the effect of that change and why 5?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC fp32 hybrid schedule for `arm_cpu` [tvm]
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1390915499
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -242,7 +242,13 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
),
name="conv2d_NHWC_quantized_interleaved.arm_cpu",
)
- if (not is_aarch64) or (data.dtype not in ["int8", "uint8"]):
+ if is_aarch64 and data.dtype not in ["int8", "uint8"]:
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.arm_cpu.compute_conv2d_NHWC_fp32_hybrid),
+ wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_fp32_hybrid),
+ name="conv2d_NHWC_fp32_hybrid.arm_cpu",
+ )
+ else:
Review Comment:
This seems to change the logic and allows the below schedule to be selected when `is_aarch64=True` and `data.dtype in ["int8", "uint8]`, is that intentional?
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -517,9 +523,12 @@ def conv2d_gemm_without_weight_transform_strategy_arm_cpu(attrs, inputs, out_typ
name="conv2d_NHWC_quantized_interleaved_without_transform.arm_cpu",
)
else:
- raise RuntimeError(
- f"Unsupported conv2d_NHWC_quantized_without_transform layout {layout}"
- f"with datatype {data.dtype}"
+ strategy.add_implementation(
Review Comment:
I think there needs to be an `is_aarch64` check here?
As a side note, I wonder if and how this schedule performs on a non-aarch64 device? Maybe we can enable it in addition to the `spatial_pack` schedule. Let's consider it in a separate change though
##########
python/tvm/topi/arm_cpu/arm_utils.py:
##########
@@ -36,59 +36,68 @@ def get_tiling_B_interleaved_t(interleave_A):
Parameters
----------
- interleave_A: bool
- determines if A is expected to be interleaved
+ interleave_A : bool
+ determines if A is expected to be interleaved
+ in_dtype : str
+ input datatype
Review Comment:
nit: I think the norm is to use one space after the ":" and one indent before the sentence i.e.
```suggestion
interleave_A : bool
determines if A is expected to be interleaved
in_dtype : str
input datatype
```
##########
python/tvm/topi/arm_cpu/conv2d_gemm.py:
##########
@@ -90,7 +91,7 @@ def compute_conv2d_gemm_without_weight_transform(
OH = (IH + pad_top + pad_down - dilated_kernel_h) // HSTR + 1
OW = (IW + pad_left + pad_right - dilated_kernel_w) // WSTR + 1
- if pad_top or pad_left:
+ if pad_top or pad_left or pad_down or pad_right:
Review Comment:
Is there a test that covers this fix?
##########
tests/python/integration/test_arm_aprofile.py:
##########
@@ -49,6 +49,7 @@ def test_conv2d(dtype):
invar,
weight,
kernel_size=kernel_size,
+ channels=2,
Review Comment:
curious why this change is needed?
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -242,7 +242,13 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
),
name="conv2d_NHWC_quantized_interleaved.arm_cpu",
)
- if (not is_aarch64) or (data.dtype not in ["int8", "uint8"]):
+ if is_aarch64 and data.dtype not in ["int8", "uint8"]:
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.arm_cpu.compute_conv2d_NHWC_fp32_hybrid),
+ wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_fp32_hybrid),
+ name="conv2d_NHWC_fp32_hybrid.arm_cpu",
Review Comment:
The schedule name suggests it can only be used for fp32, although it seems applicable to other types as well like fp16. I think we either need to check the types here i.e. `data.dtype == "float32"` or rename the schedule
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC fp32 hybrid schedule for `arm_cpu` [tvm]
Posted by "Anndrey24 (via GitHub)" <gi...@apache.org>.
Anndrey24 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1391400438
##########
tests/python/integration/test_arm_aprofile.py:
##########
@@ -49,6 +49,7 @@ def test_conv2d(dtype):
invar,
weight,
kernel_size=kernel_size,
+ channels=2,
Review Comment:
Without this change I would get an error during the AlterOpLayout pass (which is applied since we use `opt_level=3` to build the IRModule slightly below that line):
`InternalError: Check failed: (param->kernel_size.defined() && param->channels.defined()) is false: The kernel size and channels of a Conv must be set or inferred by previous pass
` which stems from this line: https://github.com/apache/tvm/blob/5e32bf0adf4b8f242bf32a3d3da598cf3083b1ac/src/relay/op/nn/convolution.cc#L1411-L1412
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC fp32 hybrid schedule for `arm_cpu` [tvm]
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1392581365
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -242,7 +242,13 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
),
name="conv2d_NHWC_quantized_interleaved.arm_cpu",
)
- if (not is_aarch64) or (data.dtype not in ["int8", "uint8"]):
+ if is_aarch64 and data.dtype not in ["int8", "uint8"]:
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.arm_cpu.compute_conv2d_NHWC_fp32_hybrid),
+ wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_fp32_hybrid),
+ name="conv2d_NHWC_fp32_hybrid.arm_cpu",
Review Comment:
I think we could go even more generic: "conv2d_NHWC_hybrid.arm_cpu"?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "Anndrey24 (via GitHub)" <gi...@apache.org>.
Anndrey24 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1399284730
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -242,7 +242,14 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
),
name="conv2d_NHWC_quantized_interleaved.arm_cpu",
)
- if (not is_aarch64) or (data.dtype not in ["int8", "uint8"]):
+ # Non-quantized cases
+ if is_aarch64 and data.dtype in ["float32", "float16"]:
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.arm_cpu.compute_conv2d_NHWC_hybrid),
+ wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_hybrid),
+ name="conv2d_NHWC_hybrid.arm_cpu",
+ )
+ if (not is_aarch64) or (data.dtype not in ["int8", "uint8", "float32", "float16"]):
Review Comment:
That makes sense, I'll make the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC fp32 hybrid schedule for `arm_cpu` [tvm]
Posted by "Anndrey24 (via GitHub)" <gi...@apache.org>.
Anndrey24 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1391365234
##########
python/tvm/topi/arm_cpu/conv2d_gemm.py:
##########
@@ -90,7 +91,7 @@ def compute_conv2d_gemm_without_weight_transform(
OH = (IH + pad_top + pad_down - dilated_kernel_h) // HSTR + 1
OW = (IW + pad_left + pad_right - dilated_kernel_w) // WSTR + 1
- if pad_top or pad_left:
+ if pad_top or pad_left or pad_down or pad_right:
Review Comment:
This test case was failing before, since it was padding bottom and right, while the schedule would only check top and left: https://github.com/apache/tvm/blob/5e32bf0adf4b8f242bf32a3d3da598cf3083b1ac/tests/python/topi/python/test_topi_conv2d_nhwc.py#L71
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC fp32 hybrid schedule for `arm_cpu` [tvm]
Posted by "Anndrey24 (via GitHub)" <gi...@apache.org>.
Anndrey24 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1391265708
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -242,7 +242,13 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
),
name="conv2d_NHWC_quantized_interleaved.arm_cpu",
)
- if (not is_aarch64) or (data.dtype not in ["int8", "uint8"]):
+ if is_aarch64 and data.dtype not in ["int8", "uint8"]:
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.arm_cpu.compute_conv2d_NHWC_fp32_hybrid),
+ wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_fp32_hybrid),
+ name="conv2d_NHWC_fp32_hybrid.arm_cpu",
+ )
+ else:
Review Comment:
It is mostly an oversight on my part, since under these two conditions `conv2d_NHWC_quantized_interleaved.arm_cpu` would have already been added to the strategy first and I only considered the cases where `conv2d_nhwc_spatial_pack.arm_cpu` was the only viable, and thus selected, strategy:
- `is_aarch64=False` and `data.dtype not in ["int8", "uint8"]`
- `is_aarch64=False` and `data.dtype in ["int8", "uint8"]`
While I think the selected implementation is still miraculously correct in the end, as this test case still passes: https://github.com/apache/tvm/blob/f719151486c452d0c5c44339c3578bf64711ef29/tests/python/relay/strategy/test_select_implementation.py#L73-L74, I do agree that the strategy selection is overly convoluted and unclear now, so I will move the conditions around a bit to try and fix that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1400765330
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -232,7 +232,7 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_quantized_native),
name="conv2d_NHWC_quantized_native.arm_cpu",
)
- if is_aarch64 and has_asimd:
+ if has_asimd:
Review Comment:
Oops, this redundancy was introduced in https://github.com/apache/tvm/pull/15468 😓
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -232,7 +232,7 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_quantized_native),
name="conv2d_NHWC_quantized_native.arm_cpu",
)
- if is_aarch64 and has_asimd:
+ if has_asimd:
Review Comment:
Oops, this redundancy was introduced in https://github.com/apache/tvm/pull/15468 😓
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "Anndrey24 (via GitHub)" <gi...@apache.org>.
Anndrey24 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1401844898
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -232,7 +232,7 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_quantized_native),
name="conv2d_NHWC_quantized_native.arm_cpu",
)
- if is_aarch64 and has_asimd:
+ if has_asimd:
Review Comment:
Sure, having a separate branch for each architecture extension combination should make it easier to tell at a glance which schedules it supports, at the cost of some repeated code.
While I'm at it, I will make this change for the `conv2d_gemm_without_weight_transform_strategy_arm_cpu` strategy too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on PR #16106:
URL: https://github.com/apache/tvm/pull/16106#issuecomment-1824039677
@tvm-bot rerun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC fp32 hybrid schedule for `arm_cpu` [tvm]
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on PR #16106:
URL: https://github.com/apache/tvm/pull/16106#issuecomment-1812161162
@tvm-bot rerun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "Anndrey24 (via GitHub)" <gi...@apache.org>.
Anndrey24 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1400897859
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -232,7 +232,7 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_quantized_native),
name="conv2d_NHWC_quantized_native.arm_cpu",
)
- if is_aarch64 and has_asimd:
+ if has_asimd:
Review Comment:
Right, at this point we can remove the `if has_asimd`condition completely, so I'll at least do that.
From my understanding, the reason we are adding the implementations as "interleaved", "native", and then "interleaved" again is to make use of the fact that when two implementations have the same priority level, the one added to the strategy first is selected by default.
We currently want the default to be:
- both i8mm & dotprod enabled: interleaved
- i8mm: interleaved
- dotprod: native
- neither (only aarch64): interleaved
We could replace the entire int8 strategy selection with
```
if has_dot_prod and not has_matmul_i8:
strategy.add_implementation(
wrap_compute_conv2d(topi.arm_cpu.compute_conv2d_NHWC_quantized_native),
wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_quantized_native),
name="conv2d_NHWC_quantized_native.arm_cpu",
)
strategy.add_implementation(
wrap_compute_conv2d(
topi.arm_cpu.compute_conv2d_NHWC_quantized_interleaved
),
wrap_topi_schedule(
topi.arm_cpu.schedule_conv2d_NHWC_quantized_interleaved
),
name="conv2d_NHWC_quantized_interleaved.arm_cpu",
)
```
but I guess this wouldn't allow autoTVM to select between the 2 implementations when both i8mm and dotprod are enabled.
Which way do you think is best?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC fp32 hybrid schedule for `arm_cpu` [tvm]
Posted by "Anndrey24 (via GitHub)" <gi...@apache.org>.
Anndrey24 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1391265708
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -242,7 +242,13 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
),
name="conv2d_NHWC_quantized_interleaved.arm_cpu",
)
- if (not is_aarch64) or (data.dtype not in ["int8", "uint8"]):
+ if is_aarch64 and data.dtype not in ["int8", "uint8"]:
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.arm_cpu.compute_conv2d_NHWC_fp32_hybrid),
+ wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_fp32_hybrid),
+ name="conv2d_NHWC_fp32_hybrid.arm_cpu",
+ )
+ else:
Review Comment:
It is mostly an oversight on my part, since under these two conditions `conv2d_NHWC_quantized_interleaved.arm_cpu` would have already been added to the strategy first and I only considered the cases where `conv2d_nhwc_spatial_pack.arm_cpu` was the only viable, and thus selected, strategy:
- `is_aarch64=False` and `data.dtype not in ["int8", "uint8"]`
- `is_aarch64=False` and `data.dtype in ["int8", "uint8"]`
While I think the selected implementation is still correct in the end, as this test case still passes: https://github.com/apache/tvm/blob/f719151486c452d0c5c44339c3578bf64711ef29/tests/python/relay/strategy/test_select_implementation.py#L74, I do agree that the strategy selection is overly convoluted now, so I will move the conditions around a bit to try and fix that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC fp32 hybrid schedule for `arm_cpu` [tvm]
Posted by "Anndrey24 (via GitHub)" <gi...@apache.org>.
Anndrey24 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1391265708
##########
python/tvm/relay/op/strategy/arm_cpu.py:
##########
@@ -242,7 +242,13 @@ def conv2d_strategy_arm_cpu(attrs, inputs, out_type, target):
),
name="conv2d_NHWC_quantized_interleaved.arm_cpu",
)
- if (not is_aarch64) or (data.dtype not in ["int8", "uint8"]):
+ if is_aarch64 and data.dtype not in ["int8", "uint8"]:
+ strategy.add_implementation(
+ wrap_compute_conv2d(topi.arm_cpu.compute_conv2d_NHWC_fp32_hybrid),
+ wrap_topi_schedule(topi.arm_cpu.schedule_conv2d_NHWC_fp32_hybrid),
+ name="conv2d_NHWC_fp32_hybrid.arm_cpu",
+ )
+ else:
Review Comment:
It is mostly an oversight on my part, since under these two conditions `conv2d_NHWC_quantized_interleaved.arm_cpu` would have already been added to the strategy first and I only considered the cases where `conv2d_nhwc_spatial_pack.arm_cpu` was the only viable, and thus selected, strategy:
- `is_aarch64=False` and `data.dtype not in ["int8", "uint8"]`
- `is_aarch64=False` and `data.dtype in ["int8", "uint8"]`
While I think the selected implementation is still correct in the end, as this test case still passes: https://github.com/apache/tvm/blob/f719151486c452d0c5c44339c3578bf64711ef29/tests/python/relay/strategy/test_select_implementation.py#L73-L74, I do agree that the strategy selection is overly convoluted now, so I will move the conditions around a bit to try and fix that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC fp32 hybrid schedule for `arm_cpu` [tvm]
Posted by "Anndrey24 (via GitHub)" <gi...@apache.org>.
Anndrey24 commented on code in PR #16106:
URL: https://github.com/apache/tvm/pull/16106#discussion_r1391365234
##########
python/tvm/topi/arm_cpu/conv2d_gemm.py:
##########
@@ -90,7 +91,7 @@ def compute_conv2d_gemm_without_weight_transform(
OH = (IH + pad_top + pad_down - dilated_kernel_h) // HSTR + 1
OW = (IW + pad_left + pad_right - dilated_kernel_w) // WSTR + 1
- if pad_top or pad_left:
+ if pad_top or pad_left or pad_down or pad_right:
Review Comment:
This test case would have failed before, since it was padding bottom and right, while the schedule would only check top and left: https://github.com/apache/tvm/blob/5e32bf0adf4b8f242bf32a3d3da598cf3083b1ac/tests/python/topi/python/test_topi_conv2d_nhwc.py#L71
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [TOPI][Relay] Add conv2d NHWC hybrid schedule for `arm_cpu` [tvm]
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on PR #16106:
URL: https://github.com/apache/tvm/pull/16106#issuecomment-1825413964
Thanks @Anndrey24 @ekalda!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org