You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "Alexey-Yazev (via GitHub)" <gi...@apache.org> on 2023/02/15 08:59:33 UTC
[GitHub] [tvm] Alexey-Yazev opened a new pull request, #13997: [microNPU] Sum legalization support
Alexey-Yazev opened a new pull request, #13997:
URL: https://github.com/apache/tvm/pull/13997
Supports legalizing a relay sum operation to an equivalent series of NPU operations. It supports case with int8 output type and channel axis.
cc @leandron, @ekalda, @lhutton1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Alexey-Yazev commented on a diff in pull request #13997: [microNPU] Sum legalization support
Posted by "Alexey-Yazev (via GitHub)" <gi...@apache.org>.
Alexey-Yazev commented on code in PR #13997:
URL: https://github.com/apache/tvm/pull/13997#discussion_r1106857681
##########
src/relay/op/contrib/ethosu/pooling.cc:
##########
@@ -46,14 +46,27 @@ bool EthosuPoolingRel(const Array<Type>& types, int num_inputs, const Attrs& att
const String operator_name = "ethosu_pooling";
- if (param->pooling_type != "AVG" && param->pooling_type != "MAX") {
+ if (param->pooling_type != "AVG" && param->pooling_type != "MAX" &&
+ param->pooling_type != "SUM") {
reporter->GetDiagCtx().EmitFatal(Diagnostic::Error(reporter->GetSpan())
<< "Invalid operator: expected " << operator_name
- << " type 'AVG' or 'MAX' but was " << param->pooling_type);
+ << " type 'AVG', 'MAX', or 'SUM' but was "
+ << param->pooling_type);
return false;
}
- CheckDataType(reporter, ifm->dtype, {DataType::UInt(8), DataType::Int(8)}, operator_name, "ifm",
Review Comment:
Should types be {uint8, int8, int16} as specified in TRM?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Alexey-Yazev commented on a diff in pull request #13997: [microNPU] Sum legalization support
Posted by "Alexey-Yazev (via GitHub)" <gi...@apache.org>.
Alexey-Yazev commented on code in PR #13997:
URL: https://github.com/apache/tvm/pull/13997#discussion_r1118436931
##########
python/tvm/relay/op/contrib/ethosu.py:
##########
@@ -1375,6 +1375,82 @@ def mean_pattern() -> tvm.relay.dataflow_pattern.DFPattern:
return pattern
+class SumParams:
+ """
+ This class will parse a call to ethosu.sum composite function
+ and extract the parameter information.
+ """
+
+ composite_name = "ethos-u.sum"
+
+ def __init__(self, func_body: Call):
+ from tvm.relay.backend.contrib.ethosu.util import RequantArgs
+
+ requantize = func_body
+ sum_op = requantize.args[0]
+ attrs = sum_op.attrs
+ cast = sum_op.args[0]
+
+ layout = "NHWC"
+ self.ifm = TensorParams(
+ cast.args[0],
+ layout,
+ requantize.args[RequantArgs.IFM_SCALE.value],
+ requantize.args[RequantArgs.IFM_ZERO_POINT.value],
+ )
+ self.ofm = TensorParams(
+ requantize,
+ layout,
+ requantize.args[RequantArgs.OFM_SCALE.value],
+ requantize.args[RequantArgs.OFM_ZERO_POINT.value],
+ )
+
+ ifm_shape = self.ifm.shape
+ self.height = ifm_shape[0] if len(ifm_shape) in (2, 3) else ifm_shape[1]
+ self.width = ifm_shape[1] if len(ifm_shape) in (2, 3) else ifm_shape[2]
+ self.keepdims = attrs.keepdims
+
+ self.axis = list(sorted(attrs.axis))
+ if attrs.exclude:
+ self.axis = [i for i in range(len(self.ifm.shape)) if i not in self.axis]
+
+ def is_valid(self) -> bool:
+ """
+ Checks whether Sum has compatible attributes with HW.
+ """
+
+ ifm_shape_len = len(self.ifm.shape)
+
+ if not check_valid_dtypes([self.ifm], [np.uint8, np.int8, np.int16, np.int32]):
+ return False
+ if not check_valid_dtypes([self.ofm], [np.int8]):
Review Comment:
I use MUL operation but there are some problems for int16 output type that require further investigation, so now only int8 type support has been added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] sergey-grovety commented on a diff in pull request #13997: [microNPU] Sum legalization support
Posted by "sergey-grovety (via GitHub)" <gi...@apache.org>.
sergey-grovety commented on code in PR #13997:
URL: https://github.com/apache/tvm/pull/13997#discussion_r1109348815
##########
python/tvm/relay/op/contrib/ethosu.py:
##########
@@ -1375,6 +1375,82 @@ def mean_pattern() -> tvm.relay.dataflow_pattern.DFPattern:
return pattern
+class SumParams:
+ """
+ This class will parse a call to ethosu.sum composite function
+ and extract the parameter information.
+ """
+
+ composite_name = "ethos-u.sum"
+
+ def __init__(self, func_body: Call):
+ from tvm.relay.backend.contrib.ethosu.util import RequantArgs
+
+ requantize = func_body
+ sum_op = requantize.args[0]
+ attrs = sum_op.attrs
+ cast = sum_op.args[0]
+
+ layout = "NHWC"
+ self.ifm = TensorParams(
+ cast.args[0],
+ layout,
+ requantize.args[RequantArgs.IFM_SCALE.value],
+ requantize.args[RequantArgs.IFM_ZERO_POINT.value],
+ )
+ self.ofm = TensorParams(
+ requantize,
+ layout,
+ requantize.args[RequantArgs.OFM_SCALE.value],
+ requantize.args[RequantArgs.OFM_ZERO_POINT.value],
+ )
+
+ ifm_shape = self.ifm.shape
+ self.height = ifm_shape[0] if len(ifm_shape) in (2, 3) else ifm_shape[1]
+ self.width = ifm_shape[1] if len(ifm_shape) in (2, 3) else ifm_shape[2]
+ self.keepdims = attrs.keepdims
+
+ self.axis = list(sorted(attrs.axis))
+ if attrs.exclude:
+ self.axis = [i for i in range(len(self.ifm.shape)) if i not in self.axis]
+
+ def is_valid(self) -> bool:
+ """
+ Checks whether Sum has compatible attributes with HW.
+ """
+
+ ifm_shape_len = len(self.ifm.shape)
+
+ if not check_valid_dtypes([self.ifm], [np.uint8, np.int8, np.int16, np.int32]):
+ return False
+ if not check_valid_dtypes([self.ofm], [np.int8]):
+ return False
+ if not ifm_shape_len in (3, 4):
+ return False
+ if ifm_shape_len == 3 and self.axis not in [[2]]:
+ return False
+ if ifm_shape_len == 4 and self.axis not in [[3]]:
+ return False
+
+ return True
+
+
+def sum_pattern() -> tvm.relay.dataflow_pattern.DFPattern:
Review Comment:
Could we add optional clip to the pattern?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 merged pull request #13997: [microNPU] Sum legalization support
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 merged PR #13997:
URL: https://github.com/apache/tvm/pull/13997
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on a diff in pull request #13997: [microNPU] Sum legalization support
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on code in PR #13997:
URL: https://github.com/apache/tvm/pull/13997#discussion_r1111691912
##########
python/tvm/relay/op/contrib/ethosu.py:
##########
@@ -1375,6 +1375,82 @@ def mean_pattern() -> tvm.relay.dataflow_pattern.DFPattern:
return pattern
+class SumParams:
+ """
+ This class will parse a call to ethosu.sum composite function
+ and extract the parameter information.
+ """
+
+ composite_name = "ethos-u.sum"
+
+ def __init__(self, func_body: Call):
+ from tvm.relay.backend.contrib.ethosu.util import RequantArgs
+
+ requantize = func_body
+ sum_op = requantize.args[0]
+ attrs = sum_op.attrs
+ cast = sum_op.args[0]
+
+ layout = "NHWC"
+ self.ifm = TensorParams(
+ cast.args[0],
+ layout,
+ requantize.args[RequantArgs.IFM_SCALE.value],
+ requantize.args[RequantArgs.IFM_ZERO_POINT.value],
+ )
+ self.ofm = TensorParams(
+ requantize,
+ layout,
+ requantize.args[RequantArgs.OFM_SCALE.value],
+ requantize.args[RequantArgs.OFM_ZERO_POINT.value],
+ )
+
+ ifm_shape = self.ifm.shape
+ self.height = ifm_shape[0] if len(ifm_shape) in (2, 3) else ifm_shape[1]
+ self.width = ifm_shape[1] if len(ifm_shape) in (2, 3) else ifm_shape[2]
+ self.keepdims = attrs.keepdims
+
+ self.axis = list(sorted(attrs.axis))
+ if attrs.exclude:
+ self.axis = [i for i in range(len(self.ifm.shape)) if i not in self.axis]
+
+ def is_valid(self) -> bool:
+ """
+ Checks whether Sum has compatible attributes with HW.
+ """
+
+ ifm_shape_len = len(self.ifm.shape)
+
+ if not check_valid_dtypes([self.ifm], [np.uint8, np.int8, np.int16, np.int32]):
+ return False
+ if not check_valid_dtypes([self.ofm], [np.int8]):
Review Comment:
Thanks that makes sense, in this case it's likely better to use the ADD operation instead since we can support more output data types - I'll double check on this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tvm-bot commented on pull request #13997: [microNPU] Sum legalization support
Posted by "tvm-bot (via GitHub)" <gi...@apache.org>.
tvm-bot commented on PR #13997:
URL: https://github.com/apache/tvm/pull/13997#issuecomment-1430970638
<!---bot-comment-->
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
<!--bot-comment-ccs-start-->
* cc @Mousius, @leandron, @lhutton1 <sub>See [#10317](https://github.com/apache/tvm/issues/10317) for details</sub><!--bot-comment-ccs-end-->
<sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on a diff in pull request #13997: [microNPU] Sum legalization support
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on code in PR #13997:
URL: https://github.com/apache/tvm/pull/13997#discussion_r1119056932
##########
python/tvm/relay/backend/contrib/ethosu/te/pooling.py:
##########
@@ -110,18 +110,21 @@ def pooling_compute(
padding = [int(v) for v in padding]
stride_h, stride_w = [int(v) for v in strides]
pool_shape_h, pool_shape_w = [int(v) for v in pool_shape]
+ ifm_channels = ofm_channels if pooling_type != "SUM" else ifm.shape[-1]
Review Comment:
Won't block on this as the functionality remains the same, but wondering if `ofm_channels` ever made sense here and whether we can just use `ifm.shape[-1]` for all pooling types?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Alexey-Yazev commented on a diff in pull request #13997: [microNPU] Sum legalization support
Posted by "Alexey-Yazev (via GitHub)" <gi...@apache.org>.
Alexey-Yazev commented on code in PR #13997:
URL: https://github.com/apache/tvm/pull/13997#discussion_r1111577237
##########
src/relay/op/contrib/ethosu/pooling.cc:
##########
@@ -46,14 +46,27 @@ bool EthosuPoolingRel(const Array<Type>& types, int num_inputs, const Attrs& att
const String operator_name = "ethosu_pooling";
- if (param->pooling_type != "AVG" && param->pooling_type != "MAX") {
+ if (param->pooling_type != "AVG" && param->pooling_type != "MAX" &&
+ param->pooling_type != "SUM") {
reporter->GetDiagCtx().EmitFatal(Diagnostic::Error(reporter->GetSpan())
<< "Invalid operator: expected " << operator_name
- << " type 'AVG' or 'MAX' but was " << param->pooling_type);
+ << " type 'AVG', 'MAX', or 'SUM' but was "
+ << param->pooling_type);
return false;
}
- CheckDataType(reporter, ifm->dtype, {DataType::UInt(8), DataType::Int(8)}, operator_name, "ifm",
+ auto max_avg_pooling_ifm_dtypes = {DataType::UInt(8), DataType::Int(8)};
+ auto sum_pooling_ifm_dtypes = {DataType::UInt(8), DataType::Int(8), DataType::Int(16),
Review Comment:
C++ code style says:
> Favor concrete type declaration over auto as long as it is short.
How short should it be?
Is it ok to use
`std::initializer_list<DataType>`
or better
```
using DTypes = std::initializer_list<DataType>;
DTypes sum_pooling_ifm_dtypes
```
?
##########
python/tvm/relay/op/contrib/ethosu.py:
##########
@@ -1375,6 +1375,82 @@ def mean_pattern() -> tvm.relay.dataflow_pattern.DFPattern:
return pattern
+class SumParams:
+ """
+ This class will parse a call to ethosu.sum composite function
+ and extract the parameter information.
+ """
+
+ composite_name = "ethos-u.sum"
+
+ def __init__(self, func_body: Call):
+ from tvm.relay.backend.contrib.ethosu.util import RequantArgs
+
+ requantize = func_body
+ sum_op = requantize.args[0]
+ attrs = sum_op.attrs
+ cast = sum_op.args[0]
+
+ layout = "NHWC"
+ self.ifm = TensorParams(
+ cast.args[0],
+ layout,
+ requantize.args[RequantArgs.IFM_SCALE.value],
+ requantize.args[RequantArgs.IFM_ZERO_POINT.value],
+ )
+ self.ofm = TensorParams(
+ requantize,
+ layout,
+ requantize.args[RequantArgs.OFM_SCALE.value],
+ requantize.args[RequantArgs.OFM_ZERO_POINT.value],
+ )
+
+ ifm_shape = self.ifm.shape
+ self.height = ifm_shape[0] if len(ifm_shape) in (2, 3) else ifm_shape[1]
+ self.width = ifm_shape[1] if len(ifm_shape) in (2, 3) else ifm_shape[2]
+ self.keepdims = attrs.keepdims
+
+ self.axis = list(sorted(attrs.axis))
+ if attrs.exclude:
+ self.axis = [i for i in range(len(self.ifm.shape)) if i not in self.axis]
+
+ def is_valid(self) -> bool:
+ """
+ Checks whether Sum has compatible attributes with HW.
+ """
+
+ ifm_shape_len = len(self.ifm.shape)
+
+ if not check_valid_dtypes([self.ifm], [np.uint8, np.int8, np.int16, np.int32]):
+ return False
+ if not check_valid_dtypes([self.ofm], [np.int8]):
+ return False
+ if not ifm_shape_len in (3, 4):
+ return False
+ if ifm_shape_len == 3 and self.axis not in [[2]]:
+ return False
+ if ifm_shape_len == 4 and self.axis not in [[3]]:
+ return False
+
+ return True
+
+
+def sum_pattern() -> tvm.relay.dataflow_pattern.DFPattern:
Review Comment:
Yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on a diff in pull request #13997: [microNPU] Sum legalization support
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on code in PR #13997:
URL: https://github.com/apache/tvm/pull/13997#discussion_r1108263890
##########
src/relay/op/contrib/ethosu/pooling.cc:
##########
@@ -46,14 +46,27 @@ bool EthosuPoolingRel(const Array<Type>& types, int num_inputs, const Attrs& att
const String operator_name = "ethosu_pooling";
- if (param->pooling_type != "AVG" && param->pooling_type != "MAX") {
+ if (param->pooling_type != "AVG" && param->pooling_type != "MAX" &&
+ param->pooling_type != "SUM") {
reporter->GetDiagCtx().EmitFatal(Diagnostic::Error(reporter->GetSpan())
<< "Invalid operator: expected " << operator_name
- << " type 'AVG' or 'MAX' but was " << param->pooling_type);
+ << " type 'AVG', 'MAX', or 'SUM' but was "
+ << param->pooling_type);
return false;
}
- CheckDataType(reporter, ifm->dtype, {DataType::UInt(8), DataType::Int(8)}, operator_name, "ifm",
+ auto max_avg_pooling_ifm_dtypes = {DataType::UInt(8), DataType::Int(8)};
+ auto sum_pooling_ifm_dtypes = {DataType::UInt(8), DataType::Int(8), DataType::Int(16),
Review Comment:
Nit: prefer types over `auto`
##########
python/tvm/relay/op/contrib/ethosu.py:
##########
@@ -1375,6 +1375,82 @@ def mean_pattern() -> tvm.relay.dataflow_pattern.DFPattern:
return pattern
+class SumParams:
+ """
+ This class will parse a call to ethosu.sum composite function
+ and extract the parameter information.
+ """
+
+ composite_name = "ethos-u.sum"
+
+ def __init__(self, func_body: Call):
+ from tvm.relay.backend.contrib.ethosu.util import RequantArgs
+
+ requantize = func_body
+ sum_op = requantize.args[0]
+ attrs = sum_op.attrs
+ cast = sum_op.args[0]
+
+ layout = "NHWC"
+ self.ifm = TensorParams(
+ cast.args[0],
+ layout,
+ requantize.args[RequantArgs.IFM_SCALE.value],
+ requantize.args[RequantArgs.IFM_ZERO_POINT.value],
+ )
+ self.ofm = TensorParams(
+ requantize,
+ layout,
+ requantize.args[RequantArgs.OFM_SCALE.value],
+ requantize.args[RequantArgs.OFM_ZERO_POINT.value],
+ )
+
+ ifm_shape = self.ifm.shape
+ self.height = ifm_shape[0] if len(ifm_shape) in (2, 3) else ifm_shape[1]
+ self.width = ifm_shape[1] if len(ifm_shape) in (2, 3) else ifm_shape[2]
+ self.keepdims = attrs.keepdims
+
+ self.axis = list(sorted(attrs.axis))
+ if attrs.exclude:
+ self.axis = [i for i in range(len(self.ifm.shape)) if i not in self.axis]
+
+ def is_valid(self) -> bool:
+ """
+ Checks whether Sum has compatible attributes with HW.
+ """
+
+ ifm_shape_len = len(self.ifm.shape)
+
+ if not check_valid_dtypes([self.ifm], [np.uint8, np.int8, np.int16, np.int32]):
+ return False
+ if not check_valid_dtypes([self.ofm], [np.int8]):
Review Comment:
Curious if it also possible to support other output data types? Would it just involve a change to the output data-type in the legalized shift operation? (guessing it's more involved than that)
##########
python/tvm/relay/backend/contrib/ethosu/te/pooling.py:
##########
@@ -110,18 +110,21 @@ def pooling_compute(
padding = [int(v) for v in padding]
stride_h, stride_w = [int(v) for v in strides]
pool_shape_h, pool_shape_w = [int(v) for v in pool_shape]
+ ifm_channels = ofm_channels if pooling_type != "SUM" else ifm.shape[-1]
upscale_factor = 2 if upscale != "NONE" else 1
# Compute operation for the IFM DMA pipeline
dmaed_ifm = dma_ifm_compute(
- ifm, ifm_layout, ifm_zero_point, ifm_scale, ofm_channels, padding, upscale_factor
+ ifm, ifm_layout, ifm_zero_point, ifm_scale, ifm_channels, padding, upscale_factor
)
# Pooling compute operation
ofm_height = (dmaed_ifm.shape[1] - pool_shape_h) // stride_h + 1
ofm_width = (dmaed_ifm.shape[2] - pool_shape_w) // stride_w + 1
rh = te.reduce_axis((0, pool_shape_h), name="ry")
rw = te.reduce_axis((0, pool_shape_w), name="rx")
+ rc = te.reduce_axis((0, 1 if pooling_type != "SUM" else ifm_channels), name="rc")
+ ofm_dtype = ifm.dtype if pooling_type != "SUM" else "int32"
Review Comment:
According to the TRM, the AVG pool output datatype can be any of {uint8, int8, int16} without regard to the input datatype - it's okay to leave like this for now, just noting as something that might come up later
##########
src/relay/op/contrib/ethosu/pooling.cc:
##########
@@ -46,14 +46,27 @@ bool EthosuPoolingRel(const Array<Type>& types, int num_inputs, const Attrs& att
const String operator_name = "ethosu_pooling";
- if (param->pooling_type != "AVG" && param->pooling_type != "MAX") {
+ if (param->pooling_type != "AVG" && param->pooling_type != "MAX" &&
+ param->pooling_type != "SUM") {
reporter->GetDiagCtx().EmitFatal(Diagnostic::Error(reporter->GetSpan())
<< "Invalid operator: expected " << operator_name
- << " type 'AVG' or 'MAX' but was " << param->pooling_type);
+ << " type 'AVG', 'MAX', or 'SUM' but was "
+ << param->pooling_type);
return false;
}
- CheckDataType(reporter, ifm->dtype, {DataType::UInt(8), DataType::Int(8)}, operator_name, "ifm",
Review Comment:
Yes I believe they should be, its likely just a case of not having needed `int16` previously, but I think it would be good to add :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on pull request #13997: [microNPU] Sum legalization support
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on PR #13997:
URL: https://github.com/apache/tvm/pull/13997#issuecomment-1447862452
Thanks @Alexey-Yazev, @sergey-grovety!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on a diff in pull request #13997: [microNPU] Sum legalization support
Posted by "lhutton1 (via GitHub)" <gi...@apache.org>.
lhutton1 commented on code in PR #13997:
URL: https://github.com/apache/tvm/pull/13997#discussion_r1111659042
##########
src/relay/op/contrib/ethosu/pooling.cc:
##########
@@ -46,14 +46,27 @@ bool EthosuPoolingRel(const Array<Type>& types, int num_inputs, const Attrs& att
const String operator_name = "ethosu_pooling";
- if (param->pooling_type != "AVG" && param->pooling_type != "MAX") {
+ if (param->pooling_type != "AVG" && param->pooling_type != "MAX" &&
+ param->pooling_type != "SUM") {
reporter->GetDiagCtx().EmitFatal(Diagnostic::Error(reporter->GetSpan())
<< "Invalid operator: expected " << operator_name
- << " type 'AVG' or 'MAX' but was " << param->pooling_type);
+ << " type 'AVG', 'MAX', or 'SUM' but was "
+ << param->pooling_type);
return false;
}
- CheckDataType(reporter, ifm->dtype, {DataType::UInt(8), DataType::Int(8)}, operator_name, "ifm",
+ auto max_avg_pooling_ifm_dtypes = {DataType::UInt(8), DataType::Int(8)};
+ auto sum_pooling_ifm_dtypes = {DataType::UInt(8), DataType::Int(8), DataType::Int(16),
Review Comment:
I think it's okay to use `std::initializer_list<DataType>` in this instance
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Alexey-Yazev commented on a diff in pull request #13997: [microNPU] Sum legalization support
Posted by "Alexey-Yazev (via GitHub)" <gi...@apache.org>.
Alexey-Yazev commented on code in PR #13997:
URL: https://github.com/apache/tvm/pull/13997#discussion_r1118437368
##########
src/relay/op/contrib/ethosu/pooling.cc:
##########
@@ -46,14 +46,27 @@ bool EthosuPoolingRel(const Array<Type>& types, int num_inputs, const Attrs& att
const String operator_name = "ethosu_pooling";
- if (param->pooling_type != "AVG" && param->pooling_type != "MAX") {
+ if (param->pooling_type != "AVG" && param->pooling_type != "MAX" &&
+ param->pooling_type != "SUM") {
reporter->GetDiagCtx().EmitFatal(Diagnostic::Error(reporter->GetSpan())
<< "Invalid operator: expected " << operator_name
- << " type 'AVG' or 'MAX' but was " << param->pooling_type);
+ << " type 'AVG', 'MAX', or 'SUM' but was "
+ << param->pooling_type);
return false;
}
- CheckDataType(reporter, ifm->dtype, {DataType::UInt(8), DataType::Int(8)}, operator_name, "ifm",
+ auto max_avg_pooling_ifm_dtypes = {DataType::UInt(8), DataType::Int(8)};
+ auto sum_pooling_ifm_dtypes = {DataType::UInt(8), DataType::Int(8), DataType::Int(16),
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Alexey-Yazev commented on a diff in pull request #13997: [microNPU] Sum legalization support
Posted by "Alexey-Yazev (via GitHub)" <gi...@apache.org>.
Alexey-Yazev commented on code in PR #13997:
URL: https://github.com/apache/tvm/pull/13997#discussion_r1118436618
##########
src/relay/op/contrib/ethosu/pooling.cc:
##########
@@ -46,14 +46,27 @@ bool EthosuPoolingRel(const Array<Type>& types, int num_inputs, const Attrs& att
const String operator_name = "ethosu_pooling";
- if (param->pooling_type != "AVG" && param->pooling_type != "MAX") {
+ if (param->pooling_type != "AVG" && param->pooling_type != "MAX" &&
+ param->pooling_type != "SUM") {
reporter->GetDiagCtx().EmitFatal(Diagnostic::Error(reporter->GetSpan())
<< "Invalid operator: expected " << operator_name
- << " type 'AVG' or 'MAX' but was " << param->pooling_type);
+ << " type 'AVG', 'MAX', or 'SUM' but was "
+ << param->pooling_type);
return false;
}
- CheckDataType(reporter, ifm->dtype, {DataType::UInt(8), DataType::Int(8)}, operator_name, "ifm",
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Alexey-Yazev commented on a diff in pull request #13997: [microNPU] Sum legalization support
Posted by "Alexey-Yazev (via GitHub)" <gi...@apache.org>.
Alexey-Yazev commented on code in PR #13997:
URL: https://github.com/apache/tvm/pull/13997#discussion_r1111576806
##########
python/tvm/relay/op/contrib/ethosu.py:
##########
@@ -1375,6 +1375,82 @@ def mean_pattern() -> tvm.relay.dataflow_pattern.DFPattern:
return pattern
+class SumParams:
+ """
+ This class will parse a call to ethosu.sum composite function
+ and extract the parameter information.
+ """
+
+ composite_name = "ethos-u.sum"
+
+ def __init__(self, func_body: Call):
+ from tvm.relay.backend.contrib.ethosu.util import RequantArgs
+
+ requantize = func_body
+ sum_op = requantize.args[0]
+ attrs = sum_op.attrs
+ cast = sum_op.args[0]
+
+ layout = "NHWC"
+ self.ifm = TensorParams(
+ cast.args[0],
+ layout,
+ requantize.args[RequantArgs.IFM_SCALE.value],
+ requantize.args[RequantArgs.IFM_ZERO_POINT.value],
+ )
+ self.ofm = TensorParams(
+ requantize,
+ layout,
+ requantize.args[RequantArgs.OFM_SCALE.value],
+ requantize.args[RequantArgs.OFM_ZERO_POINT.value],
+ )
+
+ ifm_shape = self.ifm.shape
+ self.height = ifm_shape[0] if len(ifm_shape) in (2, 3) else ifm_shape[1]
+ self.width = ifm_shape[1] if len(ifm_shape) in (2, 3) else ifm_shape[2]
+ self.keepdims = attrs.keepdims
+
+ self.axis = list(sorted(attrs.axis))
+ if attrs.exclude:
+ self.axis = [i for i in range(len(self.ifm.shape)) if i not in self.axis]
+
+ def is_valid(self) -> bool:
+ """
+ Checks whether Sum has compatible attributes with HW.
+ """
+
+ ifm_shape_len = len(self.ifm.shape)
+
+ if not check_valid_dtypes([self.ifm], [np.uint8, np.int8, np.int16, np.int32]):
+ return False
+ if not check_valid_dtypes([self.ofm], [np.int8]):
Review Comment:
It is not possible to support int16 using SHR operation as it supports int8, uint8, int32, but for example I can use ADD operation instead of SHR. Is there an operation that is better to use for type conversion or would it be optimal to use the ADD operation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org