You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/06/07 11:45:55 UTC
[GitHub] [tvm] jtuyls opened a new pull request #8205: [Relay][Convert Layout] Enable layout transformation for image.resize op
jtuyls opened a new pull request #8205:
URL: https://github.com/apache/tvm/pull/8205
This PR adds layout transformation support for the image.resize operation. This is useful for custom codegens as it allows creating larger partitions with the same layout in segmentation models (e.g. deeplabv3). In particular, this allows the Vitis AI codegen to offload a larger part of the model to the DPU accelerator if applicable.
@anijain2305 @yzhliu @trevor-m Could you please help with reviewing this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jtuyls commented on a change in pull request #8205: [Relay][Convert Layout] Enable layout transformation for image.resize op
Posted by GitBox <gi...@apache.org>.
jtuyls commented on a change in pull request #8205:
URL: https://github.com/apache/tvm/pull/8205#discussion_r646885687
##########
File path: python/tvm/relay/op/image/_image.py
##########
@@ -58,6 +58,38 @@ def compute_resize(attrs, inputs, out_type):
reg.register_injective_schedule("image.resize")
+@reg.register_convert_op_layout("image.resize")
+def convert_image_resize(attrs, inputs, tinfos, desired_layouts):
+ """Convert Layout pass registration for image resize op.
+
+ Parameters
+ ----------
+ attrs : tvm.ir.Attrs
+ Attributes of current resize op
+ inputs : list of tvm.relay.Expr
+ The args of the Relay expr to be legalized
+ tinfos : list of types
+ List of input and output types
+ desired_layouts : list of layout strings
+ List of layouts defining our desired
+ layout for the data input.
+
+ Returns
+ -------
+ result : tvm.relay.Expr
+ The transformed expr
+ """
+ # pylint: disable=import-outside-toplevel
+ from tvm import relay
Review comment:
No, I just based this on other layout conversion functions and they seem to always lazy import inside the function: https://github.com/apache/tvm/blob/2c67d7131bc820481aaaca921f9e76f5709051b3/python/tvm/relay/op/nn/_nn.py#L240 Not sure why actually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jtuyls commented on a change in pull request #8205: [Relay][Convert Layout] Enable layout transformation for image.resize op
Posted by GitBox <gi...@apache.org>.
jtuyls commented on a change in pull request #8205:
URL: https://github.com/apache/tvm/pull/8205#discussion_r646885687
##########
File path: python/tvm/relay/op/image/_image.py
##########
@@ -58,6 +58,38 @@ def compute_resize(attrs, inputs, out_type):
reg.register_injective_schedule("image.resize")
+@reg.register_convert_op_layout("image.resize")
+def convert_image_resize(attrs, inputs, tinfos, desired_layouts):
+ """Convert Layout pass registration for image resize op.
+
+ Parameters
+ ----------
+ attrs : tvm.ir.Attrs
+ Attributes of current resize op
+ inputs : list of tvm.relay.Expr
+ The args of the Relay expr to be legalized
+ tinfos : list of types
+ List of input and output types
+ desired_layouts : list of layout strings
+ List of layouts defining our desired
+ layout for the data input.
+
+ Returns
+ -------
+ result : tvm.relay.Expr
+ The transformed expr
+ """
+ # pylint: disable=import-outside-toplevel
+ from tvm import relay
Review comment:
No, I just based this on other layout conversion functions and they seem to always lazy load inside the function: https://github.com/apache/tvm/blob/2c67d7131bc820481aaaca921f9e76f5709051b3/python/tvm/relay/op/nn/_nn.py#L240 Not sure why actually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jtuyls commented on a change in pull request #8205: [Relay][Convert Layout] Enable layout transformation for image.resize op
Posted by GitBox <gi...@apache.org>.
jtuyls commented on a change in pull request #8205:
URL: https://github.com/apache/tvm/pull/8205#discussion_r646889995
##########
File path: python/tvm/relay/op/image/_image.py
##########
@@ -58,6 +58,38 @@ def compute_resize(attrs, inputs, out_type):
reg.register_injective_schedule("image.resize")
+@reg.register_convert_op_layout("image.resize")
+def convert_image_resize(attrs, inputs, tinfos, desired_layouts):
+ """Convert Layout pass registration for image resize op.
+
+ Parameters
+ ----------
+ attrs : tvm.ir.Attrs
+ Attributes of current resize op
+ inputs : list of tvm.relay.Expr
+ The args of the Relay expr to be legalized
+ tinfos : list of types
+ List of input and output types
+ desired_layouts : list of layout strings
+ List of layouts defining our desired
+ layout for the data input.
+
+ Returns
+ -------
+ result : tvm.relay.Expr
+ The transformed expr
+ """
+ # pylint: disable=import-outside-toplevel
+ from tvm import relay
+
+ new_attrs = dict(attrs)
+ assert len(desired_layouts) == 1, "Only one desired layout is expected"
+ (desired_layout,) = map(str, desired_layouts)
Review comment:
Changed it to `desired_layout = str(desired_layouts[0])`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jtuyls commented on a change in pull request #8205: [Relay][Convert Layout] Enable layout transformation for image.resize op
Posted by GitBox <gi...@apache.org>.
jtuyls commented on a change in pull request #8205:
URL: https://github.com/apache/tvm/pull/8205#discussion_r647172893
##########
File path: python/tvm/relay/op/image/_image.py
##########
@@ -58,6 +58,38 @@ def compute_resize(attrs, inputs, out_type):
reg.register_injective_schedule("image.resize")
+@reg.register_convert_op_layout("image.resize")
+def convert_image_resize(attrs, inputs, tinfos, desired_layouts):
+ """Convert Layout pass registration for image resize op.
+
+ Parameters
+ ----------
+ attrs : tvm.ir.Attrs
+ Attributes of current resize op
+ inputs : list of tvm.relay.Expr
+ The args of the Relay expr to be legalized
+ tinfos : list of types
+ List of input and output types
+ desired_layouts : list of layout strings
+ List of layouts defining our desired
+ layout for the data input.
+
+ Returns
+ -------
+ result : tvm.relay.Expr
+ The transformed expr
+ """
+ # pylint: disable=import-outside-toplevel
+ from tvm import relay
Review comment:
@anijain2305 Yes, it works with the import at the top. We can also replace it with `from .image import resize` at the top.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] anijain2305 commented on pull request #8205: [Relay][Convert Layout] Enable layout transformation for image.resize op
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on pull request #8205:
URL: https://github.com/apache/tvm/pull/8205#issuecomment-856261589
Nice, this should help TensorRT as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on a change in pull request #8205: [Relay][Convert Layout] Enable layout transformation for image.resize op
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #8205:
URL: https://github.com/apache/tvm/pull/8205#discussion_r646891504
##########
File path: python/tvm/relay/op/image/_image.py
##########
@@ -58,6 +58,38 @@ def compute_resize(attrs, inputs, out_type):
reg.register_injective_schedule("image.resize")
+@reg.register_convert_op_layout("image.resize")
+def convert_image_resize(attrs, inputs, tinfos, desired_layouts):
+ """Convert Layout pass registration for image resize op.
+
+ Parameters
+ ----------
+ attrs : tvm.ir.Attrs
+ Attributes of current resize op
+ inputs : list of tvm.relay.Expr
+ The args of the Relay expr to be legalized
+ tinfos : list of types
+ List of input and output types
+ desired_layouts : list of layout strings
+ List of layouts defining our desired
+ layout for the data input.
+
+ Returns
+ -------
+ result : tvm.relay.Expr
+ The transformed expr
+ """
+ # pylint: disable=import-outside-toplevel
+ from tvm import relay
Review comment:
I see. I'll let @anijain2305 check if this is necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] anijain2305 commented on a change in pull request #8205: [Relay][Convert Layout] Enable layout transformation for image.resize op
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on a change in pull request #8205:
URL: https://github.com/apache/tvm/pull/8205#discussion_r646946445
##########
File path: python/tvm/relay/op/image/_image.py
##########
@@ -58,6 +58,38 @@ def compute_resize(attrs, inputs, out_type):
reg.register_injective_schedule("image.resize")
+@reg.register_convert_op_layout("image.resize")
+def convert_image_resize(attrs, inputs, tinfos, desired_layouts):
+ """Convert Layout pass registration for image resize op.
+
+ Parameters
+ ----------
+ attrs : tvm.ir.Attrs
+ Attributes of current resize op
+ inputs : list of tvm.relay.Expr
+ The args of the Relay expr to be legalized
+ tinfos : list of types
+ List of input and output types
+ desired_layouts : list of layout strings
+ List of layouts defining our desired
+ layout for the data input.
+
+ Returns
+ -------
+ result : tvm.relay.Expr
+ The transformed expr
+ """
+ # pylint: disable=import-outside-toplevel
+ from tvm import relay
Review comment:
I see. I might have copied it from some other place. @jtuyls Can you please check if you can move the import to the top (or just import the right module like `image` if the relay import fails).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on a change in pull request #8205: [Relay][Convert Layout] Enable layout transformation for image.resize op
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #8205:
URL: https://github.com/apache/tvm/pull/8205#discussion_r646749604
##########
File path: python/tvm/relay/op/image/_image.py
##########
@@ -58,6 +58,38 @@ def compute_resize(attrs, inputs, out_type):
reg.register_injective_schedule("image.resize")
+@reg.register_convert_op_layout("image.resize")
+def convert_image_resize(attrs, inputs, tinfos, desired_layouts):
+ """Convert Layout pass registration for image resize op.
+
+ Parameters
+ ----------
+ attrs : tvm.ir.Attrs
+ Attributes of current resize op
+ inputs : list of tvm.relay.Expr
+ The args of the Relay expr to be legalized
+ tinfos : list of types
+ List of input and output types
+ desired_layouts : list of layout strings
+ List of layouts defining our desired
+ layout for the data input.
+
+ Returns
+ -------
+ result : tvm.relay.Expr
+ The transformed expr
+ """
+ # pylint: disable=import-outside-toplevel
+ from tvm import relay
Review comment:
Is there any reason for lazy import?
##########
File path: python/tvm/relay/op/image/_image.py
##########
@@ -58,6 +58,38 @@ def compute_resize(attrs, inputs, out_type):
reg.register_injective_schedule("image.resize")
+@reg.register_convert_op_layout("image.resize")
+def convert_image_resize(attrs, inputs, tinfos, desired_layouts):
+ """Convert Layout pass registration for image resize op.
+
+ Parameters
+ ----------
+ attrs : tvm.ir.Attrs
+ Attributes of current resize op
+ inputs : list of tvm.relay.Expr
+ The args of the Relay expr to be legalized
+ tinfos : list of types
+ List of input and output types
+ desired_layouts : list of layout strings
+ List of layouts defining our desired
+ layout for the data input.
+
+ Returns
+ -------
+ result : tvm.relay.Expr
+ The transformed expr
+ """
+ # pylint: disable=import-outside-toplevel
+ from tvm import relay
+
+ new_attrs = dict(attrs)
+ assert len(desired_layouts) == 1, "Only one desired layout is expected"
+ (desired_layout,) = map(str, desired_layouts)
Review comment:
It's better to use list comprehension instead of `map`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #8205: [Relay][Convert Layout] Enable layout transformation for image.resize op
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #8205:
URL: https://github.com/apache/tvm/pull/8205
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org