You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/04/20 23:34:33 UTC
[GitHub] [tvm] AndrewZhaoLuo opened a new pull request #7893: Update Dev. Doc. on how to add a new relay operator
AndrewZhaoLuo opened a new pull request #7893:
URL: https://github.com/apache/tvm/pull/7893
Updates the tutorial and goes more in depth by using a more substantial operator than simple element-wise addition.
Instead we follow how PR #7334 and PR #7722 added cumulative product and addition operators.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #7893: Update Dev. Doc. on how to add a new relay operator
Posted by GitBox <gi...@apache.org>.
masahi commented on pull request #7893:
URL: https://github.com/apache/tvm/pull/7893#issuecomment-823977647
@hogepodge
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo commented on pull request #7893: Update Dev. Doc. on how to add a new relay operator
Posted by GitBox <gi...@apache.org>.
AndrewZhaoLuo commented on pull request #7893:
URL: https://github.com/apache/tvm/pull/7893#issuecomment-823668861
@masahi PTAL. Also Chris Hoge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #7893: Update Dev. Doc. on how to add a new relay operator
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #7893:
URL: https://github.com/apache/tvm/pull/7893#discussion_r617435051
##########
File path: docs/dev/relay_add_op.rst
##########
@@ -15,75 +15,297 @@
specific language governing permissions and limitations
under the License.
-.. _relay-add-op:
+.. _relay-add-op:
Adding an Operator to Relay
===========================
-In order to use TVM operators from within the Relay IR, the
-operators need to be registered in Relay in order to ensure
-that they will be integrated into Relay's type system.
+In this document we will go over the steps needed to register a new TVM operator
+in Relay. We will be following this PR which adds a `cumulative product`_ operation as an example.
+The PR itself builds upon another PR which adds a `cumulative sum`_ operation.
-Registering an operator requires three steps:
+.. _cumulative product: https://github.com/apache/tvm/pull/7722
+.. _cumulative sum: https://github.com/apache/tvm/pull/7334
-- Using the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity and type information
-- Defining a C++ function to produce a call node for the operator and registering a Python API hook for the function
-- Wrapping the above Python API hook in a neater interface
+Registering a new operator requires a few steps:
-The file ``src/relay/op/tensor/binary.cc`` provides
-examples of the first two steps, while
-``python/tvm/relay/op/tensor.py`` gives examples of the
-last.
+1. Add an attribute node declaring fixed arguments which are known at compile time
+2. Write a type relation for your operation to integrate into Relay's type system.
+3. Use the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity, type, and other hints for the compiler
+4. Write how the operator is computed
+5. Register the compute, schedule with the relay operator
+6. Define a C++ function to produce a call node for the operator and registering a Python API hook for the function
+7. Wrapping the above Python API hook in a neater interface
+8. Writing tests for the new relay operator
-Registering an Operator
------------------------
+1. Defining an Attribute Node
+-----------------------------
+Attributes are fixed arguments which are supposed to be known at compile time. The stride and dilation of a convolution
+operator would be an appropriate example of fields which might belong in an attribute node for a convolution operator.
-TVM already has an operator registry, but Relay cannot properly
-incorporate TVM operators without additional type information.
+Attributes should be defined in a file within the folder `include/tvm/relay/attrs/`_.
+.. _include/tvm/relay/attrs/: https://github.com/apache/tvm/tree/main/include/tvm/relay/attrs
+
+Ultimately we want to create an operator whose interface can be seen clearly in the final python interface:
+
+.. code:: python
+
+ def cumprod(data, axis=None, dtype=None, exclusive=None):
+ """Numpy style cumprod op. Return the cumulative inclusive product of the elements along
+ a given axis.
+ Parameters
+ ----------
+ data : relay.Expr
+ The input data to the operator.
+ axis : int, optional
+ Axis along which the cumulative product is computed. The default (None) is to compute
+ the cumprod over the flattened array.
+ dtype : string, optional
+ Type of the returned array and of the accumulator in which the elements are multiplied.
+ If dtype is not specified, it defaults to the dtype of data.
+ exclusive : bool, optional
+ If true will return exclusive product in which the first element is not
+ included. In other terms, if true, the j-th output element would be
+ the product of the first (j-1) elements. Otherwise, it would be the product of
+ the first j elements. The product of zero elements will be 1.
+ Returns
+ -------
+ result : relay.Expr
+ The result has the same size as data, and the same shape as data if axis is not None.
+ If axis is None, the result is a 1-d array.
+ """
+
+A similiar interface exists for ``cumsum()``.
+
+Therefore, when defining our attributes in ``include/tvm/relay/attrs/transform.h`` we choose the axis,
+accumulation dtype, and exclusivity of the operation as appropriate fields for the struct.
+
+.. code:: c++
+
+ /*! \brief Attributes used in cumsum and cumprod operator */
+ struct ScanopAttrs : public tvm::AttrsNode<ScanopAttrs> {
+ Integer axis;
+ DataType dtype;
+ Bool exclusive = Bool(false);
+ TVM_DECLARE_ATTRS(ScanopAttrs, "relay.attrs.ScanopAttrs") {
+ TVM_ATTR_FIELD(axis).describe("The axis to operate over").set_default(NullValue<Integer>());
+ TVM_ATTR_FIELD(dtype).describe("Output data type").set_default(NullValue<DataType>());
+ TVM_ATTR_FIELD(exclusive)
+ .describe("The first element is not included")
+ .set_default(Bool(false));
+ }
+ };
+
+2. Writing a Type Relation
+--------------------------
To allow for flexibility in registering operators and greater
expressivity and granularity in expressing types in Relay, operators
are typed using relations between input and output types. These relations
are represented as functions that take in a list of input types and
output types (any of these types may be incomplete) and return a list
-of input and output types that satisfies the relation. Essentially, a
+of input and output types that satisfies the relation. This includes shape
+information which can be determined statically at compile time. Essentially, a
relation for an operator can enforce all the necessary typing rules
(namely by inspecting the input types) in addition to computing the
output type.
-For example, see ``src/relay/op/type_relations.h`` and their
-implementations. E.g., ``BroadcastRel`` takes two input types and an
-output type, checks that they are all tensor types with the same underlying
-data type, and finally ensures that the shape of the output type is the
-broadcast of the input types' shapes.
+Type relation for the cumulative product and sum operators can be found in
+``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ TVM_REGISTER_NODE_TYPE(ScanopAttrs);
+ bool ScanopRel(const Array<Type>& types, int num_inputs, const Attrs& attrs, const TypeReporter& reporter) {
+ // types: [data, output]
+ ICHECK_EQ(types.size(), 2) << "Expects two types, one for the input and another for the output";
+ const auto* data = types[0].as<TensorTypeNode>();
+ if (data == nullptr) {
+ ICHECK(types[0].as<IncompleteTypeNode>())
+ << "Scanop: expect input type to be TensorType but get " << types[0];
+ return false;
+ }
+
+ const auto* param = attrs.as<ScanopAttrs>();
+
+ auto dtype = param->dtype;
+ if (dtype.is_void()) {
+ dtype = data->dtype;
+ }
+
+ if (param->axis.defined()) {
+ reporter->Assign(types[1], TensorType(data->shape, dtype));
+ } else {
+ auto prod = data->shape[0];
+ for (size_t i = 1; i < data->shape.size(); ++i) {
+ prod = prod * data->shape[i];
+ }
+ reporter->Assign(types[1], TensorType({prod}, dtype));
+ }
+
+ return true;
+ }
-It may be necessary to add another type relation to ``type_relations.h``
-if the existing ones do not capture the behavior of the desired operator.
+3. Relating the Arity and Attributes to an Operation
+----------------------------------------------------
+We then register the name of our new ops and annotate them with the calling interface.
The ``RELAY_REGISTER_OP`` macro in C++ allows a developer
to specify the following information about an operator in Relay:
- Arity (number of arguments)
- Names and descriptions for positional arguments
- Support level (1 indicates an internal intrinsic; higher numbers indicate less integral or externally supported operators)
- A type relation for the operator
+- Other annotations useful when optimizing the operation.
+
+Once again we add this to ``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ RELAY_REGISTER_OP("cumsum")
+ .describe(
+ R"doc(Return the cumulative sum of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumsum", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+ RELAY_REGISTER_OP("cumprod")
+ .describe(
+ R"doc(Return the cumulative product of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumprod", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+In this case the ``TOpPattern`` is a hint to the compiler on the pattern of computation the operator does, which might be
+useful for reordering loops and fusing operators. ``kOpaque`` tells TVM not to not bother trying to fuse this operator.
+
+4. Defining the Compute of the Operation
+----------------------------------------
+
+While we've now defined the interface for the operation but still have not
+told TVM how to perform the actual calculations for cumulative sum and product.
Review comment:
Need to polish this sentence
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #7893: [Docs] Update Dev. Doc. on how to add a new relay operator
Posted by GitBox <gi...@apache.org>.
masahi commented on pull request #7893:
URL: https://github.com/apache/tvm/pull/7893#issuecomment-824729516
Thanks @AndrewZhaoLuo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #7893: Update Dev. Doc. on how to add a new relay operator
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #7893:
URL: https://github.com/apache/tvm/pull/7893#discussion_r617433178
##########
File path: docs/dev/relay_add_op.rst
##########
@@ -15,75 +15,297 @@
specific language governing permissions and limitations
under the License.
-.. _relay-add-op:
+.. _relay-add-op:
Adding an Operator to Relay
===========================
-In order to use TVM operators from within the Relay IR, the
-operators need to be registered in Relay in order to ensure
-that they will be integrated into Relay's type system.
+In this document we will go over the steps needed to register a new TVM operator
+in Relay. We will be following this PR which adds a `cumulative product`_ operation as an example.
+The PR itself builds upon another PR which adds a `cumulative sum`_ operation.
-Registering an operator requires three steps:
+.. _cumulative product: https://github.com/apache/tvm/pull/7722
+.. _cumulative sum: https://github.com/apache/tvm/pull/7334
-- Using the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity and type information
-- Defining a C++ function to produce a call node for the operator and registering a Python API hook for the function
-- Wrapping the above Python API hook in a neater interface
+Registering a new operator requires a few steps:
-The file ``src/relay/op/tensor/binary.cc`` provides
-examples of the first two steps, while
-``python/tvm/relay/op/tensor.py`` gives examples of the
-last.
+1. Add an attribute node declaring fixed arguments which are known at compile time
+2. Write a type relation for your operation to integrate into Relay's type system.
+3. Use the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity, type, and other hints for the compiler
+4. Write how the operator is computed
+5. Register the compute, schedule with the relay operator
+6. Define a C++ function to produce a call node for the operator and registering a Python API hook for the function
+7. Wrapping the above Python API hook in a neater interface
+8. Writing tests for the new relay operator
-Registering an Operator
------------------------
+1. Defining an Attribute Node
+-----------------------------
+Attributes are fixed arguments which are supposed to be known at compile time. The stride and dilation of a convolution
+operator would be an appropriate example of fields which might belong in an attribute node for a convolution operator.
-TVM already has an operator registry, but Relay cannot properly
-incorporate TVM operators without additional type information.
+Attributes should be defined in a file within the folder `include/tvm/relay/attrs/`_.
+.. _include/tvm/relay/attrs/: https://github.com/apache/tvm/tree/main/include/tvm/relay/attrs
+
+Ultimately we want to create an operator whose interface can be seen clearly in the final python interface:
+
+.. code:: python
+
+ def cumprod(data, axis=None, dtype=None, exclusive=None):
+ """Numpy style cumprod op. Return the cumulative inclusive product of the elements along
+ a given axis.
+ Parameters
+ ----------
+ data : relay.Expr
+ The input data to the operator.
+ axis : int, optional
+ Axis along which the cumulative product is computed. The default (None) is to compute
+ the cumprod over the flattened array.
+ dtype : string, optional
+ Type of the returned array and of the accumulator in which the elements are multiplied.
+ If dtype is not specified, it defaults to the dtype of data.
+ exclusive : bool, optional
+ If true will return exclusive product in which the first element is not
+ included. In other terms, if true, the j-th output element would be
+ the product of the first (j-1) elements. Otherwise, it would be the product of
+ the first j elements. The product of zero elements will be 1.
+ Returns
+ -------
+ result : relay.Expr
+ The result has the same size as data, and the same shape as data if axis is not None.
+ If axis is None, the result is a 1-d array.
+ """
+
+A similiar interface exists for ``cumsum()``.
+
+Therefore, when defining our attributes in ``include/tvm/relay/attrs/transform.h`` we choose the axis,
+accumulation dtype, and exclusivity of the operation as appropriate fields for the struct.
+
+.. code:: c++
+
+ /*! \brief Attributes used in cumsum and cumprod operator */
+ struct ScanopAttrs : public tvm::AttrsNode<ScanopAttrs> {
+ Integer axis;
+ DataType dtype;
+ Bool exclusive = Bool(false);
+ TVM_DECLARE_ATTRS(ScanopAttrs, "relay.attrs.ScanopAttrs") {
+ TVM_ATTR_FIELD(axis).describe("The axis to operate over").set_default(NullValue<Integer>());
+ TVM_ATTR_FIELD(dtype).describe("Output data type").set_default(NullValue<DataType>());
+ TVM_ATTR_FIELD(exclusive)
+ .describe("The first element is not included")
+ .set_default(Bool(false));
+ }
+ };
+
+2. Writing a Type Relation
+--------------------------
To allow for flexibility in registering operators and greater
expressivity and granularity in expressing types in Relay, operators
are typed using relations between input and output types. These relations
are represented as functions that take in a list of input types and
output types (any of these types may be incomplete) and return a list
-of input and output types that satisfies the relation. Essentially, a
+of input and output types that satisfies the relation. This includes shape
+information which can be determined statically at compile time. Essentially, a
relation for an operator can enforce all the necessary typing rules
(namely by inspecting the input types) in addition to computing the
output type.
-For example, see ``src/relay/op/type_relations.h`` and their
-implementations. E.g., ``BroadcastRel`` takes two input types and an
-output type, checks that they are all tensor types with the same underlying
-data type, and finally ensures that the shape of the output type is the
-broadcast of the input types' shapes.
+Type relation for the cumulative product and sum operators can be found in
+``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ TVM_REGISTER_NODE_TYPE(ScanopAttrs);
+ bool ScanopRel(const Array<Type>& types, int num_inputs, const Attrs& attrs, const TypeReporter& reporter) {
+ // types: [data, output]
+ ICHECK_EQ(types.size(), 2) << "Expects two types, one for the input and another for the output";
+ const auto* data = types[0].as<TensorTypeNode>();
+ if (data == nullptr) {
+ ICHECK(types[0].as<IncompleteTypeNode>())
+ << "Scanop: expect input type to be TensorType but get " << types[0];
+ return false;
+ }
+
+ const auto* param = attrs.as<ScanopAttrs>();
+
+ auto dtype = param->dtype;
+ if (dtype.is_void()) {
+ dtype = data->dtype;
+ }
+
+ if (param->axis.defined()) {
+ reporter->Assign(types[1], TensorType(data->shape, dtype));
+ } else {
+ auto prod = data->shape[0];
+ for (size_t i = 1; i < data->shape.size(); ++i) {
+ prod = prod * data->shape[i];
+ }
+ reporter->Assign(types[1], TensorType({prod}, dtype));
+ }
+
+ return true;
+ }
-It may be necessary to add another type relation to ``type_relations.h``
-if the existing ones do not capture the behavior of the desired operator.
+3. Relating the Arity and Attributes to an Operation
+----------------------------------------------------
+We then register the name of our new ops and annotate them with the calling interface.
The ``RELAY_REGISTER_OP`` macro in C++ allows a developer
to specify the following information about an operator in Relay:
- Arity (number of arguments)
- Names and descriptions for positional arguments
- Support level (1 indicates an internal intrinsic; higher numbers indicate less integral or externally supported operators)
- A type relation for the operator
+- Other annotations useful when optimizing the operation.
+
+Once again we add this to ``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ RELAY_REGISTER_OP("cumsum")
+ .describe(
+ R"doc(Return the cumulative sum of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumsum", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+ RELAY_REGISTER_OP("cumprod")
+ .describe(
+ R"doc(Return the cumulative product of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumprod", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+In this case the ``TOpPattern`` is a hint to the compiler on the pattern of computation the operator does, which might be
+useful for reordering loops and fusing operators. ``kOpaque`` tells TVM not to not bother trying to fuse this operator.
Review comment:
Let's drop `reordering loops`, `TOpPattern` is only used for operater fusion.
A typo: `not to not bother`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #7893: Update Dev. Doc. on how to add a new relay operator
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #7893:
URL: https://github.com/apache/tvm/pull/7893#discussion_r617435627
##########
File path: docs/dev/relay_add_op.rst
##########
@@ -15,75 +15,297 @@
specific language governing permissions and limitations
under the License.
-.. _relay-add-op:
+.. _relay-add-op:
Adding an Operator to Relay
===========================
-In order to use TVM operators from within the Relay IR, the
-operators need to be registered in Relay in order to ensure
-that they will be integrated into Relay's type system.
+In this document we will go over the steps needed to register a new TVM operator
+in Relay. We will be following this PR which adds a `cumulative product`_ operation as an example.
+The PR itself builds upon another PR which adds a `cumulative sum`_ operation.
-Registering an operator requires three steps:
+.. _cumulative product: https://github.com/apache/tvm/pull/7722
+.. _cumulative sum: https://github.com/apache/tvm/pull/7334
-- Using the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity and type information
-- Defining a C++ function to produce a call node for the operator and registering a Python API hook for the function
-- Wrapping the above Python API hook in a neater interface
+Registering a new operator requires a few steps:
-The file ``src/relay/op/tensor/binary.cc`` provides
-examples of the first two steps, while
-``python/tvm/relay/op/tensor.py`` gives examples of the
-last.
+1. Add an attribute node declaring fixed arguments which are known at compile time
+2. Write a type relation for your operation to integrate into Relay's type system.
+3. Use the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity, type, and other hints for the compiler
+4. Write how the operator is computed
+5. Register the compute, schedule with the relay operator
+6. Define a C++ function to produce a call node for the operator and registering a Python API hook for the function
+7. Wrapping the above Python API hook in a neater interface
+8. Writing tests for the new relay operator
-Registering an Operator
------------------------
+1. Defining an Attribute Node
+-----------------------------
+Attributes are fixed arguments which are supposed to be known at compile time. The stride and dilation of a convolution
+operator would be an appropriate example of fields which might belong in an attribute node for a convolution operator.
-TVM already has an operator registry, but Relay cannot properly
-incorporate TVM operators without additional type information.
+Attributes should be defined in a file within the folder `include/tvm/relay/attrs/`_.
+.. _include/tvm/relay/attrs/: https://github.com/apache/tvm/tree/main/include/tvm/relay/attrs
+
+Ultimately we want to create an operator whose interface can be seen clearly in the final python interface:
+
+.. code:: python
+
+ def cumprod(data, axis=None, dtype=None, exclusive=None):
+ """Numpy style cumprod op. Return the cumulative inclusive product of the elements along
+ a given axis.
+ Parameters
+ ----------
+ data : relay.Expr
+ The input data to the operator.
+ axis : int, optional
+ Axis along which the cumulative product is computed. The default (None) is to compute
+ the cumprod over the flattened array.
+ dtype : string, optional
+ Type of the returned array and of the accumulator in which the elements are multiplied.
+ If dtype is not specified, it defaults to the dtype of data.
+ exclusive : bool, optional
+ If true will return exclusive product in which the first element is not
+ included. In other terms, if true, the j-th output element would be
+ the product of the first (j-1) elements. Otherwise, it would be the product of
+ the first j elements. The product of zero elements will be 1.
+ Returns
+ -------
+ result : relay.Expr
+ The result has the same size as data, and the same shape as data if axis is not None.
+ If axis is None, the result is a 1-d array.
+ """
+
+A similiar interface exists for ``cumsum()``.
+
+Therefore, when defining our attributes in ``include/tvm/relay/attrs/transform.h`` we choose the axis,
+accumulation dtype, and exclusivity of the operation as appropriate fields for the struct.
+
+.. code:: c++
+
+ /*! \brief Attributes used in cumsum and cumprod operator */
+ struct ScanopAttrs : public tvm::AttrsNode<ScanopAttrs> {
+ Integer axis;
+ DataType dtype;
+ Bool exclusive = Bool(false);
+ TVM_DECLARE_ATTRS(ScanopAttrs, "relay.attrs.ScanopAttrs") {
+ TVM_ATTR_FIELD(axis).describe("The axis to operate over").set_default(NullValue<Integer>());
+ TVM_ATTR_FIELD(dtype).describe("Output data type").set_default(NullValue<DataType>());
+ TVM_ATTR_FIELD(exclusive)
+ .describe("The first element is not included")
+ .set_default(Bool(false));
+ }
+ };
+
+2. Writing a Type Relation
+--------------------------
To allow for flexibility in registering operators and greater
expressivity and granularity in expressing types in Relay, operators
are typed using relations between input and output types. These relations
are represented as functions that take in a list of input types and
output types (any of these types may be incomplete) and return a list
-of input and output types that satisfies the relation. Essentially, a
+of input and output types that satisfies the relation. This includes shape
+information which can be determined statically at compile time. Essentially, a
relation for an operator can enforce all the necessary typing rules
(namely by inspecting the input types) in addition to computing the
output type.
-For example, see ``src/relay/op/type_relations.h`` and their
-implementations. E.g., ``BroadcastRel`` takes two input types and an
-output type, checks that they are all tensor types with the same underlying
-data type, and finally ensures that the shape of the output type is the
-broadcast of the input types' shapes.
+Type relation for the cumulative product and sum operators can be found in
+``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ TVM_REGISTER_NODE_TYPE(ScanopAttrs);
+ bool ScanopRel(const Array<Type>& types, int num_inputs, const Attrs& attrs, const TypeReporter& reporter) {
+ // types: [data, output]
+ ICHECK_EQ(types.size(), 2) << "Expects two types, one for the input and another for the output";
+ const auto* data = types[0].as<TensorTypeNode>();
+ if (data == nullptr) {
+ ICHECK(types[0].as<IncompleteTypeNode>())
+ << "Scanop: expect input type to be TensorType but get " << types[0];
+ return false;
+ }
+
+ const auto* param = attrs.as<ScanopAttrs>();
+
+ auto dtype = param->dtype;
+ if (dtype.is_void()) {
+ dtype = data->dtype;
+ }
+
+ if (param->axis.defined()) {
+ reporter->Assign(types[1], TensorType(data->shape, dtype));
+ } else {
+ auto prod = data->shape[0];
+ for (size_t i = 1; i < data->shape.size(); ++i) {
+ prod = prod * data->shape[i];
+ }
+ reporter->Assign(types[1], TensorType({prod}, dtype));
+ }
+
+ return true;
+ }
-It may be necessary to add another type relation to ``type_relations.h``
-if the existing ones do not capture the behavior of the desired operator.
+3. Relating the Arity and Attributes to an Operation
+----------------------------------------------------
+We then register the name of our new ops and annotate them with the calling interface.
The ``RELAY_REGISTER_OP`` macro in C++ allows a developer
to specify the following information about an operator in Relay:
- Arity (number of arguments)
- Names and descriptions for positional arguments
- Support level (1 indicates an internal intrinsic; higher numbers indicate less integral or externally supported operators)
- A type relation for the operator
+- Other annotations useful when optimizing the operation.
+
+Once again we add this to ``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ RELAY_REGISTER_OP("cumsum")
+ .describe(
+ R"doc(Return the cumulative sum of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumsum", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+ RELAY_REGISTER_OP("cumprod")
+ .describe(
+ R"doc(Return the cumulative product of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumprod", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+In this case the ``TOpPattern`` is a hint to the compiler on the pattern of computation the operator does, which might be
+useful for reordering loops and fusing operators. ``kOpaque`` tells TVM not to not bother trying to fuse this operator.
+
+4. Defining the Compute of the Operation
+----------------------------------------
+
+While we've now defined the interface for the operation but still have not
+told TVM how to perform the actual calculations for cumulative sum and product.
+
+Writing this code is outside the scope of the tutorial. For now, we assume
+we have a well tested implementation for the operation's compute. For
+more details on how to do this, we recommend looking up the tutorials
+on `tensor expressions`_, `TVM's operator inventory (topi)`_ and looking at the
+examples cumulative sum and product found in `python/tvm/topi/scan.py`_ and
+`python/tvm/topi/cuda/scan.py`_. In the case of our cumulative sum and product operations
+we write things directly in `TIR`_ which is the representation where tensor expressions
+and topi will lower into.
+
+.. _tensor expressions: https://tvm.apache.org/docs/tutorials/get_started/tensor_expr_get_started.html
+.. _TVM's operator inventory (topi): https://tvm.apache.org/docs/tutorials/topi/intro_topi.html
+.. _TIR: https://tvm.apache.org/docs/dev/index.html?highlight=tir#tvm-tir
+.. _python/tvm/topi/scan.py: https://github.com/apache/tvm/blob/main/python/tvm/topi/scan.py
+.. _python/tvm/topi/cuda/scan.py: https://github.com/apache/tvm/blob/main/python/tvm/topi/cuda/scan.py
+
+5. Hooking up Compute and Strategy with Relay
+---------------------------------------------
+
+After you have implemented how your function can be computed we now need to glue it to our
Review comment:
After you have implemented your compute function
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #7893: [Docs] Update Dev. Doc. on how to add a new relay operator
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #7893:
URL: https://github.com/apache/tvm/pull/7893
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo commented on pull request #7893: Update Dev. Doc. on how to add a new relay operator
Posted by GitBox <gi...@apache.org>.
AndrewZhaoLuo commented on pull request #7893:
URL: https://github.com/apache/tvm/pull/7893#issuecomment-824258396
@masahi Thanks for the comments. PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #7893: Update Dev. Doc. on how to add a new relay operator
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #7893:
URL: https://github.com/apache/tvm/pull/7893#discussion_r617434021
##########
File path: docs/dev/relay_add_op.rst
##########
@@ -15,75 +15,297 @@
specific language governing permissions and limitations
under the License.
-.. _relay-add-op:
+.. _relay-add-op:
Adding an Operator to Relay
===========================
-In order to use TVM operators from within the Relay IR, the
-operators need to be registered in Relay in order to ensure
-that they will be integrated into Relay's type system.
+In this document we will go over the steps needed to register a new TVM operator
+in Relay. We will be following this PR which adds a `cumulative product`_ operation as an example.
+The PR itself builds upon another PR which adds a `cumulative sum`_ operation.
-Registering an operator requires three steps:
+.. _cumulative product: https://github.com/apache/tvm/pull/7722
+.. _cumulative sum: https://github.com/apache/tvm/pull/7334
-- Using the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity and type information
-- Defining a C++ function to produce a call node for the operator and registering a Python API hook for the function
-- Wrapping the above Python API hook in a neater interface
+Registering a new operator requires a few steps:
-The file ``src/relay/op/tensor/binary.cc`` provides
-examples of the first two steps, while
-``python/tvm/relay/op/tensor.py`` gives examples of the
-last.
+1. Add an attribute node declaring fixed arguments which are known at compile time
+2. Write a type relation for your operation to integrate into Relay's type system.
+3. Use the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity, type, and other hints for the compiler
+4. Write how the operator is computed
+5. Register the compute, schedule with the relay operator
+6. Define a C++ function to produce a call node for the operator and registering a Python API hook for the function
+7. Wrapping the above Python API hook in a neater interface
+8. Writing tests for the new relay operator
-Registering an Operator
------------------------
+1. Defining an Attribute Node
+-----------------------------
+Attributes are fixed arguments which are supposed to be known at compile time. The stride and dilation of a convolution
+operator would be an appropriate example of fields which might belong in an attribute node for a convolution operator.
-TVM already has an operator registry, but Relay cannot properly
-incorporate TVM operators without additional type information.
+Attributes should be defined in a file within the folder `include/tvm/relay/attrs/`_.
+.. _include/tvm/relay/attrs/: https://github.com/apache/tvm/tree/main/include/tvm/relay/attrs
+
+Ultimately we want to create an operator whose interface can be seen clearly in the final python interface:
+
+.. code:: python
+
+ def cumprod(data, axis=None, dtype=None, exclusive=None):
+ """Numpy style cumprod op. Return the cumulative inclusive product of the elements along
+ a given axis.
+ Parameters
+ ----------
+ data : relay.Expr
+ The input data to the operator.
+ axis : int, optional
+ Axis along which the cumulative product is computed. The default (None) is to compute
+ the cumprod over the flattened array.
+ dtype : string, optional
+ Type of the returned array and of the accumulator in which the elements are multiplied.
+ If dtype is not specified, it defaults to the dtype of data.
+ exclusive : bool, optional
+ If true will return exclusive product in which the first element is not
+ included. In other terms, if true, the j-th output element would be
+ the product of the first (j-1) elements. Otherwise, it would be the product of
+ the first j elements. The product of zero elements will be 1.
+ Returns
+ -------
+ result : relay.Expr
+ The result has the same size as data, and the same shape as data if axis is not None.
+ If axis is None, the result is a 1-d array.
+ """
+
+A similiar interface exists for ``cumsum()``.
+
+Therefore, when defining our attributes in ``include/tvm/relay/attrs/transform.h`` we choose the axis,
+accumulation dtype, and exclusivity of the operation as appropriate fields for the struct.
+
+.. code:: c++
+
+ /*! \brief Attributes used in cumsum and cumprod operator */
+ struct ScanopAttrs : public tvm::AttrsNode<ScanopAttrs> {
+ Integer axis;
+ DataType dtype;
+ Bool exclusive = Bool(false);
+ TVM_DECLARE_ATTRS(ScanopAttrs, "relay.attrs.ScanopAttrs") {
+ TVM_ATTR_FIELD(axis).describe("The axis to operate over").set_default(NullValue<Integer>());
+ TVM_ATTR_FIELD(dtype).describe("Output data type").set_default(NullValue<DataType>());
+ TVM_ATTR_FIELD(exclusive)
+ .describe("The first element is not included")
+ .set_default(Bool(false));
+ }
+ };
+
+2. Writing a Type Relation
+--------------------------
To allow for flexibility in registering operators and greater
expressivity and granularity in expressing types in Relay, operators
are typed using relations between input and output types. These relations
are represented as functions that take in a list of input types and
output types (any of these types may be incomplete) and return a list
-of input and output types that satisfies the relation. Essentially, a
+of input and output types that satisfies the relation. This includes shape
+information which can be determined statically at compile time. Essentially, a
relation for an operator can enforce all the necessary typing rules
(namely by inspecting the input types) in addition to computing the
output type.
-For example, see ``src/relay/op/type_relations.h`` and their
-implementations. E.g., ``BroadcastRel`` takes two input types and an
-output type, checks that they are all tensor types with the same underlying
-data type, and finally ensures that the shape of the output type is the
-broadcast of the input types' shapes.
+Type relation for the cumulative product and sum operators can be found in
+``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ TVM_REGISTER_NODE_TYPE(ScanopAttrs);
+ bool ScanopRel(const Array<Type>& types, int num_inputs, const Attrs& attrs, const TypeReporter& reporter) {
+ // types: [data, output]
+ ICHECK_EQ(types.size(), 2) << "Expects two types, one for the input and another for the output";
+ const auto* data = types[0].as<TensorTypeNode>();
+ if (data == nullptr) {
+ ICHECK(types[0].as<IncompleteTypeNode>())
+ << "Scanop: expect input type to be TensorType but get " << types[0];
+ return false;
+ }
+
+ const auto* param = attrs.as<ScanopAttrs>();
+
+ auto dtype = param->dtype;
+ if (dtype.is_void()) {
+ dtype = data->dtype;
+ }
+
+ if (param->axis.defined()) {
+ reporter->Assign(types[1], TensorType(data->shape, dtype));
+ } else {
+ auto prod = data->shape[0];
+ for (size_t i = 1; i < data->shape.size(); ++i) {
+ prod = prod * data->shape[i];
+ }
+ reporter->Assign(types[1], TensorType({prod}, dtype));
+ }
+
+ return true;
+ }
-It may be necessary to add another type relation to ``type_relations.h``
-if the existing ones do not capture the behavior of the desired operator.
+3. Relating the Arity and Attributes to an Operation
+----------------------------------------------------
+We then register the name of our new ops and annotate them with the calling interface.
The ``RELAY_REGISTER_OP`` macro in C++ allows a developer
to specify the following information about an operator in Relay:
- Arity (number of arguments)
- Names and descriptions for positional arguments
- Support level (1 indicates an internal intrinsic; higher numbers indicate less integral or externally supported operators)
- A type relation for the operator
+- Other annotations useful when optimizing the operation.
+
+Once again we add this to ``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ RELAY_REGISTER_OP("cumsum")
+ .describe(
+ R"doc(Return the cumulative sum of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumsum", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+ RELAY_REGISTER_OP("cumprod")
+ .describe(
+ R"doc(Return the cumulative product of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumprod", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+In this case the ``TOpPattern`` is a hint to the compiler on the pattern of computation the operator does, which might be
+useful for reordering loops and fusing operators. ``kOpaque`` tells TVM not to not bother trying to fuse this operator.
+
+4. Defining the Compute of the Operation
+----------------------------------------
+
+While we've now defined the interface for the operation but still have not
+told TVM how to perform the actual calculations for cumulative sum and product.
+
+Writing this code is outside the scope of the tutorial. For now, we assume
+we have a well tested implementation for the operation's compute. For
+more details on how to do this, we recommend looking up the tutorials
+on `tensor expressions`_, `TVM's operator inventory (topi)`_ and looking at the
+examples cumulative sum and product found in `python/tvm/topi/scan.py`_ and
+`python/tvm/topi/cuda/scan.py`_. In the case of our cumulative sum and product operations
Review comment:
Multiple references to the same file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo commented on a change in pull request #7893: Update Dev. Doc. on how to add a new relay operator
Posted by GitBox <gi...@apache.org>.
AndrewZhaoLuo commented on a change in pull request #7893:
URL: https://github.com/apache/tvm/pull/7893#discussion_r617772065
##########
File path: docs/dev/relay_add_op.rst
##########
@@ -15,75 +15,297 @@
specific language governing permissions and limitations
under the License.
-.. _relay-add-op:
+.. _relay-add-op:
Adding an Operator to Relay
===========================
-In order to use TVM operators from within the Relay IR, the
-operators need to be registered in Relay in order to ensure
-that they will be integrated into Relay's type system.
+In this document we will go over the steps needed to register a new TVM operator
+in Relay. We will be following this PR which adds a `cumulative product`_ operation as an example.
+The PR itself builds upon another PR which adds a `cumulative sum`_ operation.
-Registering an operator requires three steps:
+.. _cumulative product: https://github.com/apache/tvm/pull/7722
+.. _cumulative sum: https://github.com/apache/tvm/pull/7334
-- Using the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity and type information
-- Defining a C++ function to produce a call node for the operator and registering a Python API hook for the function
-- Wrapping the above Python API hook in a neater interface
+Registering a new operator requires a few steps:
-The file ``src/relay/op/tensor/binary.cc`` provides
-examples of the first two steps, while
-``python/tvm/relay/op/tensor.py`` gives examples of the
-last.
+1. Add an attribute node declaring fixed arguments which are known at compile time
+2. Write a type relation for your operation to integrate into Relay's type system.
+3. Use the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity, type, and other hints for the compiler
+4. Write how the operator is computed
+5. Register the compute, schedule with the relay operator
+6. Define a C++ function to produce a call node for the operator and registering a Python API hook for the function
+7. Wrapping the above Python API hook in a neater interface
+8. Writing tests for the new relay operator
-Registering an Operator
------------------------
+1. Defining an Attribute Node
+-----------------------------
+Attributes are fixed arguments which are supposed to be known at compile time. The stride and dilation of a convolution
+operator would be an appropriate example of fields which might belong in an attribute node for a convolution operator.
-TVM already has an operator registry, but Relay cannot properly
-incorporate TVM operators without additional type information.
+Attributes should be defined in a file within the folder `include/tvm/relay/attrs/`_.
+.. _include/tvm/relay/attrs/: https://github.com/apache/tvm/tree/main/include/tvm/relay/attrs
+
+Ultimately we want to create an operator whose interface can be seen clearly in the final python interface:
+
+.. code:: python
+
+ def cumprod(data, axis=None, dtype=None, exclusive=None):
+ """Numpy style cumprod op. Return the cumulative inclusive product of the elements along
+ a given axis.
+ Parameters
+ ----------
+ data : relay.Expr
+ The input data to the operator.
+ axis : int, optional
+ Axis along which the cumulative product is computed. The default (None) is to compute
+ the cumprod over the flattened array.
+ dtype : string, optional
+ Type of the returned array and of the accumulator in which the elements are multiplied.
+ If dtype is not specified, it defaults to the dtype of data.
+ exclusive : bool, optional
+ If true will return exclusive product in which the first element is not
+ included. In other terms, if true, the j-th output element would be
+ the product of the first (j-1) elements. Otherwise, it would be the product of
+ the first j elements. The product of zero elements will be 1.
+ Returns
+ -------
+ result : relay.Expr
+ The result has the same size as data, and the same shape as data if axis is not None.
+ If axis is None, the result is a 1-d array.
+ """
+
+A similiar interface exists for ``cumsum()``.
+
+Therefore, when defining our attributes in ``include/tvm/relay/attrs/transform.h`` we choose the axis,
+accumulation dtype, and exclusivity of the operation as appropriate fields for the struct.
+
+.. code:: c++
+
+ /*! \brief Attributes used in cumsum and cumprod operator */
+ struct ScanopAttrs : public tvm::AttrsNode<ScanopAttrs> {
+ Integer axis;
+ DataType dtype;
+ Bool exclusive = Bool(false);
+ TVM_DECLARE_ATTRS(ScanopAttrs, "relay.attrs.ScanopAttrs") {
+ TVM_ATTR_FIELD(axis).describe("The axis to operate over").set_default(NullValue<Integer>());
+ TVM_ATTR_FIELD(dtype).describe("Output data type").set_default(NullValue<DataType>());
+ TVM_ATTR_FIELD(exclusive)
+ .describe("The first element is not included")
+ .set_default(Bool(false));
+ }
+ };
+
+2. Writing a Type Relation
+--------------------------
To allow for flexibility in registering operators and greater
expressivity and granularity in expressing types in Relay, operators
are typed using relations between input and output types. These relations
are represented as functions that take in a list of input types and
output types (any of these types may be incomplete) and return a list
-of input and output types that satisfies the relation. Essentially, a
+of input and output types that satisfies the relation. This includes shape
+information which can be determined statically at compile time. Essentially, a
relation for an operator can enforce all the necessary typing rules
(namely by inspecting the input types) in addition to computing the
output type.
-For example, see ``src/relay/op/type_relations.h`` and their
-implementations. E.g., ``BroadcastRel`` takes two input types and an
-output type, checks that they are all tensor types with the same underlying
-data type, and finally ensures that the shape of the output type is the
-broadcast of the input types' shapes.
+Type relation for the cumulative product and sum operators can be found in
+``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ TVM_REGISTER_NODE_TYPE(ScanopAttrs);
+ bool ScanopRel(const Array<Type>& types, int num_inputs, const Attrs& attrs, const TypeReporter& reporter) {
+ // types: [data, output]
+ ICHECK_EQ(types.size(), 2) << "Expects two types, one for the input and another for the output";
+ const auto* data = types[0].as<TensorTypeNode>();
+ if (data == nullptr) {
+ ICHECK(types[0].as<IncompleteTypeNode>())
+ << "Scanop: expect input type to be TensorType but get " << types[0];
+ return false;
+ }
+
+ const auto* param = attrs.as<ScanopAttrs>();
+
+ auto dtype = param->dtype;
+ if (dtype.is_void()) {
+ dtype = data->dtype;
+ }
+
+ if (param->axis.defined()) {
+ reporter->Assign(types[1], TensorType(data->shape, dtype));
+ } else {
+ auto prod = data->shape[0];
+ for (size_t i = 1; i < data->shape.size(); ++i) {
+ prod = prod * data->shape[i];
+ }
+ reporter->Assign(types[1], TensorType({prod}, dtype));
+ }
+
+ return true;
+ }
-It may be necessary to add another type relation to ``type_relations.h``
-if the existing ones do not capture the behavior of the desired operator.
+3. Relating the Arity and Attributes to an Operation
+----------------------------------------------------
+We then register the name of our new ops and annotate them with the calling interface.
The ``RELAY_REGISTER_OP`` macro in C++ allows a developer
to specify the following information about an operator in Relay:
- Arity (number of arguments)
- Names and descriptions for positional arguments
- Support level (1 indicates an internal intrinsic; higher numbers indicate less integral or externally supported operators)
- A type relation for the operator
+- Other annotations useful when optimizing the operation.
+
+Once again we add this to ``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ RELAY_REGISTER_OP("cumsum")
+ .describe(
+ R"doc(Return the cumulative sum of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumsum", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+ RELAY_REGISTER_OP("cumprod")
+ .describe(
+ R"doc(Return the cumulative product of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumprod", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+In this case the ``TOpPattern`` is a hint to the compiler on the pattern of computation the operator does, which might be
+useful for reordering loops and fusing operators. ``kOpaque`` tells TVM not to not bother trying to fuse this operator.
Review comment:
Done
##########
File path: docs/dev/relay_add_op.rst
##########
@@ -15,75 +15,297 @@
specific language governing permissions and limitations
under the License.
-.. _relay-add-op:
+.. _relay-add-op:
Adding an Operator to Relay
===========================
-In order to use TVM operators from within the Relay IR, the
-operators need to be registered in Relay in order to ensure
-that they will be integrated into Relay's type system.
+In this document we will go over the steps needed to register a new TVM operator
+in Relay. We will be following this PR which adds a `cumulative product`_ operation as an example.
+The PR itself builds upon another PR which adds a `cumulative sum`_ operation.
-Registering an operator requires three steps:
+.. _cumulative product: https://github.com/apache/tvm/pull/7722
+.. _cumulative sum: https://github.com/apache/tvm/pull/7334
-- Using the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity and type information
-- Defining a C++ function to produce a call node for the operator and registering a Python API hook for the function
-- Wrapping the above Python API hook in a neater interface
+Registering a new operator requires a few steps:
-The file ``src/relay/op/tensor/binary.cc`` provides
-examples of the first two steps, while
-``python/tvm/relay/op/tensor.py`` gives examples of the
-last.
+1. Add an attribute node declaring fixed arguments which are known at compile time
+2. Write a type relation for your operation to integrate into Relay's type system.
+3. Use the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity, type, and other hints for the compiler
+4. Write how the operator is computed
+5. Register the compute, schedule with the relay operator
+6. Define a C++ function to produce a call node for the operator and registering a Python API hook for the function
+7. Wrapping the above Python API hook in a neater interface
+8. Writing tests for the new relay operator
-Registering an Operator
------------------------
+1. Defining an Attribute Node
+-----------------------------
+Attributes are fixed arguments which are supposed to be known at compile time. The stride and dilation of a convolution
+operator would be an appropriate example of fields which might belong in an attribute node for a convolution operator.
-TVM already has an operator registry, but Relay cannot properly
-incorporate TVM operators without additional type information.
+Attributes should be defined in a file within the folder `include/tvm/relay/attrs/`_.
+.. _include/tvm/relay/attrs/: https://github.com/apache/tvm/tree/main/include/tvm/relay/attrs
+
+Ultimately we want to create an operator whose interface can be seen clearly in the final python interface:
+
+.. code:: python
+
+ def cumprod(data, axis=None, dtype=None, exclusive=None):
+ """Numpy style cumprod op. Return the cumulative inclusive product of the elements along
+ a given axis.
+ Parameters
+ ----------
+ data : relay.Expr
+ The input data to the operator.
+ axis : int, optional
+ Axis along which the cumulative product is computed. The default (None) is to compute
+ the cumprod over the flattened array.
+ dtype : string, optional
+ Type of the returned array and of the accumulator in which the elements are multiplied.
+ If dtype is not specified, it defaults to the dtype of data.
+ exclusive : bool, optional
+ If true will return exclusive product in which the first element is not
+ included. In other terms, if true, the j-th output element would be
+ the product of the first (j-1) elements. Otherwise, it would be the product of
+ the first j elements. The product of zero elements will be 1.
+ Returns
+ -------
+ result : relay.Expr
+ The result has the same size as data, and the same shape as data if axis is not None.
+ If axis is None, the result is a 1-d array.
+ """
+
+A similiar interface exists for ``cumsum()``.
+
+Therefore, when defining our attributes in ``include/tvm/relay/attrs/transform.h`` we choose the axis,
+accumulation dtype, and exclusivity of the operation as appropriate fields for the struct.
+
+.. code:: c++
+
+ /*! \brief Attributes used in cumsum and cumprod operator */
+ struct ScanopAttrs : public tvm::AttrsNode<ScanopAttrs> {
+ Integer axis;
+ DataType dtype;
+ Bool exclusive = Bool(false);
+ TVM_DECLARE_ATTRS(ScanopAttrs, "relay.attrs.ScanopAttrs") {
+ TVM_ATTR_FIELD(axis).describe("The axis to operate over").set_default(NullValue<Integer>());
+ TVM_ATTR_FIELD(dtype).describe("Output data type").set_default(NullValue<DataType>());
+ TVM_ATTR_FIELD(exclusive)
+ .describe("The first element is not included")
+ .set_default(Bool(false));
+ }
+ };
+
+2. Writing a Type Relation
+--------------------------
To allow for flexibility in registering operators and greater
expressivity and granularity in expressing types in Relay, operators
are typed using relations between input and output types. These relations
are represented as functions that take in a list of input types and
output types (any of these types may be incomplete) and return a list
-of input and output types that satisfies the relation. Essentially, a
+of input and output types that satisfies the relation. This includes shape
+information which can be determined statically at compile time. Essentially, a
relation for an operator can enforce all the necessary typing rules
(namely by inspecting the input types) in addition to computing the
output type.
-For example, see ``src/relay/op/type_relations.h`` and their
-implementations. E.g., ``BroadcastRel`` takes two input types and an
-output type, checks that they are all tensor types with the same underlying
-data type, and finally ensures that the shape of the output type is the
-broadcast of the input types' shapes.
+Type relation for the cumulative product and sum operators can be found in
+``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ TVM_REGISTER_NODE_TYPE(ScanopAttrs);
+ bool ScanopRel(const Array<Type>& types, int num_inputs, const Attrs& attrs, const TypeReporter& reporter) {
+ // types: [data, output]
+ ICHECK_EQ(types.size(), 2) << "Expects two types, one for the input and another for the output";
+ const auto* data = types[0].as<TensorTypeNode>();
+ if (data == nullptr) {
+ ICHECK(types[0].as<IncompleteTypeNode>())
+ << "Scanop: expect input type to be TensorType but get " << types[0];
+ return false;
+ }
+
+ const auto* param = attrs.as<ScanopAttrs>();
+
+ auto dtype = param->dtype;
+ if (dtype.is_void()) {
+ dtype = data->dtype;
+ }
+
+ if (param->axis.defined()) {
+ reporter->Assign(types[1], TensorType(data->shape, dtype));
+ } else {
+ auto prod = data->shape[0];
+ for (size_t i = 1; i < data->shape.size(); ++i) {
+ prod = prod * data->shape[i];
+ }
+ reporter->Assign(types[1], TensorType({prod}, dtype));
+ }
+
+ return true;
+ }
-It may be necessary to add another type relation to ``type_relations.h``
-if the existing ones do not capture the behavior of the desired operator.
+3. Relating the Arity and Attributes to an Operation
+----------------------------------------------------
+We then register the name of our new ops and annotate them with the calling interface.
The ``RELAY_REGISTER_OP`` macro in C++ allows a developer
to specify the following information about an operator in Relay:
- Arity (number of arguments)
- Names and descriptions for positional arguments
- Support level (1 indicates an internal intrinsic; higher numbers indicate less integral or externally supported operators)
- A type relation for the operator
+- Other annotations useful when optimizing the operation.
+
+Once again we add this to ``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ RELAY_REGISTER_OP("cumsum")
+ .describe(
+ R"doc(Return the cumulative sum of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumsum", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+ RELAY_REGISTER_OP("cumprod")
+ .describe(
+ R"doc(Return the cumulative product of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumprod", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+In this case the ``TOpPattern`` is a hint to the compiler on the pattern of computation the operator does, which might be
+useful for reordering loops and fusing operators. ``kOpaque`` tells TVM not to not bother trying to fuse this operator.
+
+4. Defining the Compute of the Operation
+----------------------------------------
+
+While we've now defined the interface for the operation but still have not
+told TVM how to perform the actual calculations for cumulative sum and product.
+
+Writing this code is outside the scope of the tutorial. For now, we assume
+we have a well tested implementation for the operation's compute. For
+more details on how to do this, we recommend looking up the tutorials
+on `tensor expressions`_, `TVM's operator inventory (topi)`_ and looking at the
+examples cumulative sum and product found in `python/tvm/topi/scan.py`_ and
+`python/tvm/topi/cuda/scan.py`_. In the case of our cumulative sum and product operations
Review comment:
They actually are different files. One is the generic implementation and one is the cuda/gpu implementation. I've made this clearer in the text
##########
File path: docs/dev/relay_add_op.rst
##########
@@ -15,75 +15,297 @@
specific language governing permissions and limitations
under the License.
-.. _relay-add-op:
+.. _relay-add-op:
Adding an Operator to Relay
===========================
-In order to use TVM operators from within the Relay IR, the
-operators need to be registered in Relay in order to ensure
-that they will be integrated into Relay's type system.
+In this document we will go over the steps needed to register a new TVM operator
+in Relay. We will be following this PR which adds a `cumulative product`_ operation as an example.
+The PR itself builds upon another PR which adds a `cumulative sum`_ operation.
-Registering an operator requires three steps:
+.. _cumulative product: https://github.com/apache/tvm/pull/7722
+.. _cumulative sum: https://github.com/apache/tvm/pull/7334
-- Using the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity and type information
-- Defining a C++ function to produce a call node for the operator and registering a Python API hook for the function
-- Wrapping the above Python API hook in a neater interface
+Registering a new operator requires a few steps:
-The file ``src/relay/op/tensor/binary.cc`` provides
-examples of the first two steps, while
-``python/tvm/relay/op/tensor.py`` gives examples of the
-last.
+1. Add an attribute node declaring fixed arguments which are known at compile time
+2. Write a type relation for your operation to integrate into Relay's type system.
+3. Use the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity, type, and other hints for the compiler
+4. Write how the operator is computed
+5. Register the compute, schedule with the relay operator
+6. Define a C++ function to produce a call node for the operator and registering a Python API hook for the function
+7. Wrapping the above Python API hook in a neater interface
+8. Writing tests for the new relay operator
-Registering an Operator
------------------------
+1. Defining an Attribute Node
+-----------------------------
+Attributes are fixed arguments which are supposed to be known at compile time. The stride and dilation of a convolution
+operator would be an appropriate example of fields which might belong in an attribute node for a convolution operator.
-TVM already has an operator registry, but Relay cannot properly
-incorporate TVM operators without additional type information.
+Attributes should be defined in a file within the folder `include/tvm/relay/attrs/`_.
+.. _include/tvm/relay/attrs/: https://github.com/apache/tvm/tree/main/include/tvm/relay/attrs
+
+Ultimately we want to create an operator whose interface can be seen clearly in the final python interface:
+
+.. code:: python
+
+ def cumprod(data, axis=None, dtype=None, exclusive=None):
+ """Numpy style cumprod op. Return the cumulative inclusive product of the elements along
+ a given axis.
+ Parameters
+ ----------
+ data : relay.Expr
+ The input data to the operator.
+ axis : int, optional
+ Axis along which the cumulative product is computed. The default (None) is to compute
+ the cumprod over the flattened array.
+ dtype : string, optional
+ Type of the returned array and of the accumulator in which the elements are multiplied.
+ If dtype is not specified, it defaults to the dtype of data.
+ exclusive : bool, optional
+ If true will return exclusive product in which the first element is not
+ included. In other terms, if true, the j-th output element would be
+ the product of the first (j-1) elements. Otherwise, it would be the product of
+ the first j elements. The product of zero elements will be 1.
+ Returns
+ -------
+ result : relay.Expr
+ The result has the same size as data, and the same shape as data if axis is not None.
+ If axis is None, the result is a 1-d array.
+ """
+
+A similiar interface exists for ``cumsum()``.
+
+Therefore, when defining our attributes in ``include/tvm/relay/attrs/transform.h`` we choose the axis,
+accumulation dtype, and exclusivity of the operation as appropriate fields for the struct.
+
+.. code:: c++
+
+ /*! \brief Attributes used in cumsum and cumprod operator */
+ struct ScanopAttrs : public tvm::AttrsNode<ScanopAttrs> {
+ Integer axis;
+ DataType dtype;
+ Bool exclusive = Bool(false);
+ TVM_DECLARE_ATTRS(ScanopAttrs, "relay.attrs.ScanopAttrs") {
+ TVM_ATTR_FIELD(axis).describe("The axis to operate over").set_default(NullValue<Integer>());
+ TVM_ATTR_FIELD(dtype).describe("Output data type").set_default(NullValue<DataType>());
+ TVM_ATTR_FIELD(exclusive)
+ .describe("The first element is not included")
+ .set_default(Bool(false));
+ }
+ };
+
+2. Writing a Type Relation
+--------------------------
To allow for flexibility in registering operators and greater
expressivity and granularity in expressing types in Relay, operators
are typed using relations between input and output types. These relations
are represented as functions that take in a list of input types and
output types (any of these types may be incomplete) and return a list
-of input and output types that satisfies the relation. Essentially, a
+of input and output types that satisfies the relation. This includes shape
+information which can be determined statically at compile time. Essentially, a
relation for an operator can enforce all the necessary typing rules
(namely by inspecting the input types) in addition to computing the
output type.
-For example, see ``src/relay/op/type_relations.h`` and their
-implementations. E.g., ``BroadcastRel`` takes two input types and an
-output type, checks that they are all tensor types with the same underlying
-data type, and finally ensures that the shape of the output type is the
-broadcast of the input types' shapes.
+Type relation for the cumulative product and sum operators can be found in
+``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ TVM_REGISTER_NODE_TYPE(ScanopAttrs);
+ bool ScanopRel(const Array<Type>& types, int num_inputs, const Attrs& attrs, const TypeReporter& reporter) {
+ // types: [data, output]
+ ICHECK_EQ(types.size(), 2) << "Expects two types, one for the input and another for the output";
+ const auto* data = types[0].as<TensorTypeNode>();
+ if (data == nullptr) {
+ ICHECK(types[0].as<IncompleteTypeNode>())
+ << "Scanop: expect input type to be TensorType but get " << types[0];
+ return false;
+ }
+
+ const auto* param = attrs.as<ScanopAttrs>();
+
+ auto dtype = param->dtype;
+ if (dtype.is_void()) {
+ dtype = data->dtype;
+ }
+
+ if (param->axis.defined()) {
+ reporter->Assign(types[1], TensorType(data->shape, dtype));
+ } else {
+ auto prod = data->shape[0];
+ for (size_t i = 1; i < data->shape.size(); ++i) {
+ prod = prod * data->shape[i];
+ }
+ reporter->Assign(types[1], TensorType({prod}, dtype));
+ }
+
+ return true;
+ }
-It may be necessary to add another type relation to ``type_relations.h``
-if the existing ones do not capture the behavior of the desired operator.
+3. Relating the Arity and Attributes to an Operation
+----------------------------------------------------
+We then register the name of our new ops and annotate them with the calling interface.
The ``RELAY_REGISTER_OP`` macro in C++ allows a developer
to specify the following information about an operator in Relay:
- Arity (number of arguments)
- Names and descriptions for positional arguments
- Support level (1 indicates an internal intrinsic; higher numbers indicate less integral or externally supported operators)
- A type relation for the operator
+- Other annotations useful when optimizing the operation.
+
+Once again we add this to ``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ RELAY_REGISTER_OP("cumsum")
+ .describe(
+ R"doc(Return the cumulative sum of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumsum", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+ RELAY_REGISTER_OP("cumprod")
+ .describe(
+ R"doc(Return the cumulative product of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumprod", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+In this case the ``TOpPattern`` is a hint to the compiler on the pattern of computation the operator does, which might be
+useful for reordering loops and fusing operators. ``kOpaque`` tells TVM not to not bother trying to fuse this operator.
+
+4. Defining the Compute of the Operation
+----------------------------------------
+
+While we've now defined the interface for the operation but still have not
+told TVM how to perform the actual calculations for cumulative sum and product.
Review comment:
PTAL
##########
File path: docs/dev/relay_add_op.rst
##########
@@ -15,75 +15,297 @@
specific language governing permissions and limitations
under the License.
-.. _relay-add-op:
+.. _relay-add-op:
Adding an Operator to Relay
===========================
-In order to use TVM operators from within the Relay IR, the
-operators need to be registered in Relay in order to ensure
-that they will be integrated into Relay's type system.
+In this document we will go over the steps needed to register a new TVM operator
+in Relay. We will be following this PR which adds a `cumulative product`_ operation as an example.
+The PR itself builds upon another PR which adds a `cumulative sum`_ operation.
-Registering an operator requires three steps:
+.. _cumulative product: https://github.com/apache/tvm/pull/7722
+.. _cumulative sum: https://github.com/apache/tvm/pull/7334
-- Using the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity and type information
-- Defining a C++ function to produce a call node for the operator and registering a Python API hook for the function
-- Wrapping the above Python API hook in a neater interface
+Registering a new operator requires a few steps:
-The file ``src/relay/op/tensor/binary.cc`` provides
-examples of the first two steps, while
-``python/tvm/relay/op/tensor.py`` gives examples of the
-last.
+1. Add an attribute node declaring fixed arguments which are known at compile time
+2. Write a type relation for your operation to integrate into Relay's type system.
+3. Use the ``RELAY_REGISTER_OP`` macro in C++ to register the operator's arity, type, and other hints for the compiler
+4. Write how the operator is computed
+5. Register the compute, schedule with the relay operator
+6. Define a C++ function to produce a call node for the operator and registering a Python API hook for the function
+7. Wrapping the above Python API hook in a neater interface
+8. Writing tests for the new relay operator
-Registering an Operator
------------------------
+1. Defining an Attribute Node
+-----------------------------
+Attributes are fixed arguments which are supposed to be known at compile time. The stride and dilation of a convolution
+operator would be an appropriate example of fields which might belong in an attribute node for a convolution operator.
-TVM already has an operator registry, but Relay cannot properly
-incorporate TVM operators without additional type information.
+Attributes should be defined in a file within the folder `include/tvm/relay/attrs/`_.
+.. _include/tvm/relay/attrs/: https://github.com/apache/tvm/tree/main/include/tvm/relay/attrs
+
+Ultimately we want to create an operator whose interface can be seen clearly in the final python interface:
+
+.. code:: python
+
+ def cumprod(data, axis=None, dtype=None, exclusive=None):
+ """Numpy style cumprod op. Return the cumulative inclusive product of the elements along
+ a given axis.
+ Parameters
+ ----------
+ data : relay.Expr
+ The input data to the operator.
+ axis : int, optional
+ Axis along which the cumulative product is computed. The default (None) is to compute
+ the cumprod over the flattened array.
+ dtype : string, optional
+ Type of the returned array and of the accumulator in which the elements are multiplied.
+ If dtype is not specified, it defaults to the dtype of data.
+ exclusive : bool, optional
+ If true will return exclusive product in which the first element is not
+ included. In other terms, if true, the j-th output element would be
+ the product of the first (j-1) elements. Otherwise, it would be the product of
+ the first j elements. The product of zero elements will be 1.
+ Returns
+ -------
+ result : relay.Expr
+ The result has the same size as data, and the same shape as data if axis is not None.
+ If axis is None, the result is a 1-d array.
+ """
+
+A similiar interface exists for ``cumsum()``.
+
+Therefore, when defining our attributes in ``include/tvm/relay/attrs/transform.h`` we choose the axis,
+accumulation dtype, and exclusivity of the operation as appropriate fields for the struct.
+
+.. code:: c++
+
+ /*! \brief Attributes used in cumsum and cumprod operator */
+ struct ScanopAttrs : public tvm::AttrsNode<ScanopAttrs> {
+ Integer axis;
+ DataType dtype;
+ Bool exclusive = Bool(false);
+ TVM_DECLARE_ATTRS(ScanopAttrs, "relay.attrs.ScanopAttrs") {
+ TVM_ATTR_FIELD(axis).describe("The axis to operate over").set_default(NullValue<Integer>());
+ TVM_ATTR_FIELD(dtype).describe("Output data type").set_default(NullValue<DataType>());
+ TVM_ATTR_FIELD(exclusive)
+ .describe("The first element is not included")
+ .set_default(Bool(false));
+ }
+ };
+
+2. Writing a Type Relation
+--------------------------
To allow for flexibility in registering operators and greater
expressivity and granularity in expressing types in Relay, operators
are typed using relations between input and output types. These relations
are represented as functions that take in a list of input types and
output types (any of these types may be incomplete) and return a list
-of input and output types that satisfies the relation. Essentially, a
+of input and output types that satisfies the relation. This includes shape
+information which can be determined statically at compile time. Essentially, a
relation for an operator can enforce all the necessary typing rules
(namely by inspecting the input types) in addition to computing the
output type.
-For example, see ``src/relay/op/type_relations.h`` and their
-implementations. E.g., ``BroadcastRel`` takes two input types and an
-output type, checks that they are all tensor types with the same underlying
-data type, and finally ensures that the shape of the output type is the
-broadcast of the input types' shapes.
+Type relation for the cumulative product and sum operators can be found in
+``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ TVM_REGISTER_NODE_TYPE(ScanopAttrs);
+ bool ScanopRel(const Array<Type>& types, int num_inputs, const Attrs& attrs, const TypeReporter& reporter) {
+ // types: [data, output]
+ ICHECK_EQ(types.size(), 2) << "Expects two types, one for the input and another for the output";
+ const auto* data = types[0].as<TensorTypeNode>();
+ if (data == nullptr) {
+ ICHECK(types[0].as<IncompleteTypeNode>())
+ << "Scanop: expect input type to be TensorType but get " << types[0];
+ return false;
+ }
+
+ const auto* param = attrs.as<ScanopAttrs>();
+
+ auto dtype = param->dtype;
+ if (dtype.is_void()) {
+ dtype = data->dtype;
+ }
+
+ if (param->axis.defined()) {
+ reporter->Assign(types[1], TensorType(data->shape, dtype));
+ } else {
+ auto prod = data->shape[0];
+ for (size_t i = 1; i < data->shape.size(); ++i) {
+ prod = prod * data->shape[i];
+ }
+ reporter->Assign(types[1], TensorType({prod}, dtype));
+ }
+
+ return true;
+ }
-It may be necessary to add another type relation to ``type_relations.h``
-if the existing ones do not capture the behavior of the desired operator.
+3. Relating the Arity and Attributes to an Operation
+----------------------------------------------------
+We then register the name of our new ops and annotate them with the calling interface.
The ``RELAY_REGISTER_OP`` macro in C++ allows a developer
to specify the following information about an operator in Relay:
- Arity (number of arguments)
- Names and descriptions for positional arguments
- Support level (1 indicates an internal intrinsic; higher numbers indicate less integral or externally supported operators)
- A type relation for the operator
+- Other annotations useful when optimizing the operation.
+
+Once again we add this to ``src/relay/op/tensor/transform.cc``:
+
+.. code:: c++
+
+ RELAY_REGISTER_OP("cumsum")
+ .describe(
+ R"doc(Return the cumulative sum of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumsum", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+ RELAY_REGISTER_OP("cumprod")
+ .describe(
+ R"doc(Return the cumulative product of the elements along a given axis.)doc" TVM_ADD_FILELINE)
+ .set_num_inputs(1)
+ .add_argument("data", "Tensor", "The input tensor.")
+ .set_support_level(3)
+ .add_type_rel("Cumprod", ScanopRel)
+ .set_attr<TOpPattern>("TOpPattern", kOpaque);
+
+In this case the ``TOpPattern`` is a hint to the compiler on the pattern of computation the operator does, which might be
+useful for reordering loops and fusing operators. ``kOpaque`` tells TVM not to not bother trying to fuse this operator.
+
+4. Defining the Compute of the Operation
+----------------------------------------
+
+While we've now defined the interface for the operation but still have not
+told TVM how to perform the actual calculations for cumulative sum and product.
+
+Writing this code is outside the scope of the tutorial. For now, we assume
+we have a well tested implementation for the operation's compute. For
+more details on how to do this, we recommend looking up the tutorials
+on `tensor expressions`_, `TVM's operator inventory (topi)`_ and looking at the
+examples cumulative sum and product found in `python/tvm/topi/scan.py`_ and
+`python/tvm/topi/cuda/scan.py`_. In the case of our cumulative sum and product operations
+we write things directly in `TIR`_ which is the representation where tensor expressions
+and topi will lower into.
+
+.. _tensor expressions: https://tvm.apache.org/docs/tutorials/get_started/tensor_expr_get_started.html
+.. _TVM's operator inventory (topi): https://tvm.apache.org/docs/tutorials/topi/intro_topi.html
+.. _TIR: https://tvm.apache.org/docs/dev/index.html?highlight=tir#tvm-tir
+.. _python/tvm/topi/scan.py: https://github.com/apache/tvm/blob/main/python/tvm/topi/scan.py
+.. _python/tvm/topi/cuda/scan.py: https://github.com/apache/tvm/blob/main/python/tvm/topi/cuda/scan.py
+
+5. Hooking up Compute and Strategy with Relay
+---------------------------------------------
+
+After you have implemented how your function can be computed we now need to glue it to our
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org