You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "Lunderberg (via GitHub)" <gi...@apache.org> on 2024/04/09 13:56:12 UTC
[PR] [QoL][Relax] Use SeqExpr in IR types when SeqExpr is required [tvm]
Lunderberg opened a new pull request, #16859:
URL: https://github.com/apache/tvm/pull/16859
The Relax IR requires the `FunctionNode::body`, `IfNode::true_branch`, and `IfNode::false_branch` to be instances of `relax::SeqExpr`. If these Relax requirements are violated, correctly-implemented transformations may raise exceptsion
(e.g. from `Downcast` in `Downcast<SeqExpr>(func->body)->blocks`), or even segfault (e.g. when `.as` returns a nullptr in `func->body.as<SeqExprNode>()->blocks`). Debugging these failure modes is also difficult, as even the TVMScript printer relies on the body of the function being a `SeqExprNode`.
This commit updates the C++ type of `FunctionNode::body`, `IfNode::true_branch`, and `IfNode::false_branch` to be `relax::SeqExpr` instead of `relax::Expr`. This does not impact any well-formed Relax IR, and allows this type of ill-formed Relax IR type to be checked at compile-time. A large number of checks applied during TVM runtime can now be removed, as they duplicate the new compile-time check.
To maintain backwards compatibility, this commit adds a new constructor to `relax::SeqExpr`, which accepts a single `Expr body` argument. This constructor provides either an additional reference to the same underlying `relax::SeqExprNode`, if `body` already contains a `relax::SeqExprNode`, and otherwise wraps the body in a `relax::SeqExpr`. For implementations that previously produced well-formed Relax IR, this change has no effect. For implementations that previously produced ill-formed Relax IR, this change results in the equivalent well-formed Relax IR.
Alternate implementations considered:
* Perform the backwards-compatibility wrapping within the `relax::Function` and `relax::If` constructors. While this would provide the intended conversion when these constructors are used, Relax transforms make frequent use of copy-on-write (e.g. `func.CopyOnWrite()->body = new_body`), which does not use the constructor. Maintaining backwards compatibility for this usage requires the implicit conversion constructor that was chosen for this PR.
* Remove the Relax IR requirement for these expressions to be `SeqExpr`. While this would make Relax more internally consistent, such a change would break backwards compatibility that relies on `SeqExpr` being present. While the callsites within TVM could be updated to resolve this breakage, callsites outside of TVM (e.g. MLC-LLM) could not. Exposing the special case within the C++ type, as done in this PR, maintains backwards compatibility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [QoL][Relax] Use SeqExpr in IR types when SeqExpr is required [tvm]
Posted by "quic-sanirudh (via GitHub)" <gi...@apache.org>.
quic-sanirudh commented on code in PR #16859:
URL: https://github.com/apache/tvm/pull/16859#discussion_r1573204564
##########
include/tvm/relax/expr.h:
##########
@@ -915,18 +843,113 @@ class SeqExprNode : public ExprNode {
class SeqExpr : public Expr {
public:
+ /* \brief Implicit conversion constructor
+ *
+ * Relax nodes that introduce a new scope (e.g. `relax::Function`)
+ * are required to be held as SeqExpr. This implicit conversion
+ * provides allows callsites to use these member variables when the
+ * C++ compile-time type is a `relax::Expr`. For example,
+ * a transform may use `func.CopyOnWrite()->body = expr;`.
+ *
+ * If the expression is already a `relax::SeqExpr`, the same
+ * underlying `relax::SeqExprNode` is used, and no copies are made.
+ */
+ TVM_DLL SeqExpr(Expr body); // NOLINT(*)
+
TVM_DLL explicit SeqExpr(Array<BindingBlock> blocks, Expr body, Span span = Span());
TVM_DEFINE_OBJECT_REF_METHODS(SeqExpr, Expr, SeqExprNode);
TVM_DEFINE_OBJECT_REF_COW_METHOD(SeqExprNode);
};
+/*!
+ * \brief Condition expression
+ *
+ * Unlike traditional statement `if`s, the if evalutes
+ * to the result of the branch taken.
+ *
+ * x = if (true) { 1 } else { 0 }; // x is 1
+ * y = if (false) { 1 } else { 0 }; // y is 0
+ *
+ * \note This is similar to C's ternary operator.
+ */
+class IfNode : public ExprNode {
Review Comment:
Ah yes, I missed that it was declared by value. Sorry about that and thanks for the clarification and yes, I agree that "templatizing" the `ObjectRef` would be a much bigger change, and probably not needed for this since the concern was more of a stylistic one to preserve git history.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [QoL][Relax] Use SeqExpr in IR types when SeqExpr is required [tvm]
Posted by "Lunderberg (via GitHub)" <gi...@apache.org>.
Lunderberg commented on code in PR #16859:
URL: https://github.com/apache/tvm/pull/16859#discussion_r1572728664
##########
include/tvm/relax/expr.h:
##########
@@ -915,18 +843,113 @@ class SeqExprNode : public ExprNode {
class SeqExpr : public Expr {
public:
+ /* \brief Implicit conversion constructor
+ *
+ * Relax nodes that introduce a new scope (e.g. `relax::Function`)
+ * are required to be held as SeqExpr. This implicit conversion
+ * provides allows callsites to use these member variables when the
+ * C++ compile-time type is a `relax::Expr`. For example,
+ * a transform may use `func.CopyOnWrite()->body = expr;`.
+ *
+ * If the expression is already a `relax::SeqExpr`, the same
+ * underlying `relax::SeqExprNode` is used, and no copies are made.
+ */
+ TVM_DLL SeqExpr(Expr body); // NOLINT(*)
+
TVM_DLL explicit SeqExpr(Array<BindingBlock> blocks, Expr body, Span span = Span());
TVM_DEFINE_OBJECT_REF_METHODS(SeqExpr, Expr, SeqExprNode);
TVM_DEFINE_OBJECT_REF_COW_METHOD(SeqExprNode);
};
+/*!
+ * \brief Condition expression
+ *
+ * Unlike traditional statement `if`s, the if evalutes
+ * to the result of the branch taken.
+ *
+ * x = if (true) { 1 } else { 0 }; // x is 1
+ * y = if (false) { 1 } else { 0 }; // y is 0
+ *
+ * \note This is similar to C's ternary operator.
+ */
+class IfNode : public ExprNode {
Review Comment:
Unfortunately, I wasn't able to do so. Declaring an member variable in a class requires the size of that member variable to be defined when the member variable is declared. For cases where the size of a member variable is independent of the size of `T` (e.g. `T*`, `const T&`, or some templates such as `std::unique_ptr<T>`), a forward-declaration can be used. Since the `relax::SeqExpr` is held by value, it cannot be defined with a forward-reference, and so the definitions of `relax::Function` and `relax::If` must occur after `relax::SeqExpr`.
(Even though all `ObjectRef` subclasses store their data in a reference-managed pointer, and have the exact same size, I'm not aware of any way to expose this to the compiler in a forward declaration. A few years ago, I looked into what it would take to turn `ObjectRef` into a template over the internal object (e.g. `using SeqExpr = ObjectRefImpl<SeqExprNode>;`), and decided that it would be a bit more effort than I'd like. It may be worth doing so, with care to ensure that a forward reference of `SeqExprNode` would be sufficient to define `ObjectRefImpl<SeqExprNode>`, but that would be a much larger change.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [QoL][Relax] Use SeqExpr in IR types when SeqExpr is required [tvm]
Posted by "quic-sanirudh (via GitHub)" <gi...@apache.org>.
quic-sanirudh commented on code in PR #16859:
URL: https://github.com/apache/tvm/pull/16859#discussion_r1572692210
##########
include/tvm/relax/expr.h:
##########
@@ -915,18 +843,113 @@ class SeqExprNode : public ExprNode {
class SeqExpr : public Expr {
public:
+ /* \brief Implicit conversion constructor
+ *
+ * Relax nodes that introduce a new scope (e.g. `relax::Function`)
+ * are required to be held as SeqExpr. This implicit conversion
+ * provides allows callsites to use these member variables when the
+ * C++ compile-time type is a `relax::Expr`. For example,
+ * a transform may use `func.CopyOnWrite()->body = expr;`.
+ *
+ * If the expression is already a `relax::SeqExpr`, the same
+ * underlying `relax::SeqExprNode` is used, and no copies are made.
+ */
+ TVM_DLL SeqExpr(Expr body); // NOLINT(*)
+
TVM_DLL explicit SeqExpr(Array<BindingBlock> blocks, Expr body, Span span = Span());
TVM_DEFINE_OBJECT_REF_METHODS(SeqExpr, Expr, SeqExprNode);
TVM_DEFINE_OBJECT_REF_COW_METHOD(SeqExprNode);
};
+/*!
+ * \brief Condition expression
+ *
+ * Unlike traditional statement `if`s, the if evalutes
+ * to the result of the branch taken.
+ *
+ * x = if (true) { 1 } else { 0 }; // x is 1
+ * y = if (false) { 1 } else { 0 }; // y is 0
+ *
+ * \note This is similar to C's ternary operator.
+ */
+class IfNode : public ExprNode {
Review Comment:
Do we have to move the whole definition of `IfNode` to a new location, because this would make the git history a bit harder to read later on. Could we perhaps do a forward declaration of `SeqExpr` class and change the definition of `IfNode` in place?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [QoL][Relax] Use SeqExpr in IR types when SeqExpr is required [tvm]
Posted by "Lunderberg (via GitHub)" <gi...@apache.org>.
Lunderberg commented on PR #16859:
URL: https://github.com/apache/tvm/pull/16859#issuecomment-2045272700
This PR is currently marked as a draft, to allow time for community discussion. A forum thread ([link](https://discuss.tvm.apache.org/t/relax-ir-fields-that-require-seqexpr/17055)) for discussion has also been started for this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [QoL][Relax] Use SeqExpr in IR types when SeqExpr is required [tvm]
Posted by "quic-sanirudh (via GitHub)" <gi...@apache.org>.
quic-sanirudh merged PR #16859:
URL: https://github.com/apache/tvm/pull/16859
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org