You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/12/10 20:40:39 UTC

[GitHub] [tvm] ZihengJiang opened a new pull request #7084: [WIP][TIR] Support Return in TIR

ZihengJiang opened a new pull request #7084:
URL: https://github.com/apache/tvm/pull/7084


   This PR supports building TIR function directly and allows TIR returns value directly. 
   Current approach is by adding an intrinsic for `return`.  Example:
   ```python
   def add():
       a = tir.Var("a", "float32") 
       b = tir.Var("b", "float32") 
       c = a + b
       c = tir.call_intrin("float32", "tir.myreturn", c) 
       c = tir.Evaluate(c)
       func = tir.PrimFunc([a, b], c)
       mod = tvm.IRModule({'add': func})
       func = tvm.build(mod['add'])
       out = func(1.0, 2.0)
       print(out)
   ```
   
   In the future, with the tvm script, we should be able to write some like:
   ```python
   @tvm.script
   def add(a, b):
       tir.myreturn(a + b)
   ```
   
   There are two things need to be discussed:
   - Because `return` is a keyword in Python, currently I use `myreturn`, we should have another name for it.
   - Whether we should support multiple `return` in a single function.
   
   @tqchen @junrushao1994 @areusch 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] jroesch commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
jroesch commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544728295



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       I agree with Tristan, I think we should consider bare `CHECK` statements as a code-smell as any new users trying to understand why an invariant doesn't hold have zero context, given the fact that some are invariants and some are just lazy error handling we should better differentiate which is which and not leave it up to the user.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r554276298



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    ICHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::ret())) {
+        ICHECK_EQ(call->args.size(), 1) << "tir.ret expect a single argument.";
+        ret = WriteToOut(call->args[0], ret_var_, ret_tcode_);
+      }
+    }
+    return ret;
+  }
+
+ private:
+  std::pair<int, PrimExpr> ConvertForFFI(PrimExpr val) {
+    DataType dtype = val.dtype();
+    if (dtype.is_int() || dtype.is_uint()) {
+      return {kTVMArgInt, Cast(DataType::Int(64), val)};
+    } else if (dtype.is_float()) {
+      return {kTVMArgFloat, Cast(DataType::Float(64), val)};
+    } else if (dtype.is_void()) {
+      return {kTVMNullptr, val};
+    } else {
+      LOG(FATAL) << "data type " << dtype << " not supported yet";
+    }
+    return {kTVMNullptr, val};
+  }
+  // convert val's data type to FFI data type, return type code
+  Stmt WriteToOut(PrimExpr val, Var ret_var, Var ret_tcode) {
+    auto p = ConvertForFFI(val);
+    int tcode = p.first;
+    val = p.second;
+    Stmt store_val = Store(ret_var_, val, 0, const_true());
+    Stmt store_tcode = Store(ret_tcode_, tcode, 0, const_true());
+    return SeqStmt({store_val, store_tcode});

Review comment:
       We might need a low level ret intrinsic (that translate to return 0) to terminate the control flow, instead of continue running things.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [WIP][TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r540609823



##########
File path: python/tvm/driver/build_module.py
##########
@@ -159,17 +159,27 @@ def lower(sch, args, name="main", binds=None, simple_mode=False):
     lower_phase2 = [x[1] for x in add_lower_pass if x[0] == 2]
     lower_phase3 = [x[1] for x in add_lower_pass if x[0] > 2]
 
+    is_tir_schedule = False
+
     # Phase 0
     if isinstance(sch, schedule.Schedule):
         mod = form_irmodule(sch, args, name, binds)
+    elif isinstance(sch, tvm.tir.PrimFunc):

Review comment:
       Let us only support build from IRModule not PrimFunc

##########
File path: python/tvm/driver/build_module.py
##########
@@ -159,17 +159,27 @@ def lower(sch, args, name="main", binds=None, simple_mode=False):
     lower_phase2 = [x[1] for x in add_lower_pass if x[0] == 2]
     lower_phase3 = [x[1] for x in add_lower_pass if x[0] > 2]
 
+    is_tir_schedule = False
+
     # Phase 0
     if isinstance(sch, schedule.Schedule):
         mod = form_irmodule(sch, args, name, binds)
+    elif isinstance(sch, tvm.tir.PrimFunc):
+        func = sch.with_attr("global_symbol", name)
+        if pass_ctx.config.get("tir.restricted_func"):
+            func = func.with_attr("tir.noalias", True)

Review comment:
       no alias and global symbol should be part of the IRModule before passing into build




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tkonolige commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544734076



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       Yeah, we certainly could add another `ICHECK` macro for that.
   
   To be honest though, I don't really understand this invariant. Does the base class (StmtMutator) always return the same type of node you pass into it?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] ZihengJiang commented on a change in pull request #7084: [WIP][TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
ZihengJiang commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r540483035



##########
File path: include/tvm/tir/builtin.h
##########
@@ -41,6 +41,10 @@ namespace tir {
 
 /*! \brief Collection of builtin intrinsics as ops */
 namespace builtin {
+/*!
+ * \brief Return value.
+ */
+TVM_DLL const Op& myreturn();

Review comment:
       @tqchen  how about `tir.ret`




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tkonolige commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544722927



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       As someone who has hit many of these errors, an error message goes a long way towards helping me debug these problems. Even if the error message is as simple as "Internal invariant violated: an EvaluateNode was expected, but we got a ___". The message "CHECK(eval)" tells me nothing.
   
   I don't think the argument for binary size maxes much of a difference here. We already include a string for the CHECK statement, this just makes it marginally larger.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544715878



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       I would agree for cases where the invariant was less clear an error message would be helpful. In this particular case the impl of StmtMutator was quite direct and invariant is clear, so adding error message won't buy us too much in this particular case. Additionally, having to have a long error message for all cases would increase the binary size.
   
   I think we should certainly deliberate error messages that we do hit. And prioritize on clear error messages on those cases. While still encourage deliberate add of ICHECKs to more places without requiring error message on the obvious cases.
   
   One way to do so is to re-actively add them when we hit the problem. This case is certainly fine




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544730063



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       Thanks @tkonolige. 
   
   In this particular case, the error message point to this line, and by looking around the context, one line above, we know that the error is caused by the type, so the context gives enough info. Notably, from the impl of StmtMutator this is an internal error that won't trigger and the check here is mainly a reasonable check.
   
   I agree that in some other cases the context is certainly not enough for debug, and having a better error message would help. While adding error messages is certainly fine in this case, I do think not putting constraint on development in this particular case is helpful and @ZihengJiang can make the decision of what level of error message to be put in this internal check.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544707446



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::ret())) {
+        CHECK_EQ(call->args.size(), 1);

Review comment:
       This is a code invariant so changing to ICHECK should suffice




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544707225



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       in this particular case because it is a code invariant(StmtMutator returns the same type). So changing to ICHECK without error message should be fine




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] ZihengJiang commented on a change in pull request #7084: [WIP][TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
ZihengJiang commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r540622156



##########
File path: python/tvm/driver/build_module.py
##########
@@ -159,17 +159,27 @@ def lower(sch, args, name="main", binds=None, simple_mode=False):
     lower_phase2 = [x[1] for x in add_lower_pass if x[0] == 2]
     lower_phase3 = [x[1] for x in add_lower_pass if x[0] > 2]
 
+    is_tir_schedule = False
+
     # Phase 0
     if isinstance(sch, schedule.Schedule):
         mod = form_irmodule(sch, args, name, binds)
+    elif isinstance(sch, tvm.tir.PrimFunc):

Review comment:
       @tkonolige for building TIR function 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] ZihengJiang commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
ZihengJiang commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544730479



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       Given that informations like "internal invariant violated: an EvaluateNode was expected, but we got a ___".  is quite boilerplate, maybe `ICHECK` should be improved instead of writing error message in every place.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#issuecomment-747478194


   Thanks @tkonolige @jroesch @ZihengJiang on discussing the issue of ICHECK and comments. While I in general agree that for cases that are user facing, having clear error message would be helpful.
   
   I would be hesitant to enforce having error message for every internal invariance check. This decision is more of an engineering tradeoff. There are many ways to improve code robustness and readability, code comment is another.  However, it does not necessarily mean we need to comment every line of code, even though it is possible to do so and doing so might improve readability slightly. 
   
   The main reason is developer may want to spend their attentions in other matters and we also need to balance the need to land things reasonable quickly under the quality bar, so as long as the code structure is clear, we could focus the code review on more important things -- e.g. documenting the interface clearly, making sure the overall logics are correct.
   
   Similarly, while it is always possible to add error message to every internal invariance checks, I don't think we are required to do so for every care. We can find many similar examples(like the ones below) in other large code base like TensorFlow and PyTorch, doing so doesn't make pytorch or tensorflow not product ready.
   
   - https://github.com/pytorch/pytorch/blob/master/torch/csrc/utils/throughput_benchmark.cpp#L25
   - https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/common_runtime/replicate_per_replica_nodes_test.cc#L54
   
   Of course this does not mean that we should not add error messages to the checks that could cause user facing errors. We should actually shift our attention to address these checks quickly, while giving flexibility to the developers in other cases that have minimum gain.
   
   In summary, context of the code matters, and I believe code review should focus on understanding the overall logics, trying to think about readability in a broader context(other than error message itself), while encouraging things to move on smoothly once things met a good quality. In this case we have discussed and I believe the ICHECK on pointer null invariant checking is not necessary. While it is indeed helpful to have better error message in other two cases.
   
   
    
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tkonolige commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544735615



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::ret())) {
+        CHECK_EQ(call->args.size(), 1);

Review comment:
       When you've written the code thinks may appear simple to you, but a new user or outside maybe be very confused.
   
   Yes, I think we should add error messages in all those places.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] ZihengJiang commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
ZihengJiang commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544733728



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::ret())) {
+        CHECK_EQ(call->args.size(), 1);

Review comment:
       @tkonolige Things like this is quite easy to understand by eyeball the error throwing, I don't think error message is necessary for every place or would you like to add error message for every place like here? https://github.com/apache/tvm/blob/main/src/relay/op/nn/pad.cc#L58-L59




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#issuecomment-746322447


   One additional comment, overall lgtm. @junrushao1994 @tkonolige @areusch please take another look and https://tvm.apache.org/docs/contribute/code_review.html#approve-and-request-changes-explicitly


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen merged pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen merged pull request #7084:
URL: https://github.com/apache/tvm/pull/7084


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#issuecomment-761605334


   Thanks @ZihengJiang @tkonolige @junrushao1994 @jroesch @areusch 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544730063



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       Thanks @tkonolige. 
   
   In this particular case, the error message point to this line, and by looking around the context, one line above, we know that the error is caused by the type, so the context gives enough info. Notably, from the impl of StmtMutator this is an internal error that won't trigger and the check here is mainly a pre-caution.
   
   I agree that in some other cases the context is certainly not enough for debug, and having a better error message would help. While adding error messages is certainly fine in this case, I do think not putting constraint on development in this particular case is helpful and @ZihengJiang can make the decision of what level of error message to be put in this internal check.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] junrushao1994 commented on a change in pull request #7084: [WIP][TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r540486888



##########
File path: include/tvm/tir/builtin.h
##########
@@ -41,6 +41,10 @@ namespace tir {
 
 /*! \brief Collection of builtin intrinsics as ops */
 namespace builtin {
+/*!
+ * \brief Return value.
+ */
+TVM_DLL const Op& myreturn();

Review comment:
       I think `tir.ret` looks better




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#issuecomment-755392079


   @ZihengJiang would be great if we can act on the thread and bring it in :)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r554276298



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    ICHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::ret())) {
+        ICHECK_EQ(call->args.size(), 1) << "tir.ret expect a single argument.";
+        ret = WriteToOut(call->args[0], ret_var_, ret_tcode_);
+      }
+    }
+    return ret;
+  }
+
+ private:
+  std::pair<int, PrimExpr> ConvertForFFI(PrimExpr val) {
+    DataType dtype = val.dtype();
+    if (dtype.is_int() || dtype.is_uint()) {
+      return {kTVMArgInt, Cast(DataType::Int(64), val)};
+    } else if (dtype.is_float()) {
+      return {kTVMArgFloat, Cast(DataType::Float(64), val)};
+    } else if (dtype.is_void()) {
+      return {kTVMNullptr, val};
+    } else {
+      LOG(FATAL) << "data type " << dtype << " not supported yet";
+    }
+    return {kTVMNullptr, val};
+  }
+  // convert val's data type to FFI data type, return type code
+  Stmt WriteToOut(PrimExpr val, Var ret_var, Var ret_tcode) {
+    auto p = ConvertForFFI(val);
+    int tcode = p.first;
+    val = p.second;
+    Stmt store_val = Store(ret_var_, val, 0, const_true());
+    Stmt store_tcode = Store(ret_tcode_, tcode, 0, const_true());
+    return SeqStmt({store_val, store_tcode});

Review comment:
       We might need a low level ret intrinsic to terminate the control flow, instead of continue running things.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] ZihengJiang commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
ZihengJiang commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544733728



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::ret())) {
+        CHECK_EQ(call->args.size(), 1);

Review comment:
       Things like this is quite easy to understand by eyeball the error throwing, I don't think error message is necessary for every place or would you like to add error message for every place like here? https://github.com/apache/tvm/blob/main/src/relay/op/nn/pad.cc#L58-L59




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544769579



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       It is important to take the context into consideration. For the checks that would leads to a user facing error, e.g. the API expects inputs of two elements but only get one, a more comprehensive error message should be used -- and we should use CHECK instead of ICHECK in the user facing API part.
   
   In cases like this one, this is an invariant, and we could give the developer the freedom to choose the appropriate error message, since the ease of development is also another factor we want to balance.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] ZihengJiang commented on pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
ZihengJiang commented on pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#issuecomment-747613626


   @tkonolige 
   This will generate a function that only evaluate but not return:
   ``` python
   def func(a, b):
       c = a + b
       # return c
   ```
   Think and have a try by yourself would make you understand other's PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tkonolige commented on pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tkonolige commented on pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#issuecomment-747598083


   I'm a little confused on how we use return in tir? How is the example you give different if we just remove the return?
   ```python
   a = tir.Var("a", "float32") 
   b = tir.Var("b", "float32") 
   c = a + b
   # c = tir.call_intrin("float32", "tir.myreturn", c)
   c = tir.Evaluate(c) /# doesn't this return a + b?
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tkonolige commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544722927



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       As someone who has hit many of these errors, an error message goes a long way towards helping me debug these problems. Even if the error message is as simple as "Internal invariant violated: an EvaluateNode was expected, but we got a ___". The message "CHECK(eval)" tells me nothing.
   
   I don't think the argument for binary size maxes much of a difference here. We already include a string for the CHECK statement, this just makes it marginally larger. If this is really an issue, we can strip the error message on release builds.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tkonolige commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544710171



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       We always say things are invariant, yet I've hit many CHECKs for invariants. It doesn't take much time to add a helpful error message




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544772948



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::ret())) {
+        CHECK_EQ(call->args.size(), 1);

Review comment:
       I think the rule of thumb is whether or not a user might hit the error, and we should look at the context of the check. 
   
   Notably, we encourage putting ICHECK deliberately in many places and the ease of adding checks quickly facilitate development. Notably, if it is an ICHECK, it usually means it is not a user facing error. We need to consider the context in each case 
   
   In this particular case, an error could be hit when the call is incorrectly constructed and the IR being invalid. Which is a rare case, but user could hit it in certain cases when invalid IR is constructed , so I would say an error message would be helpful. Although we should also think to which point is helpful.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544730502



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::ret())) {
+        CHECK_EQ(call->args.size(), 1);

Review comment:
       Thanks @tkonolige , I now agree that in this particular case i agree that having an error message like "ret expect a single argument" would be helpful.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544715878



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       I would agree for cases where the invariant was less clear. In this particular case the impl of StmtMutator was quite direct and invariant is clear, so adding error message won't buy us too much in this particular case. Additionally, having to have a long error message for all cases would increase the binary size.
   
   I think we should certainly deliberate error messages that we do hit. One way to do so is to re-actively add them when we hit the problem. This case is certainly fine




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] ZihengJiang commented on pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
ZihengJiang commented on pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#issuecomment-747634376


   @tkonolige 
   Happy to address your comment and hope it will make you understand TIR. Yes, usually a zero value would be returned before this PR and now a value can be explicitly returned.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r554276298



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    ICHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::ret())) {
+        ICHECK_EQ(call->args.size(), 1) << "tir.ret expect a single argument.";
+        ret = WriteToOut(call->args[0], ret_var_, ret_tcode_);
+      }
+    }
+    return ret;
+  }
+
+ private:
+  std::pair<int, PrimExpr> ConvertForFFI(PrimExpr val) {
+    DataType dtype = val.dtype();
+    if (dtype.is_int() || dtype.is_uint()) {
+      return {kTVMArgInt, Cast(DataType::Int(64), val)};
+    } else if (dtype.is_float()) {
+      return {kTVMArgFloat, Cast(DataType::Float(64), val)};
+    } else if (dtype.is_void()) {
+      return {kTVMNullptr, val};
+    } else {
+      LOG(FATAL) << "data type " << dtype << " not supported yet";
+    }
+    return {kTVMNullptr, val};
+  }
+  // convert val's data type to FFI data type, return type code
+  Stmt WriteToOut(PrimExpr val, Var ret_var, Var ret_tcode) {
+    auto p = ConvertForFFI(val);
+    int tcode = p.first;
+    val = p.second;
+    Stmt store_val = Store(ret_var_, val, 0, const_true());
+    Stmt store_tcode = Store(ret_tcode_, tcode, 0, const_true());
+    return SeqStmt({store_val, store_tcode});

Review comment:
       We might need a low level ret intrinsic to terminate the control flow, instead of continue running things.

##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    ICHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::ret())) {
+        ICHECK_EQ(call->args.size(), 1) << "tir.ret expect a single argument.";
+        ret = WriteToOut(call->args[0], ret_var_, ret_tcode_);
+      }
+    }
+    return ret;
+  }
+
+ private:
+  std::pair<int, PrimExpr> ConvertForFFI(PrimExpr val) {
+    DataType dtype = val.dtype();
+    if (dtype.is_int() || dtype.is_uint()) {
+      return {kTVMArgInt, Cast(DataType::Int(64), val)};
+    } else if (dtype.is_float()) {
+      return {kTVMArgFloat, Cast(DataType::Float(64), val)};
+    } else if (dtype.is_void()) {
+      return {kTVMNullptr, val};
+    } else {
+      LOG(FATAL) << "data type " << dtype << " not supported yet";
+    }
+    return {kTVMNullptr, val};
+  }
+  // convert val's data type to FFI data type, return type code
+  Stmt WriteToOut(PrimExpr val, Var ret_var, Var ret_tcode) {
+    auto p = ConvertForFFI(val);
+    int tcode = p.first;
+    val = p.second;
+    Stmt store_val = Store(ret_var_, val, 0, const_true());
+    Stmt store_tcode = Store(ret_tcode_, tcode, 0, const_true());
+    return SeqStmt({store_val, store_tcode});

Review comment:
       We might need a low level ret intrinsic (that translate to return 0) to terminate the control flow, instead of continue running things.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544764730



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       yes, the base class always return the same type(as they recursively mutate the child and reconstructs the same node) in this case and it is more of an invariant check.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen edited a comment on pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen edited a comment on pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#issuecomment-747478194


   Thanks @tkonolige @jroesch @ZihengJiang on discussing the issue of ICHECK and comments. While I in general agree that for cases that are user facing, having clear error message would be helpful.
   
   I would be hesitant to enforce error message for every internal invariance check. This decision is more of an engineering tradeoff. For example are many ways to improve code robustness and readability, code comment is one of them.  However, it does not necessarily mean we need to comment every line of code, even though it is possible to do so and doing so might improve readability slightly. 
   
   Developers may want to spend their attentions in other matters and we also need to balance these concerns, so as long as the code structure is clear, we could focus the code review on more important things -- e.g. documenting the interface clearly, making sure the overall logics are correct.
   
   Similarly, while it is always possible to add error message to every internal invariance checks, I don't think we are required to do so for every care. We can find many similar examples(like the ones below) in other large code base like TensorFlow and PyTorch, doing so doesn't make pytorch or tensorflow not product ready.
   
   - https://github.com/pytorch/pytorch/blob/master/torch/csrc/utils/throughput_benchmark.cpp#L25
   - https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/common_runtime/replicate_per_replica_nodes_test.cc#L54
   
   Of course this does not mean that we should not add error messages to the checks that could cause user facing errors. We should actually shift our attention to address these checks quickly, while giving flexibility to the developers in other cases that have minimum gain.
   
   In summary, **context of the code** matters, and I believe code review should focus on understanding the overall logics, trying to think about readability in a broader context(other than error message itself), while encouraging things to move on smoothly once things met a good quality. 
   
   In this case we have discussed and I believe the ICHECK on pointer null invariant checking is not necessary. While it is indeed helpful to have better error message in other two cases.
   
   
    
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tkonolige commented on pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tkonolige commented on pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#issuecomment-747632715


   I'm just trying to understand things better. There's not really any documentation on TIR.
   
   I had thought that the last statement in every TIR expression was implicitly returned. Am I correct in saying that before this PR calling a TIR function would always return nullptr/None? And that now you can actually return a value?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tkonolige commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544710171



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       We always say things are invariant, yet I've hit many of them. It doesn't take much time to add a helpful error message

##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::ret())) {
+        CHECK_EQ(call->args.size(), 1);

Review comment:
       See above




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544315890



##########
File path: src/tir/op/builtin.cc
##########
@@ -42,6 +42,10 @@ TIR_DEFINE_BUILTIN_FUNC(reinterpret)
     .set_attr<TCallEffectKind>("TCallEffectKind", Integer(CallEffectKind::kPure))
     .set_num_inputs(1);
 
+TIR_DEFINE_BUILTIN_FUNC(ret)
+    .set_attr<TCallEffectKind>("TCallEffectKind", Integer(CallEffectKind::kOpaque))

Review comment:
       We might want to introduce another call type, as this call changes the control flow. CallEffectKind::kControlJump




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] areusch commented on a change in pull request #7084: [WIP][TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r540496971



##########
File path: python/tvm/driver/build_module.py
##########
@@ -159,17 +159,30 @@ def lower(sch, args, name="main", binds=None, simple_mode=False):
     lower_phase2 = [x[1] for x in add_lower_pass if x[0] == 2]
     lower_phase3 = [x[1] for x in add_lower_pass if x[0] > 2]
 
+    is_tir_schedule = False
+
     # Phase 0
     if isinstance(sch, schedule.Schedule):
         mod = form_irmodule(sch, args, name, binds)
+    elif isinstance(sch, tvm.tir.PrimFunc):
+        func = sch.with_attr("global_symbol", name)
+        if pass_ctx.config.get("tir.restricted_func"):
+            func = func.with_attr("tir.noalias", True)
+        mod = tvm.IRModule({name: func})
+        is_tir_schedule = True
     else:
         mod = sch
 
     pass_list = lower_phase0
     # Phase 1
+    pass_list += [tvm.tir.transform.InjectPrefetch()]
+
+    if is_tir_schedule:
+        pass
+        # pass_list += [tvm.tir.transform.BufferFlatten()]

Review comment:
       remove the comment and invert the if

##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -222,6 +275,7 @@ namespace transform {
 
 Pass MakePackedAPI(int num_unpacked_args) {
   auto pass_func = [num_unpacked_args](IRModule m, PassContext ctx) {
+    LOG(INFO) << "Before Make Packed API:\n" << m;

Review comment:
       remove

##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -239,6 +293,7 @@ Pass MakePackedAPI(int num_unpacked_args) {
     for (const auto& pair : updates) {
       mptr->AddUnchecked(pair.first, pair.second);
     }
+    LOG(INFO) << "After Make Packed API:\n" << m;

Review comment:
       remove

##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,58 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode)
+    : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::myreturn())) {
+        CHECK_EQ(call->args.size(), 1);
+        ret = WriteToOut(call->args[0], ret_var_, ret_tcode_);
+      }
+    }
+    return ret;
+  }
+ private:
+  std::pair<int, PrimExpr> ConvertForFFI(PrimExpr val) {
+    DataType dtype = val.dtype();
+    if (dtype.is_int() || dtype.is_uint()) {
+      return {kTVMArgInt, Cast(DataType::Int(64), val)};
+    } else if (dtype.is_float()) {
+      return {kTVMArgFloat, Cast(DataType::Float(64), val)};
+    } else if (dtype.is_void()) {
+      return {kTVMNullptr, val};
+    } else {
+      LOG(FATAL) << "data type " << dtype << " not supported yet";

Review comment:
       I think we may need to return at least DLTensor for AOT

##########
File path: tests/python/unittest/test_tir_build.py
##########
@@ -0,0 +1,16 @@
+import tvm
+from tvm import tir
+
+def add():
+    a = tir.Var("a", "float32") 
+    b = tir.Var("b", "float32") 
+    c = a + b
+    c = tir.call_intrin("float32", "tir.myreturn", c) 
+    c = tir.Evaluate(c)
+    func = tir.PrimFunc([a, b], c)
+    mod = tvm.IRModule({'add': func})
+    func = tvm.build(mod['add'])
+    out = func(1.0, 2.0)
+    print(out)

Review comment:
       add an assert here




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#issuecomment-747644045


   Thanks @tkonolige , I agree, and perhaps https://github.com/apache/tvm/blob/main/python/tvm/tir/op.py is a better place given other intrinsics are in there


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tkonolige commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544443419



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       Please change these to ICHECK because they are internal errors. Also, please add an error message. Something along the lines of "Expected EvaluateNode, but got ...."

##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::ret())) {
+        CHECK_EQ(call->args.size(), 1);

Review comment:
       ICHECK_EQ with error message

##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);
+    if (const CallNode* call = eval->value.as<CallNode>()) {
+      if (call->op.same_as(builtin::ret())) {
+        CHECK_EQ(call->args.size(), 1);
+        ret = WriteToOut(call->args[0], ret_var_, ret_tcode_);
+      }
+    }
+    return ret;
+  }
+
+ private:
+  std::pair<int, PrimExpr> ConvertForFFI(PrimExpr val) {
+    DataType dtype = val.dtype();
+    if (dtype.is_int() || dtype.is_uint()) {
+      return {kTVMArgInt, Cast(DataType::Int(64), val)};
+    } else if (dtype.is_float()) {
+      return {kTVMArgFloat, Cast(DataType::Float(64), val)};
+    } else if (dtype.is_void()) {
+      return {kTVMNullptr, val};
+    } else {
+      LOG(FATAL) << "data type " << dtype << " not supported yet";

Review comment:
       Please add which data datatypes are supported to this error message




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tkonolige commented on a change in pull request #7084: [WIP][TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r540600882



##########
File path: python/tvm/driver/build_module.py
##########
@@ -159,17 +159,27 @@ def lower(sch, args, name="main", binds=None, simple_mode=False):
     lower_phase2 = [x[1] for x in add_lower_pass if x[0] == 2]
     lower_phase3 = [x[1] for x in add_lower_pass if x[0] > 2]
 
+    is_tir_schedule = False
+
     # Phase 0
     if isinstance(sch, schedule.Schedule):
         mod = form_irmodule(sch, args, name, binds)
+    elif isinstance(sch, tvm.tir.PrimFunc):

Review comment:
       I'm a little confused on how all the changes in this file it into having return for TIR?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [WIP][TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r540482265



##########
File path: include/tvm/tir/builtin.h
##########
@@ -41,6 +41,10 @@ namespace tir {
 
 /*! \brief Collection of builtin intrinsics as ops */
 namespace builtin {
+/*!
+ * \brief Return value.
+ */
+TVM_DLL const Op& myreturn();

Review comment:
       How about return_value




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r558329105



##########
File path: src/target/llvm/codegen_llvm.cc
##########
@@ -927,6 +927,12 @@ llvm::Value* CodeGenLLVM::CreateIntrinsic(const CallNode* op) {
     value->addIncoming(then_value, then_value_block);
     value->addIncoming(else_value, else_value_block);
     return value;
+  } else if (op->op.same_as(builtin::ret())) {
+    ICHECK_EQ(Downcast<IntImm>(op->args[0])->value, 0);
+    builder_->CreateRet(ConstInt32(0));
+    llvm::BasicBlock* ret_dummy = llvm::BasicBlock::Create(*ctx_, "ret_dummy", function_);

Review comment:
       Please add a comment block on why this is needed, so future readers can understand what is going on

##########
File path: src/target/llvm/codegen_llvm.cc
##########
@@ -927,6 +927,12 @@ llvm::Value* CodeGenLLVM::CreateIntrinsic(const CallNode* op) {
     value->addIncoming(then_value, then_value_block);
     value->addIncoming(else_value, else_value_block);
     return value;
+  } else if (op->op.same_as(builtin::ret())) {
+    ICHECK_EQ(Downcast<IntImm>(op->args[0])->value, 0);

Review comment:
       I believe we can also safely support other return values as int. 
   
   We need to ICHECK op->args is an IntImm of type i32, return that constant value. Add an error message that we only support returning int value for now. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] ZihengJiang commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
ZihengJiang commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544730479



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       Given that informations like "internal invariant violated: an EvaluateNode was expected, but we got a ___".  is quite boilerplate, maybe `ICHECK` should be improved instead of write error message in every place.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544764730



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       yes, the base class always return the same type in this case




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [tvm] tqchen commented on a change in pull request #7084: [TIR] Support Return in TIR

Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7084:
URL: https://github.com/apache/tvm/pull/7084#discussion_r544715878



##########
File path: src/tir/transforms/make_packed_api.cc
##########
@@ -41,6 +41,56 @@
 namespace tvm {
 namespace tir {
 
+class ReturnRewriter : public StmtMutator {
+ public:
+  explicit ReturnRewriter(Var ret_var, Var ret_tcode) : ret_var_(ret_var), ret_tcode_(ret_tcode) {}
+
+  Stmt VisitStmt_(const EvaluateNode* node) override {
+    Stmt ret = StmtMutator::VisitStmt_(node);
+    const EvaluateNode* eval = ret.as<EvaluateNode>();
+    CHECK(eval);

Review comment:
       I would agree for cases where the invariant was less clear an error message would be helpful. In this particular case the impl of StmtMutator was quite direct and invariant is clear, so adding error message won't buy us too much in this particular case. Additionally, having to have a long error message for all cases would increase the binary size.
   
   I think we should certainly deliberate error messages that we do hit. One way to do so is to re-actively add them when we hit the problem. This case is certainly fine




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org