You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "Lunderberg (via GitHub)" <gi...@apache.org> on 2024/01/12 18:53:42 UTC

Re: [PR] [Unity][Transform] Replace eligible operators with in-place versions in dataflow blocks [tvm]

Lunderberg commented on code in PR #16129:
URL: https://github.com/apache/tvm/pull/16129#discussion_r1450691632


##########
include/tvm/relax/transform.h:
##########
@@ -526,6 +526,15 @@ TVM_DLL Pass ConvertLayout(Map<String, Array<String>> desired_layouts);
  */
 TVM_DLL Pass DeadCodeElimination(Array<runtime::String> entry_functions);
 
+/*!
+ * \brief Pass that changes calls to supported operators in dataflow blocks into in-place
+ * implementations. Supported operators will be replaced by calls to `call_tir_inplace` that invoke

Review Comment:
   Does "supported operators" include `R.call_tir`?  From the description alone, I can't tell where I would expect to use this pass in a lowering pipeline.  Can we explicitly state whether it should be called on a module with relax-level operators  such as `R.matmul`, or lowered versions such as `R.call_tir("matmul_impl")`.



##########
tests/python/relax/test_dataflow_inplace.py:
##########
@@ -0,0 +1,464 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from typing import List, Set, Tuple
+import tvm
+from tvm import relax, testing
+from tvm.relax.transform import DataflowUseInplaceCalls
+from tvm.relax.testing.transform import (
+    dataflow_liveness_analysis,
+    dataflow_alias_analysis,
+    dataflow_inplace_analysis,
+    dataflow_single_inplace_call,
+)
+from tvm.script.parser import ir as I, relax as R, tir as T
+
+import numpy as np
+
+
+def test_liveness_analysis():
+    @I.ir_module
+    class BasicLiveness:
+        @R.function
+        def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            with R.dataflow():
+                y = R.const(1, dtype="int32")
+                z = R.add(x, y)
+                q = R.multiply(z, y)
+                p = R.add(z, q)
+                n = R.multiply(p, p)
+                R.output(n)
+            return n
+
+    block = BasicLiveness["main"].body.blocks[0]
+    live_ranges = dataflow_liveness_analysis(block)
+    expected_ranges = {
+        "x": (-1, 1),
+        "y": (0, 2),
+        "z": (1, 3),
+        "q": (2, 3),
+        "p": (3, 4),
+        "n": (4, 5),
+    }
+    for var, live_range in live_ranges.items():

Review Comment:
   This test case would erroneously pass if `live_ranges` were an empty dictionary.  Instead of asserting on each item individually, it's better to make a single dictionary and assert on it.  That way, size mismatches are handled, and the pytest error messages are also improved by showing all mismatched items.
   
   ```python
   live_ranges = {var.name_hint: live_range for var,live_range in live_ranges.items()}
   assert live_ranges == expected_ranges
   ```



##########
tests/python/relax/test_dataflow_inplace.py:
##########
@@ -0,0 +1,464 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from typing import List, Set, Tuple
+import tvm
+from tvm import relax, testing
+from tvm.relax.transform import DataflowUseInplaceCalls
+from tvm.relax.testing.transform import (
+    dataflow_liveness_analysis,
+    dataflow_alias_analysis,
+    dataflow_inplace_analysis,
+    dataflow_single_inplace_call,
+)
+from tvm.script.parser import ir as I, relax as R, tir as T
+
+import numpy as np
+
+
+def test_liveness_analysis():
+    @I.ir_module
+    class BasicLiveness:
+        @R.function
+        def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            with R.dataflow():

Review Comment:
   Is a dataflow block required for the analysis, or just for the transformation?  For the analysis, it would be good to define the behavior, whether that is "works equivalently", "throws error", "ignores non-dataflow", etc.



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {
+      auto target_var = GetRef<Var>(target_var_node);
+      if (alias_map_.count(target_var)) {
+        ret.insert(alias_map_[target_var].begin(), alias_map_[target_var].end());
+      } else {
+        ret.insert(-1);
+      }
+    } else if (auto* target_tuple = value.as<TupleNode>()) {
+      // fresh idx but we update the tuple map
+      int tup_idx = get_fresh_idx();
+      ret.insert(tup_idx);
+      std::vector<std::unordered_set<int>> new_tuple_map;
+      for (auto field : target_tuple->fields) {
+        new_tuple_map.push_back(GetAliasSet(field, bound_var));
+      }
+      tuple_map_[tup_idx] = new_tuple_map;
+    } else if (auto* target_tgi = value.as<TupleGetItemNode>()) {
+      std::unordered_set<int> tuple_set = GetAliasSet(target_tgi->tuple, bound_var);
+      // if -1 is a member of the tuple set, then we have to assume the result is -1
+      if (tuple_set.count(-1)) {
+        ret.insert(-1);
+      } else {
+        // otherwise, consider all members that are tuples of appropriate size and index into them
+        // (this is safe because the type system will ensure we're not indexing into a tuple
+        // of the wrong size)
+        for (int member : tuple_set) {
+          if (tuple_map_.count(member) &&
+              static_cast<int>(tuple_map_[member].size()) > target_tgi->index) {
+            auto member_set = tuple_map_[member][target_tgi->index];
+            ret.insert(member_set.begin(), member_set.end());
+          }
+        }
+      }
+    } else if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        // call_pure_packed: treat as non-op call
+        if (op_node->name == "relax.call_pure_packed") {
+          return HandleMysteryCall(call_node, bound_var, true);
+        } else if (op_node->name == "relax.split") {
+          // split: Returns a tuple, treat as allocation
+
+          // tuple is freshly allocated, but also add components to the tuple map
+          int tup_idx = get_fresh_idx();
+          ret.insert(tup_idx);
+          // the LHS (the bound var) will definitely have a tuple struct info
+          InsertFreshTuple(tup_idx, GetStructInfoAs<TupleStructInfoNode>(bound_var));
+        } else if (op_node->name == "relax.call_tir") {
+          // call_tir: can potentially return a tuple
+          if (auto* tuple_struct_info = call_node->sinfo_args[0].as<TupleStructInfoNode>()) {
+            int tup_idx = get_fresh_idx();
+            ret.insert(tup_idx);
+            InsertFreshTuple(tup_idx, tuple_struct_info);
+          } else {
+            ret.insert(get_fresh_idx());
+          }
+        } else {
+          // We are assuming most op calls return a single fresh allocation.
+          // We may have to track more exceptions
+          ret.insert(get_fresh_idx());
+        }
+      } else {
+        // assume any non-op call can be extremely dangerous and do anything
+        return HandleMysteryCall(call_node, bound_var);
+      }
+    }
+
+    return ret;
+  }
+
+  std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual> alias_map_;
+  std::unordered_map<int, std::vector<std::unordered_set<int>>> tuple_map_;
+  int mem_idx_;
+};
+
+// given a shape, return the allocation size corresponding to it (product of elements)
+int ShapeSize(const ShapeExpr& shape) {
+  int ret = 1;
+  for (auto dim : shape->values) {
+    if (auto int_dim = dim.as<IntImmNode>()) {
+      ret *= static_cast<int>(int_dim->value);
+    } else {
+      return -1;
+    }
+  }
+  return ret;
+}
+
+// Given the struct info of the result, return any struct info nested in it
+// that is eleigible to be used for in-place computations (tensors are eligible
+// only if all their dimensions are integer constants, tuples are eligible if
+// all members are eligible though we can consider only individual members separately)
+std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> GatherCandidateSinfo(
+    const StructInfo& result_sinfo) {
+  if (auto* tensor_info = result_sinfo.as<TensorStructInfoNode>()) {
+    // don't consider void dtype (don't know the size at compile time)
+    if (tensor_info->dtype.is_void()) {
+      return {};
+    }
+    // don't consider cases where we don't know the shape at compile time
+    // TODO(@slyubomirsky): variables might be okay if we use the arithmetic analyzer
+    if (auto* shape_node = tensor_info->shape.as<ShapeExprNode>()) {
+      for (auto dim : shape_node->values) {
+        if (!dim.as<IntImmNode>()) {
+          return {};
+        }
+      }
+      return {GetRef<TensorStructInfo>(tensor_info)};
+    } else {
+      return {};
+    }
+  } else if (auto* tuple_info = result_sinfo.as<TupleStructInfoNode>()) {
+    // we can see if the whole tuple matches or go for any of the components
+    std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> ret;
+    for (auto field : tuple_info->fields) {
+      auto field_candidates = GatherCandidateSinfo(field);
+      ret.insert(field_candidates.begin(), field_candidates.end());
+    }
+    // at least one field should be eligible to be done in-place
+    if (!ret.empty()) {
+      ret.insert(GetRef<StructInfo>(tuple_info));
+    }
+    return ret;
+  } else {
+    // don't consider any other types
+    return {};
+  }
+}
+
+// Given the two struct info, return a pair of bools where the first element is true if
+// the two struct info are the same _size_ in memory and the second element is true
+// if the shapes match _exactly_. Performs this check recursively and ensures the
+// stated condition is true for all tensor members of the struct info (return false
+// if a single pair of corresponding tensors does not meet the condition).
+std::pair<bool, bool> SizeMatches(const StructInfo& target_info, const StructInfo& arg_info) {
+  if (target_info.as<TensorStructInfoNode>() && arg_info.as<TensorStructInfoNode>()) {
+    auto target_tensor = Downcast<TensorStructInfo>(target_info);
+    auto arg_tensor = Downcast<TensorStructInfo>(arg_info);
+    if (target_tensor->shape.defined() && target_tensor->shape.as<ShapeExprNode>() &&
+        arg_tensor->shape.defined() && arg_tensor->shape.as<ShapeExprNode>()) {
+      if (target_tensor->dtype != arg_tensor->dtype) {
+        return {false, false};
+      }
+      auto target_shape = Downcast<ShapeExpr>(target_tensor->shape);
+      auto arg_shape = Downcast<ShapeExpr>(arg_tensor->shape);
+      int target_size = ShapeSize(target_shape);
+      int arg_size = ShapeSize(arg_shape);
+      if (target_size == -1 || arg_size == -1 || target_size < arg_size) {
+        return {false, false};
+      }
+      // exact match: number of dims and each dim matches
+      if (target_shape->values.size() == arg_shape->values.size()) {
+        for (size_t i = 0; i < target_shape->values.size(); i++) {
+          if (!arg_shape->values[i].as<IntImmNode>()) {
+            return {false, false};
+          }
+          if (Downcast<IntImm>(target_shape->values[i])->value !=
+              Downcast<IntImm>(arg_shape->values[i])->value) {
+            return {true, false};
+          }
+        }
+        return {true, true};
+      }
+      return {true, false};
+    } else {
+      return {false, false};
+    }
+  } else if (target_info.as<TupleStructInfoNode>() && arg_info.as<TupleStructInfoNode>()) {
+    auto target_tup = Downcast<TupleStructInfo>(target_info);
+    auto arg_tup = Downcast<TupleStructInfo>(arg_info);
+    if (target_tup->fields.size() != arg_tup->fields.size()) {
+      return {false, false};
+    }
+    bool all_exact = true;
+    for (size_t i = 0; i < target_tup->fields.size(); i++) {
+      auto element_match = SizeMatches(target_tup->fields[i], arg_tup->fields[i]);

Review Comment:
   Structured bindings can improve readability here:
   
   ```c++
   auto [field_size_match, field_exact_match] = SizeMatches(target_tup->fields[i], arg_tup->fields[i]);
   ```



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {
+      auto target_var = GetRef<Var>(target_var_node);
+      if (alias_map_.count(target_var)) {
+        ret.insert(alias_map_[target_var].begin(), alias_map_[target_var].end());
+      } else {
+        ret.insert(-1);
+      }
+    } else if (auto* target_tuple = value.as<TupleNode>()) {
+      // fresh idx but we update the tuple map
+      int tup_idx = get_fresh_idx();
+      ret.insert(tup_idx);
+      std::vector<std::unordered_set<int>> new_tuple_map;
+      for (auto field : target_tuple->fields) {
+        new_tuple_map.push_back(GetAliasSet(field, bound_var));
+      }
+      tuple_map_[tup_idx] = new_tuple_map;
+    } else if (auto* target_tgi = value.as<TupleGetItemNode>()) {
+      std::unordered_set<int> tuple_set = GetAliasSet(target_tgi->tuple, bound_var);
+      // if -1 is a member of the tuple set, then we have to assume the result is -1
+      if (tuple_set.count(-1)) {
+        ret.insert(-1);
+      } else {
+        // otherwise, consider all members that are tuples of appropriate size and index into them
+        // (this is safe because the type system will ensure we're not indexing into a tuple
+        // of the wrong size)
+        for (int member : tuple_set) {
+          if (tuple_map_.count(member) &&
+              static_cast<int>(tuple_map_[member].size()) > target_tgi->index) {
+            auto member_set = tuple_map_[member][target_tgi->index];
+            ret.insert(member_set.begin(), member_set.end());
+          }
+        }
+      }
+    } else if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        // call_pure_packed: treat as non-op call
+        if (op_node->name == "relax.call_pure_packed") {
+          return HandleMysteryCall(call_node, bound_var, true);
+        } else if (op_node->name == "relax.split") {
+          // split: Returns a tuple, treat as allocation
+
+          // tuple is freshly allocated, but also add components to the tuple map
+          int tup_idx = get_fresh_idx();
+          ret.insert(tup_idx);
+          // the LHS (the bound var) will definitely have a tuple struct info
+          InsertFreshTuple(tup_idx, GetStructInfoAs<TupleStructInfoNode>(bound_var));
+        } else if (op_node->name == "relax.call_tir") {
+          // call_tir: can potentially return a tuple
+          if (auto* tuple_struct_info = call_node->sinfo_args[0].as<TupleStructInfoNode>()) {
+            int tup_idx = get_fresh_idx();
+            ret.insert(tup_idx);
+            InsertFreshTuple(tup_idx, tuple_struct_info);
+          } else {
+            ret.insert(get_fresh_idx());
+          }
+        } else {
+          // We are assuming most op calls return a single fresh allocation.
+          // We may have to track more exceptions
+          ret.insert(get_fresh_idx());
+        }
+      } else {
+        // assume any non-op call can be extremely dangerous and do anything
+        return HandleMysteryCall(call_node, bound_var);
+      }
+    }
+
+    return ret;
+  }
+
+  std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual> alias_map_;
+  std::unordered_map<int, std::vector<std::unordered_set<int>>> tuple_map_;
+  int mem_idx_;
+};
+
+// given a shape, return the allocation size corresponding to it (product of elements)
+int ShapeSize(const ShapeExpr& shape) {
+  int ret = 1;
+  for (auto dim : shape->values) {
+    if (auto int_dim = dim.as<IntImmNode>()) {
+      ret *= static_cast<int>(int_dim->value);
+    } else {
+      return -1;
+    }
+  }
+  return ret;
+}
+
+// Given the struct info of the result, return any struct info nested in it
+// that is eleigible to be used for in-place computations (tensors are eligible
+// only if all their dimensions are integer constants, tuples are eligible if
+// all members are eligible though we can consider only individual members separately)
+std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> GatherCandidateSinfo(
+    const StructInfo& result_sinfo) {
+  if (auto* tensor_info = result_sinfo.as<TensorStructInfoNode>()) {
+    // don't consider void dtype (don't know the size at compile time)
+    if (tensor_info->dtype.is_void()) {
+      return {};
+    }
+    // don't consider cases where we don't know the shape at compile time
+    // TODO(@slyubomirsky): variables might be okay if we use the arithmetic analyzer
+    if (auto* shape_node = tensor_info->shape.as<ShapeExprNode>()) {
+      for (auto dim : shape_node->values) {
+        if (!dim.as<IntImmNode>()) {
+          return {};
+        }
+      }
+      return {GetRef<TensorStructInfo>(tensor_info)};
+    } else {
+      return {};
+    }
+  } else if (auto* tuple_info = result_sinfo.as<TupleStructInfoNode>()) {
+    // we can see if the whole tuple matches or go for any of the components
+    std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> ret;
+    for (auto field : tuple_info->fields) {
+      auto field_candidates = GatherCandidateSinfo(field);
+      ret.insert(field_candidates.begin(), field_candidates.end());
+    }
+    // at least one field should be eligible to be done in-place
+    if (!ret.empty()) {
+      ret.insert(GetRef<StructInfo>(tuple_info));
+    }
+    return ret;
+  } else {
+    // don't consider any other types
+    return {};
+  }
+}
+
+// Given the two struct info, return a pair of bools where the first element is true if
+// the two struct info are the same _size_ in memory and the second element is true

Review Comment:
   Nitpick: Change from "same size" to "same dtype and same number of elements".



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {
+      auto target_var = GetRef<Var>(target_var_node);
+      if (alias_map_.count(target_var)) {
+        ret.insert(alias_map_[target_var].begin(), alias_map_[target_var].end());
+      } else {
+        ret.insert(-1);
+      }
+    } else if (auto* target_tuple = value.as<TupleNode>()) {
+      // fresh idx but we update the tuple map
+      int tup_idx = get_fresh_idx();
+      ret.insert(tup_idx);
+      std::vector<std::unordered_set<int>> new_tuple_map;
+      for (auto field : target_tuple->fields) {
+        new_tuple_map.push_back(GetAliasSet(field, bound_var));
+      }
+      tuple_map_[tup_idx] = new_tuple_map;
+    } else if (auto* target_tgi = value.as<TupleGetItemNode>()) {
+      std::unordered_set<int> tuple_set = GetAliasSet(target_tgi->tuple, bound_var);
+      // if -1 is a member of the tuple set, then we have to assume the result is -1
+      if (tuple_set.count(-1)) {
+        ret.insert(-1);
+      } else {
+        // otherwise, consider all members that are tuples of appropriate size and index into them
+        // (this is safe because the type system will ensure we're not indexing into a tuple
+        // of the wrong size)
+        for (int member : tuple_set) {
+          if (tuple_map_.count(member) &&
+              static_cast<int>(tuple_map_[member].size()) > target_tgi->index) {
+            auto member_set = tuple_map_[member][target_tgi->index];
+            ret.insert(member_set.begin(), member_set.end());
+          }
+        }
+      }
+    } else if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        // call_pure_packed: treat as non-op call
+        if (op_node->name == "relax.call_pure_packed") {
+          return HandleMysteryCall(call_node, bound_var, true);
+        } else if (op_node->name == "relax.split") {

Review Comment:
   Instead of checking specific operators that may return tuples, can we just check whether the operator returns `TupleStructInfo`?



##########
python/tvm/relax/testing/transform.py:
##########
@@ -128,3 +129,62 @@ def transform(self):
 def ApplyEmptyCppMutator() -> tvm.ir.transform.Pass:
     packed_func = tvm.get_global_func("relax.testing.transform.ApplyEmptyCppMutator")
     return packed_func()
+
+
+def dataflow_liveness_analysis(block: DataflowBlock) -> Dict[Var, Tuple[int, int]]:
+    """
+    Inner function for the dataflow inplace transformation exposed for testing.
+    """
+    live_ranges = tvm.get_global_func("relax.testing.transform.DataflowLivenessAnalysis")(

Review Comment:
   I like that the functions are only exposed here for testing.  The `Tuple[int, int]` in the return type would be a bit opaque for exposure otherwise, I think.
   
   Could we add a warning if these are used outside of a testing environment?  The `relax.testing` namespace has a few utilities that may be used outside of testing, (e.g. the old version of `nn.Module`), so it would be good to make sure users are aware.
   
   ```python
   if "PYTEST_CURRENT_TEST" not in os.environ:
       logging.warning(
           "The function $FUNC_NAME is exposed for testing only."
       )
   ```



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {
+      auto target_var = GetRef<Var>(target_var_node);
+      if (alias_map_.count(target_var)) {
+        ret.insert(alias_map_[target_var].begin(), alias_map_[target_var].end());
+      } else {
+        ret.insert(-1);
+      }
+    } else if (auto* target_tuple = value.as<TupleNode>()) {
+      // fresh idx but we update the tuple map
+      int tup_idx = get_fresh_idx();
+      ret.insert(tup_idx);
+      std::vector<std::unordered_set<int>> new_tuple_map;
+      for (auto field : target_tuple->fields) {
+        new_tuple_map.push_back(GetAliasSet(field, bound_var));
+      }
+      tuple_map_[tup_idx] = new_tuple_map;
+    } else if (auto* target_tgi = value.as<TupleGetItemNode>()) {
+      std::unordered_set<int> tuple_set = GetAliasSet(target_tgi->tuple, bound_var);
+      // if -1 is a member of the tuple set, then we have to assume the result is -1
+      if (tuple_set.count(-1)) {
+        ret.insert(-1);
+      } else {
+        // otherwise, consider all members that are tuples of appropriate size and index into them
+        // (this is safe because the type system will ensure we're not indexing into a tuple
+        // of the wrong size)
+        for (int member : tuple_set) {
+          if (tuple_map_.count(member) &&
+              static_cast<int>(tuple_map_[member].size()) > target_tgi->index) {
+            auto member_set = tuple_map_[member][target_tgi->index];
+            ret.insert(member_set.begin(), member_set.end());
+          }
+        }
+      }
+    } else if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        // call_pure_packed: treat as non-op call
+        if (op_node->name == "relax.call_pure_packed") {
+          return HandleMysteryCall(call_node, bound_var, true);
+        } else if (op_node->name == "relax.split") {
+          // split: Returns a tuple, treat as allocation
+
+          // tuple is freshly allocated, but also add components to the tuple map
+          int tup_idx = get_fresh_idx();
+          ret.insert(tup_idx);
+          // the LHS (the bound var) will definitely have a tuple struct info
+          InsertFreshTuple(tup_idx, GetStructInfoAs<TupleStructInfoNode>(bound_var));
+        } else if (op_node->name == "relax.call_tir") {
+          // call_tir: can potentially return a tuple
+          if (auto* tuple_struct_info = call_node->sinfo_args[0].as<TupleStructInfoNode>()) {
+            int tup_idx = get_fresh_idx();
+            ret.insert(tup_idx);
+            InsertFreshTuple(tup_idx, tuple_struct_info);
+          } else {
+            ret.insert(get_fresh_idx());
+          }
+        } else {
+          // We are assuming most op calls return a single fresh allocation.
+          // We may have to track more exceptions
+          ret.insert(get_fresh_idx());
+        }
+      } else {
+        // assume any non-op call can be extremely dangerous and do anything
+        return HandleMysteryCall(call_node, bound_var);
+      }
+    }
+
+    return ret;
+  }
+
+  std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual> alias_map_;
+  std::unordered_map<int, std::vector<std::unordered_set<int>>> tuple_map_;
+  int mem_idx_;
+};
+
+// given a shape, return the allocation size corresponding to it (product of elements)
+int ShapeSize(const ShapeExpr& shape) {
+  int ret = 1;
+  for (auto dim : shape->values) {
+    if (auto int_dim = dim.as<IntImmNode>()) {
+      ret *= static_cast<int>(int_dim->value);
+    } else {
+      return -1;
+    }
+  }
+  return ret;
+}
+
+// Given the struct info of the result, return any struct info nested in it
+// that is eleigible to be used for in-place computations (tensors are eligible
+// only if all their dimensions are integer constants, tuples are eligible if
+// all members are eligible though we can consider only individual members separately)
+std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> GatherCandidateSinfo(
+    const StructInfo& result_sinfo) {
+  if (auto* tensor_info = result_sinfo.as<TensorStructInfoNode>()) {
+    // don't consider void dtype (don't know the size at compile time)
+    if (tensor_info->dtype.is_void()) {
+      return {};
+    }
+    // don't consider cases where we don't know the shape at compile time
+    // TODO(@slyubomirsky): variables might be okay if we use the arithmetic analyzer
+    if (auto* shape_node = tensor_info->shape.as<ShapeExprNode>()) {
+      for (auto dim : shape_node->values) {
+        if (!dim.as<IntImmNode>()) {
+          return {};
+        }
+      }
+      return {GetRef<TensorStructInfo>(tensor_info)};
+    } else {
+      return {};
+    }
+  } else if (auto* tuple_info = result_sinfo.as<TupleStructInfoNode>()) {
+    // we can see if the whole tuple matches or go for any of the components
+    std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> ret;
+    for (auto field : tuple_info->fields) {
+      auto field_candidates = GatherCandidateSinfo(field);
+      ret.insert(field_candidates.begin(), field_candidates.end());
+    }
+    // at least one field should be eligible to be done in-place
+    if (!ret.empty()) {
+      ret.insert(GetRef<StructInfo>(tuple_info));
+    }
+    return ret;
+  } else {
+    // don't consider any other types
+    return {};
+  }
+}
+
+// Given the two struct info, return a pair of bools where the first element is true if
+// the two struct info are the same _size_ in memory and the second element is true
+// if the shapes match _exactly_. Performs this check recursively and ensures the
+// stated condition is true for all tensor members of the struct info (return false
+// if a single pair of corresponding tensors does not meet the condition).
+std::pair<bool, bool> SizeMatches(const StructInfo& target_info, const StructInfo& arg_info) {
+  if (target_info.as<TensorStructInfoNode>() && arg_info.as<TensorStructInfoNode>()) {
+    auto target_tensor = Downcast<TensorStructInfo>(target_info);
+    auto arg_tensor = Downcast<TensorStructInfo>(arg_info);
+    if (target_tensor->shape.defined() && target_tensor->shape.as<ShapeExprNode>() &&
+        arg_tensor->shape.defined() && arg_tensor->shape.as<ShapeExprNode>()) {
+      if (target_tensor->dtype != arg_tensor->dtype) {
+        return {false, false};
+      }
+      auto target_shape = Downcast<ShapeExpr>(target_tensor->shape);
+      auto arg_shape = Downcast<ShapeExpr>(arg_tensor->shape);
+      int target_size = ShapeSize(target_shape);
+      int arg_size = ShapeSize(arg_shape);
+      if (target_size == -1 || arg_size == -1 || target_size < arg_size) {
+        return {false, false};
+      }
+      // exact match: number of dims and each dim matches
+      if (target_shape->values.size() == arg_shape->values.size()) {
+        for (size_t i = 0; i < target_shape->values.size(); i++) {
+          if (!arg_shape->values[i].as<IntImmNode>()) {
+            return {false, false};
+          }
+          if (Downcast<IntImm>(target_shape->values[i])->value !=
+              Downcast<IntImm>(arg_shape->values[i])->value) {
+            return {true, false};
+          }
+        }
+        return {true, true};
+      }
+      return {true, false};
+    } else {
+      return {false, false};
+    }
+  } else if (target_info.as<TupleStructInfoNode>() && arg_info.as<TupleStructInfoNode>()) {
+    auto target_tup = Downcast<TupleStructInfo>(target_info);
+    auto arg_tup = Downcast<TupleStructInfo>(arg_info);
+    if (target_tup->fields.size() != arg_tup->fields.size()) {
+      return {false, false};
+    }
+    bool all_exact = true;
+    for (size_t i = 0; i < target_tup->fields.size(); i++) {
+      auto element_match = SizeMatches(target_tup->fields[i], arg_tup->fields[i]);
+      if (!element_match.first) {
+        return {false, false};
+      }
+      if (!element_match.second) {
+        all_exact = false;
+      }
+    }
+    return {true, all_exact};
+  } else if (target_info.as<PrimStructInfoNode>() && arg_info.as<PrimStructInfoNode>()) {

Review Comment:
   This branch would only be used for tuples that contain some tensors (might be in-place) and some primvalues (passed by value, irrelevant), correct?  Can we add a comment to call it out?  Otherwise, this branch reads like we might do an in-place update of a PrimValue.



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {
+      auto target_var = GetRef<Var>(target_var_node);
+      if (alias_map_.count(target_var)) {
+        ret.insert(alias_map_[target_var].begin(), alias_map_[target_var].end());
+      } else {
+        ret.insert(-1);
+      }
+    } else if (auto* target_tuple = value.as<TupleNode>()) {
+      // fresh idx but we update the tuple map
+      int tup_idx = get_fresh_idx();
+      ret.insert(tup_idx);
+      std::vector<std::unordered_set<int>> new_tuple_map;
+      for (auto field : target_tuple->fields) {
+        new_tuple_map.push_back(GetAliasSet(field, bound_var));
+      }
+      tuple_map_[tup_idx] = new_tuple_map;
+    } else if (auto* target_tgi = value.as<TupleGetItemNode>()) {
+      std::unordered_set<int> tuple_set = GetAliasSet(target_tgi->tuple, bound_var);
+      // if -1 is a member of the tuple set, then we have to assume the result is -1
+      if (tuple_set.count(-1)) {
+        ret.insert(-1);
+      } else {
+        // otherwise, consider all members that are tuples of appropriate size and index into them
+        // (this is safe because the type system will ensure we're not indexing into a tuple
+        // of the wrong size)
+        for (int member : tuple_set) {
+          if (tuple_map_.count(member) &&
+              static_cast<int>(tuple_map_[member].size()) > target_tgi->index) {
+            auto member_set = tuple_map_[member][target_tgi->index];
+            ret.insert(member_set.begin(), member_set.end());
+          }
+        }
+      }
+    } else if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        // call_pure_packed: treat as non-op call
+        if (op_node->name == "relax.call_pure_packed") {
+          return HandleMysteryCall(call_node, bound_var, true);
+        } else if (op_node->name == "relax.split") {
+          // split: Returns a tuple, treat as allocation
+
+          // tuple is freshly allocated, but also add components to the tuple map
+          int tup_idx = get_fresh_idx();
+          ret.insert(tup_idx);
+          // the LHS (the bound var) will definitely have a tuple struct info
+          InsertFreshTuple(tup_idx, GetStructInfoAs<TupleStructInfoNode>(bound_var));
+        } else if (op_node->name == "relax.call_tir") {
+          // call_tir: can potentially return a tuple
+          if (auto* tuple_struct_info = call_node->sinfo_args[0].as<TupleStructInfoNode>()) {
+            int tup_idx = get_fresh_idx();
+            ret.insert(tup_idx);
+            InsertFreshTuple(tup_idx, tuple_struct_info);
+          } else {
+            ret.insert(get_fresh_idx());
+          }
+        } else {
+          // We are assuming most op calls return a single fresh allocation.
+          // We may have to track more exceptions
+          ret.insert(get_fresh_idx());
+        }
+      } else {
+        // assume any non-op call can be extremely dangerous and do anything
+        return HandleMysteryCall(call_node, bound_var);
+      }
+    }
+
+    return ret;
+  }
+
+  std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual> alias_map_;
+  std::unordered_map<int, std::vector<std::unordered_set<int>>> tuple_map_;
+  int mem_idx_;
+};
+
+// given a shape, return the allocation size corresponding to it (product of elements)
+int ShapeSize(const ShapeExpr& shape) {
+  int ret = 1;
+  for (auto dim : shape->values) {
+    if (auto int_dim = dim.as<IntImmNode>()) {
+      ret *= static_cast<int>(int_dim->value);
+    } else {
+      return -1;
+    }
+  }
+  return ret;
+}
+
+// Given the struct info of the result, return any struct info nested in it
+// that is eleigible to be used for in-place computations (tensors are eligible
+// only if all their dimensions are integer constants, tuples are eligible if
+// all members are eligible though we can consider only individual members separately)
+std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> GatherCandidateSinfo(
+    const StructInfo& result_sinfo) {
+  if (auto* tensor_info = result_sinfo.as<TensorStructInfoNode>()) {
+    // don't consider void dtype (don't know the size at compile time)
+    if (tensor_info->dtype.is_void()) {
+      return {};
+    }
+    // don't consider cases where we don't know the shape at compile time
+    // TODO(@slyubomirsky): variables might be okay if we use the arithmetic analyzer
+    if (auto* shape_node = tensor_info->shape.as<ShapeExprNode>()) {
+      for (auto dim : shape_node->values) {
+        if (!dim.as<IntImmNode>()) {
+          return {};
+        }
+      }
+      return {GetRef<TensorStructInfo>(tensor_info)};
+    } else {
+      return {};
+    }
+  } else if (auto* tuple_info = result_sinfo.as<TupleStructInfoNode>()) {
+    // we can see if the whole tuple matches or go for any of the components
+    std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> ret;
+    for (auto field : tuple_info->fields) {
+      auto field_candidates = GatherCandidateSinfo(field);
+      ret.insert(field_candidates.begin(), field_candidates.end());
+    }
+    // at least one field should be eligible to be done in-place
+    if (!ret.empty()) {
+      ret.insert(GetRef<StructInfo>(tuple_info));
+    }
+    return ret;
+  } else {
+    // don't consider any other types
+    return {};
+  }
+}
+
+// Given the two struct info, return a pair of bools where the first element is true if
+// the two struct info are the same _size_ in memory and the second element is true
+// if the shapes match _exactly_. Performs this check recursively and ensures the
+// stated condition is true for all tensor members of the struct info (return false
+// if a single pair of corresponding tensors does not meet the condition).
+std::pair<bool, bool> SizeMatches(const StructInfo& target_info, const StructInfo& arg_info) {
+  if (target_info.as<TensorStructInfoNode>() && arg_info.as<TensorStructInfoNode>()) {
+    auto target_tensor = Downcast<TensorStructInfo>(target_info);
+    auto arg_tensor = Downcast<TensorStructInfo>(arg_info);
+    if (target_tensor->shape.defined() && target_tensor->shape.as<ShapeExprNode>() &&
+        arg_tensor->shape.defined() && arg_tensor->shape.as<ShapeExprNode>()) {
+      if (target_tensor->dtype != arg_tensor->dtype) {
+        return {false, false};
+      }
+      auto target_shape = Downcast<ShapeExpr>(target_tensor->shape);
+      auto arg_shape = Downcast<ShapeExpr>(arg_tensor->shape);
+      int target_size = ShapeSize(target_shape);
+      int arg_size = ShapeSize(arg_shape);
+      if (target_size == -1 || arg_size == -1 || target_size < arg_size) {
+        return {false, false};
+      }
+      // exact match: number of dims and each dim matches
+      if (target_shape->values.size() == arg_shape->values.size()) {
+        for (size_t i = 0; i < target_shape->values.size(); i++) {
+          if (!arg_shape->values[i].as<IntImmNode>()) {
+            return {false, false};
+          }
+          if (Downcast<IntImm>(target_shape->values[i])->value !=
+              Downcast<IntImm>(arg_shape->values[i])->value) {
+            return {true, false};
+          }
+        }
+        return {true, true};
+      }
+      return {true, false};
+    } else {
+      return {false, false};
+    }
+  } else if (target_info.as<TupleStructInfoNode>() && arg_info.as<TupleStructInfoNode>()) {
+    auto target_tup = Downcast<TupleStructInfo>(target_info);
+    auto arg_tup = Downcast<TupleStructInfo>(arg_info);
+    if (target_tup->fields.size() != arg_tup->fields.size()) {
+      return {false, false};
+    }
+    bool all_exact = true;
+    for (size_t i = 0; i < target_tup->fields.size(); i++) {
+      auto element_match = SizeMatches(target_tup->fields[i], arg_tup->fields[i]);
+      if (!element_match.first) {
+        return {false, false};
+      }
+      if (!element_match.second) {
+        all_exact = false;
+      }
+    }
+    return {true, all_exact};
+  } else if (target_info.as<PrimStructInfoNode>() && arg_info.as<PrimStructInfoNode>()) {
+    return {true, true};
+  } else {
+    return {false, false};
+  }
+}
+
+// Given an alias index, check if it's a tuple and gather the sets of aliases for the tuple
+// members if so (apply recursively if any of those members are tuples).
+// Return false if the alias set contains -1, meaning a reference to an unknown or
+// possibly dangerous value (no checking we can do for that).
+bool GatherSetsToCheckForLiveness(
+    const std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>&
+        alias_sets,
+    const std::unordered_map<int, std::vector<std::unordered_set<int>>>& tuple_map,
+    std::vector<std::unordered_set<int>>* sets_to_check, int alias_idx) {
+  if (tuple_map.count(alias_idx)) {
+    for (auto member_set : tuple_map.at(alias_idx)) {
+      // contains -1 -> unknown and dangerous, we can short-circuit
+      if (member_set.count(-1)) {
+        return false;
+      }
+      sets_to_check->push_back(member_set);
+
+      // if a member can be a tuple, check it recursively
+      for (int member : member_set) {
+        if (tuple_map.count(member)) {
+          if (!GatherSetsToCheckForLiveness(alias_sets, tuple_map, sets_to_check, member)) {
+            return false;
+          }
+        }
+      }
+    }
+  }
+  return true;
+}
+
+// Check that the target is not live past the index and that no alias of it is live past the
+// binding index (if the target is a tuple, check the conditions recursively for the members)
+bool InplaceConditionsMet(
+    const std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual>& live_ranges,
+    const std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>&
+        alias_sets,
+    const std::unordered_map<int, std::vector<std::unordered_set<int>>>& tuple_map,
+    const std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual>& currently_live,
+    const Expr& target, int binding_idx) {
+  if (auto* var_node = target.as<VarNode>()) {
+    auto current_var = GetRef<Var>(var_node);
+    // if the var is live past this point, we can't use it for in-place computations anyway
+    if (live_ranges.count(current_var)) {
+      auto live_range = live_ranges.at(current_var);
+      if (live_range.second > binding_idx) {
+        return false;
+      }
+    }
+
+    // no entry for the current var -> it must be something external and we have to assume the worst
+    if (!alias_sets.count(current_var)) {
+      return false;
+    }
+    auto alias_set = alias_sets.at(current_var);
+    // -1 -> an external value and we must assume the worst
+    if (alias_set.count(-1)) {
+      return false;
+    }
+    std::vector<std::unordered_set<int>> sets_to_check = {alias_set};
+    std::unordered_set<int> indices_checked;
+    // If a possible alias is a tuple, we will also check for aliases of the members
+    // (possibly recursively)
+    for (int alias_idx : alias_set) {
+      if (!GatherSetsToCheckForLiveness(alias_sets, tuple_map, &sets_to_check, alias_idx)) {
+        return false;
+      }
+    }
+
+    for (Var other_var : currently_live) {
+      if (other_var.same_as(target)) {
+        continue;
+      }
+      // not represented = spooky unknown value that should be modeled by -1
+      if (!alias_sets.count(other_var) || !live_ranges.count(other_var)) {
+        continue;
+      }
+      // var is not live past this point => don't need to worry
+      if (live_ranges.at(other_var).second <= binding_idx) {
+        continue;
+      }
+      auto other_alias_set = alias_sets.at(other_var);
+      for (int alias_idx : other_alias_set) {
+        for (auto check_set : sets_to_check) {
+          if (check_set.count(alias_idx)) {
+            return false;
+          }
+        }
+      }
+    }
+    return true;
+  } else if (auto* tup_node = target.as<TupleNode>()) {
+    for (auto field : tup_node->fields) {
+      if (!InplaceConditionsMet(live_ranges, alias_sets, tuple_map, currently_live, field,
+                                binding_idx)) {
+        return false;
+      }
+    }
+    return true;
+  } else {
+    return true;
+  }
+}
+
+// this is obviously not a complete list
+static std::unordered_set<std::string> SUPPORTED_OPS = {"relax.add",      "relax.subtract",
+                                                        "relax.multiply", "relax.divide",
+                                                        "relax.nn.silu",  "relax.nn.relu"};
+bool OpSupportsInplace(const Op& op) { return SUPPORTED_OPS.count(op->name); }
+
+// Check for in-place eligibility:
+//  1. see if there's an arg big enough to hold the result
+//  2. see if the arg is live past the call
+//  3. see if the arg has an alias that's live past the call
+// If the conditions are met, record the index of that binding.
+// Returns two lists of lists:
+// 1. A list of bindings where at least one argument meets the in-place conditions and the *size*

Review Comment:
   What is the nested layer of lists specifying?  From the description, I would expect `std::vector<int>` as "a list of bindings", not a `std::vector<std::vector<int>>`.
   
   Instead of having `std::pair<std::vector<std::vector<int>>, std::vector<std::vector<int>>>`, could we change to named structs for readability?  I'm thinking something like the following.
   
   ```c++
   struct InplaceOpportunity {
     // Argument indices for which the argument is the exact same size as
     // the output. (e.g. The output of `R.add(A,B)`, when neither
     // argument is broadcasted.)
     std::vector<int> shape_match;
   
     // Argument indices, where the argument is the same size as the
     // output, but has a different shape (e.g. A fused
     // `R.reshape(R.add(A,B), new_shape)`).
     std::vector<int> size_match;
   };
   
   std::unordered_map<const VarNode*, InplaceOpportunity> FindInplaceOpportunities(...) { ... }
   ```



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {
+      auto target_var = GetRef<Var>(target_var_node);
+      if (alias_map_.count(target_var)) {
+        ret.insert(alias_map_[target_var].begin(), alias_map_[target_var].end());
+      } else {
+        ret.insert(-1);
+      }
+    } else if (auto* target_tuple = value.as<TupleNode>()) {
+      // fresh idx but we update the tuple map
+      int tup_idx = get_fresh_idx();
+      ret.insert(tup_idx);
+      std::vector<std::unordered_set<int>> new_tuple_map;
+      for (auto field : target_tuple->fields) {
+        new_tuple_map.push_back(GetAliasSet(field, bound_var));
+      }
+      tuple_map_[tup_idx] = new_tuple_map;
+    } else if (auto* target_tgi = value.as<TupleGetItemNode>()) {
+      std::unordered_set<int> tuple_set = GetAliasSet(target_tgi->tuple, bound_var);
+      // if -1 is a member of the tuple set, then we have to assume the result is -1
+      if (tuple_set.count(-1)) {
+        ret.insert(-1);
+      } else {
+        // otherwise, consider all members that are tuples of appropriate size and index into them
+        // (this is safe because the type system will ensure we're not indexing into a tuple
+        // of the wrong size)
+        for (int member : tuple_set) {
+          if (tuple_map_.count(member) &&
+              static_cast<int>(tuple_map_[member].size()) > target_tgi->index) {
+            auto member_set = tuple_map_[member][target_tgi->index];
+            ret.insert(member_set.begin(), member_set.end());
+          }
+        }
+      }
+    } else if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        // call_pure_packed: treat as non-op call
+        if (op_node->name == "relax.call_pure_packed") {
+          return HandleMysteryCall(call_node, bound_var, true);
+        } else if (op_node->name == "relax.split") {
+          // split: Returns a tuple, treat as allocation
+
+          // tuple is freshly allocated, but also add components to the tuple map
+          int tup_idx = get_fresh_idx();
+          ret.insert(tup_idx);
+          // the LHS (the bound var) will definitely have a tuple struct info
+          InsertFreshTuple(tup_idx, GetStructInfoAs<TupleStructInfoNode>(bound_var));
+        } else if (op_node->name == "relax.call_tir") {
+          // call_tir: can potentially return a tuple
+          if (auto* tuple_struct_info = call_node->sinfo_args[0].as<TupleStructInfoNode>()) {
+            int tup_idx = get_fresh_idx();
+            ret.insert(tup_idx);
+            InsertFreshTuple(tup_idx, tuple_struct_info);
+          } else {
+            ret.insert(get_fresh_idx());
+          }
+        } else {
+          // We are assuming most op calls return a single fresh allocation.
+          // We may have to track more exceptions
+          ret.insert(get_fresh_idx());
+        }
+      } else {
+        // assume any non-op call can be extremely dangerous and do anything
+        return HandleMysteryCall(call_node, bound_var);
+      }
+    }
+
+    return ret;
+  }
+
+  std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual> alias_map_;
+  std::unordered_map<int, std::vector<std::unordered_set<int>>> tuple_map_;
+  int mem_idx_;
+};
+
+// given a shape, return the allocation size corresponding to it (product of elements)
+int ShapeSize(const ShapeExpr& shape) {
+  int ret = 1;
+  for (auto dim : shape->values) {
+    if (auto int_dim = dim.as<IntImmNode>()) {
+      ret *= static_cast<int>(int_dim->value);
+    } else {
+      return -1;
+    }
+  }
+  return ret;
+}
+
+// Given the struct info of the result, return any struct info nested in it
+// that is eleigible to be used for in-place computations (tensors are eligible
+// only if all their dimensions are integer constants, tuples are eligible if
+// all members are eligible though we can consider only individual members separately)
+std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> GatherCandidateSinfo(
+    const StructInfo& result_sinfo) {
+  if (auto* tensor_info = result_sinfo.as<TensorStructInfoNode>()) {
+    // don't consider void dtype (don't know the size at compile time)
+    if (tensor_info->dtype.is_void()) {
+      return {};
+    }
+    // don't consider cases where we don't know the shape at compile time
+    // TODO(@slyubomirsky): variables might be okay if we use the arithmetic analyzer
+    if (auto* shape_node = tensor_info->shape.as<ShapeExprNode>()) {
+      for (auto dim : shape_node->values) {
+        if (!dim.as<IntImmNode>()) {
+          return {};
+        }
+      }
+      return {GetRef<TensorStructInfo>(tensor_info)};
+    } else {
+      return {};
+    }
+  } else if (auto* tuple_info = result_sinfo.as<TupleStructInfoNode>()) {
+    // we can see if the whole tuple matches or go for any of the components
+    std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> ret;
+    for (auto field : tuple_info->fields) {
+      auto field_candidates = GatherCandidateSinfo(field);
+      ret.insert(field_candidates.begin(), field_candidates.end());
+    }
+    // at least one field should be eligible to be done in-place
+    if (!ret.empty()) {
+      ret.insert(GetRef<StructInfo>(tuple_info));
+    }
+    return ret;
+  } else {
+    // don't consider any other types
+    return {};
+  }
+}
+
+// Given the two struct info, return a pair of bools where the first element is true if
+// the two struct info are the same _size_ in memory and the second element is true
+// if the shapes match _exactly_. Performs this check recursively and ensures the
+// stated condition is true for all tensor members of the struct info (return false
+// if a single pair of corresponding tensors does not meet the condition).
+std::pair<bool, bool> SizeMatches(const StructInfo& target_info, const StructInfo& arg_info) {
+  if (target_info.as<TensorStructInfoNode>() && arg_info.as<TensorStructInfoNode>()) {
+    auto target_tensor = Downcast<TensorStructInfo>(target_info);
+    auto arg_tensor = Downcast<TensorStructInfo>(arg_info);
+    if (target_tensor->shape.defined() && target_tensor->shape.as<ShapeExprNode>() &&
+        arg_tensor->shape.defined() && arg_tensor->shape.as<ShapeExprNode>()) {
+      if (target_tensor->dtype != arg_tensor->dtype) {
+        return {false, false};
+      }
+      auto target_shape = Downcast<ShapeExpr>(target_tensor->shape);
+      auto arg_shape = Downcast<ShapeExpr>(arg_tensor->shape);
+      int target_size = ShapeSize(target_shape);
+      int arg_size = ShapeSize(arg_shape);
+      if (target_size == -1 || arg_size == -1 || target_size < arg_size) {
+        return {false, false};
+      }
+      // exact match: number of dims and each dim matches
+      if (target_shape->values.size() == arg_shape->values.size()) {
+        for (size_t i = 0; i < target_shape->values.size(); i++) {
+          if (!arg_shape->values[i].as<IntImmNode>()) {
+            return {false, false};
+          }
+          if (Downcast<IntImm>(target_shape->values[i])->value !=
+              Downcast<IntImm>(arg_shape->values[i])->value) {
+            return {true, false};
+          }
+        }
+        return {true, true};
+      }
+      return {true, false};
+    } else {
+      return {false, false};
+    }
+  } else if (target_info.as<TupleStructInfoNode>() && arg_info.as<TupleStructInfoNode>()) {
+    auto target_tup = Downcast<TupleStructInfo>(target_info);
+    auto arg_tup = Downcast<TupleStructInfo>(arg_info);
+    if (target_tup->fields.size() != arg_tup->fields.size()) {
+      return {false, false};
+    }
+    bool all_exact = true;
+    for (size_t i = 0; i < target_tup->fields.size(); i++) {
+      auto element_match = SizeMatches(target_tup->fields[i], arg_tup->fields[i]);
+      if (!element_match.first) {
+        return {false, false};
+      }
+      if (!element_match.second) {

Review Comment:
   It's mostly a stylistic difference, but I think this conditional becomes easier to read when written as an expression.
   
   ```c++
   all_exact = all_exact && field_exact_match;
   ```



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {
+      auto target_var = GetRef<Var>(target_var_node);
+      if (alias_map_.count(target_var)) {
+        ret.insert(alias_map_[target_var].begin(), alias_map_[target_var].end());
+      } else {
+        ret.insert(-1);
+      }
+    } else if (auto* target_tuple = value.as<TupleNode>()) {
+      // fresh idx but we update the tuple map
+      int tup_idx = get_fresh_idx();
+      ret.insert(tup_idx);
+      std::vector<std::unordered_set<int>> new_tuple_map;
+      for (auto field : target_tuple->fields) {
+        new_tuple_map.push_back(GetAliasSet(field, bound_var));
+      }
+      tuple_map_[tup_idx] = new_tuple_map;
+    } else if (auto* target_tgi = value.as<TupleGetItemNode>()) {
+      std::unordered_set<int> tuple_set = GetAliasSet(target_tgi->tuple, bound_var);
+      // if -1 is a member of the tuple set, then we have to assume the result is -1
+      if (tuple_set.count(-1)) {
+        ret.insert(-1);
+      } else {
+        // otherwise, consider all members that are tuples of appropriate size and index into them
+        // (this is safe because the type system will ensure we're not indexing into a tuple
+        // of the wrong size)
+        for (int member : tuple_set) {
+          if (tuple_map_.count(member) &&
+              static_cast<int>(tuple_map_[member].size()) > target_tgi->index) {
+            auto member_set = tuple_map_[member][target_tgi->index];
+            ret.insert(member_set.begin(), member_set.end());
+          }
+        }
+      }
+    } else if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        // call_pure_packed: treat as non-op call
+        if (op_node->name == "relax.call_pure_packed") {
+          return HandleMysteryCall(call_node, bound_var, true);
+        } else if (op_node->name == "relax.split") {
+          // split: Returns a tuple, treat as allocation
+
+          // tuple is freshly allocated, but also add components to the tuple map
+          int tup_idx = get_fresh_idx();
+          ret.insert(tup_idx);
+          // the LHS (the bound var) will definitely have a tuple struct info
+          InsertFreshTuple(tup_idx, GetStructInfoAs<TupleStructInfoNode>(bound_var));
+        } else if (op_node->name == "relax.call_tir") {
+          // call_tir: can potentially return a tuple
+          if (auto* tuple_struct_info = call_node->sinfo_args[0].as<TupleStructInfoNode>()) {
+            int tup_idx = get_fresh_idx();
+            ret.insert(tup_idx);
+            InsertFreshTuple(tup_idx, tuple_struct_info);
+          } else {
+            ret.insert(get_fresh_idx());
+          }
+        } else {
+          // We are assuming most op calls return a single fresh allocation.
+          // We may have to track more exceptions
+          ret.insert(get_fresh_idx());
+        }
+      } else {
+        // assume any non-op call can be extremely dangerous and do anything
+        return HandleMysteryCall(call_node, bound_var);
+      }
+    }
+
+    return ret;
+  }
+
+  std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual> alias_map_;
+  std::unordered_map<int, std::vector<std::unordered_set<int>>> tuple_map_;
+  int mem_idx_;
+};
+
+// given a shape, return the allocation size corresponding to it (product of elements)
+int ShapeSize(const ShapeExpr& shape) {
+  int ret = 1;
+  for (auto dim : shape->values) {
+    if (auto int_dim = dim.as<IntImmNode>()) {
+      ret *= static_cast<int>(int_dim->value);
+    } else {
+      return -1;
+    }
+  }
+  return ret;
+}
+
+// Given the struct info of the result, return any struct info nested in it
+// that is eleigible to be used for in-place computations (tensors are eligible
+// only if all their dimensions are integer constants, tuples are eligible if
+// all members are eligible though we can consider only individual members separately)
+std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> GatherCandidateSinfo(
+    const StructInfo& result_sinfo) {
+  if (auto* tensor_info = result_sinfo.as<TensorStructInfoNode>()) {
+    // don't consider void dtype (don't know the size at compile time)
+    if (tensor_info->dtype.is_void()) {
+      return {};
+    }
+    // don't consider cases where we don't know the shape at compile time
+    // TODO(@slyubomirsky): variables might be okay if we use the arithmetic analyzer
+    if (auto* shape_node = tensor_info->shape.as<ShapeExprNode>()) {
+      for (auto dim : shape_node->values) {
+        if (!dim.as<IntImmNode>()) {
+          return {};
+        }
+      }
+      return {GetRef<TensorStructInfo>(tensor_info)};
+    } else {
+      return {};
+    }
+  } else if (auto* tuple_info = result_sinfo.as<TupleStructInfoNode>()) {
+    // we can see if the whole tuple matches or go for any of the components
+    std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> ret;
+    for (auto field : tuple_info->fields) {
+      auto field_candidates = GatherCandidateSinfo(field);
+      ret.insert(field_candidates.begin(), field_candidates.end());
+    }
+    // at least one field should be eligible to be done in-place
+    if (!ret.empty()) {
+      ret.insert(GetRef<StructInfo>(tuple_info));
+    }
+    return ret;
+  } else {
+    // don't consider any other types
+    return {};
+  }
+}
+
+// Given the two struct info, return a pair of bools where the first element is true if
+// the two struct info are the same _size_ in memory and the second element is true
+// if the shapes match _exactly_. Performs this check recursively and ensures the
+// stated condition is true for all tensor members of the struct info (return false
+// if a single pair of corresponding tensors does not meet the condition).
+std::pair<bool, bool> SizeMatches(const StructInfo& target_info, const StructInfo& arg_info) {
+  if (target_info.as<TensorStructInfoNode>() && arg_info.as<TensorStructInfoNode>()) {
+    auto target_tensor = Downcast<TensorStructInfo>(target_info);
+    auto arg_tensor = Downcast<TensorStructInfo>(arg_info);
+    if (target_tensor->shape.defined() && target_tensor->shape.as<ShapeExprNode>() &&
+        arg_tensor->shape.defined() && arg_tensor->shape.as<ShapeExprNode>()) {
+      if (target_tensor->dtype != arg_tensor->dtype) {
+        return {false, false};
+      }
+      auto target_shape = Downcast<ShapeExpr>(target_tensor->shape);
+      auto arg_shape = Downcast<ShapeExpr>(arg_tensor->shape);
+      int target_size = ShapeSize(target_shape);
+      int arg_size = ShapeSize(arg_shape);
+      if (target_size == -1 || arg_size == -1 || target_size < arg_size) {
+        return {false, false};
+      }
+      // exact match: number of dims and each dim matches
+      if (target_shape->values.size() == arg_shape->values.size()) {
+        for (size_t i = 0; i < target_shape->values.size(); i++) {
+          if (!arg_shape->values[i].as<IntImmNode>()) {
+            return {false, false};
+          }
+          if (Downcast<IntImm>(target_shape->values[i])->value !=
+              Downcast<IntImm>(arg_shape->values[i])->value) {
+            return {true, false};
+          }
+        }
+        return {true, true};
+      }
+      return {true, false};
+    } else {
+      return {false, false};
+    }
+  } else if (target_info.as<TupleStructInfoNode>() && arg_info.as<TupleStructInfoNode>()) {
+    auto target_tup = Downcast<TupleStructInfo>(target_info);
+    auto arg_tup = Downcast<TupleStructInfo>(arg_info);
+    if (target_tup->fields.size() != arg_tup->fields.size()) {
+      return {false, false};
+    }
+    bool all_exact = true;
+    for (size_t i = 0; i < target_tup->fields.size(); i++) {
+      auto element_match = SizeMatches(target_tup->fields[i], arg_tup->fields[i]);
+      if (!element_match.first) {
+        return {false, false};
+      }
+      if (!element_match.second) {
+        all_exact = false;
+      }
+    }
+    return {true, all_exact};
+  } else if (target_info.as<PrimStructInfoNode>() && arg_info.as<PrimStructInfoNode>()) {
+    return {true, true};
+  } else {
+    return {false, false};
+  }
+}
+
+// Given an alias index, check if it's a tuple and gather the sets of aliases for the tuple
+// members if so (apply recursively if any of those members are tuples).
+// Return false if the alias set contains -1, meaning a reference to an unknown or
+// possibly dangerous value (no checking we can do for that).
+bool GatherSetsToCheckForLiveness(
+    const std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>&
+        alias_sets,
+    const std::unordered_map<int, std::vector<std::unordered_set<int>>>& tuple_map,
+    std::vector<std::unordered_set<int>>* sets_to_check, int alias_idx) {
+  if (tuple_map.count(alias_idx)) {
+    for (auto member_set : tuple_map.at(alias_idx)) {
+      // contains -1 -> unknown and dangerous, we can short-circuit
+      if (member_set.count(-1)) {
+        return false;
+      }
+      sets_to_check->push_back(member_set);
+
+      // if a member can be a tuple, check it recursively
+      for (int member : member_set) {
+        if (tuple_map.count(member)) {
+          if (!GatherSetsToCheckForLiveness(alias_sets, tuple_map, sets_to_check, member)) {
+            return false;
+          }
+        }
+      }
+    }
+  }
+  return true;
+}
+
+// Check that the target is not live past the index and that no alias of it is live past the
+// binding index (if the target is a tuple, check the conditions recursively for the members)
+bool InplaceConditionsMet(
+    const std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual>& live_ranges,
+    const std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>&
+        alias_sets,
+    const std::unordered_map<int, std::vector<std::unordered_set<int>>>& tuple_map,
+    const std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual>& currently_live,
+    const Expr& target, int binding_idx) {
+  if (auto* var_node = target.as<VarNode>()) {
+    auto current_var = GetRef<Var>(var_node);
+    // if the var is live past this point, we can't use it for in-place computations anyway
+    if (live_ranges.count(current_var)) {
+      auto live_range = live_ranges.at(current_var);
+      if (live_range.second > binding_idx) {
+        return false;
+      }
+    }
+
+    // no entry for the current var -> it must be something external and we have to assume the worst
+    if (!alias_sets.count(current_var)) {
+      return false;
+    }
+    auto alias_set = alias_sets.at(current_var);
+    // -1 -> an external value and we must assume the worst
+    if (alias_set.count(-1)) {
+      return false;
+    }
+    std::vector<std::unordered_set<int>> sets_to_check = {alias_set};
+    std::unordered_set<int> indices_checked;
+    // If a possible alias is a tuple, we will also check for aliases of the members
+    // (possibly recursively)
+    for (int alias_idx : alias_set) {
+      if (!GatherSetsToCheckForLiveness(alias_sets, tuple_map, &sets_to_check, alias_idx)) {
+        return false;
+      }
+    }
+
+    for (Var other_var : currently_live) {
+      if (other_var.same_as(target)) {
+        continue;
+      }
+      // not represented = spooky unknown value that should be modeled by -1
+      if (!alias_sets.count(other_var) || !live_ranges.count(other_var)) {
+        continue;
+      }
+      // var is not live past this point => don't need to worry
+      if (live_ranges.at(other_var).second <= binding_idx) {
+        continue;
+      }
+      auto other_alias_set = alias_sets.at(other_var);
+      for (int alias_idx : other_alias_set) {
+        for (auto check_set : sets_to_check) {
+          if (check_set.count(alias_idx)) {
+            return false;
+          }
+        }
+      }
+    }
+    return true;
+  } else if (auto* tup_node = target.as<TupleNode>()) {
+    for (auto field : tup_node->fields) {
+      if (!InplaceConditionsMet(live_ranges, alias_sets, tuple_map, currently_live, field,
+                                binding_idx)) {
+        return false;
+      }
+    }
+    return true;
+  } else {
+    return true;
+  }
+}
+
+// this is obviously not a complete list
+static std::unordered_set<std::string> SUPPORTED_OPS = {"relax.add",      "relax.subtract",
+                                                        "relax.multiply", "relax.divide",
+                                                        "relax.nn.silu",  "relax.nn.relu"};
+bool OpSupportsInplace(const Op& op) { return SUPPORTED_OPS.count(op->name); }
+
+// Check for in-place eligibility:
+//  1. see if there's an arg big enough to hold the result
+//  2. see if the arg is live past the call
+//  3. see if the arg has an alias that's live past the call
+// If the conditions are met, record the index of that binding.
+// Returns two lists of lists:
+// 1. A list of bindings where at least one argument meets the in-place conditions and the *size*
+//    matches the size of the result.
+// 2. A list of bindings where at least one argument meets the in-place conditions
+//    and *exactly* matches the shape of the result.
+// For both lists, each element is a list of ints of the following format:
+//   The first element is the index of the *binding* in the block.
+//   All remaining elements are the indices of *eligible arguments* in that call.
+std::pair<std::vector<std::vector<int>>, std::vector<std::vector<int>>> FindInplaceOpportunities(
+    const DataflowBlock& block, const Array<Var>& inputs) {
+  auto live_ranges = AnalyzeLiveness(block);
+  AliasAnalyzer analyzer;
+  auto alias_info = analyzer.Analyze(block, inputs);
+  auto alias_sets = alias_info.first;
+  auto tuple_map = alias_info.second;
+
+  std::vector<std::vector<int>> size_match_list;
+  std::vector<std::vector<int>> exact_match_list;
+
+  // sort the live ranges by starting index
+  std::vector<Var> live_order;
+  for (auto kv : live_ranges) {
+    live_order.push_back(kv.first);
+  }
+  std::sort(live_order.begin(), live_order.end(),
+            [&live_ranges](const Var& var1, const Var& var2) -> bool {
+              return live_ranges[var1].first < live_ranges[var2].first;
+            });
+
+  std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual> currently_live;
+  int last_live = 0;
+
+  for (size_t i = 0; i < block->bindings.size(); i++) {
+    // include all vars that are currently live
+    for (int j = last_live; j < static_cast<int>(live_order.size()); j++) {
+      auto live_var = live_order[j];
+      auto live_range = live_ranges[live_var];
+      if (live_range.first > static_cast<int>(i)) {
+        break;
+      }
+      currently_live.insert(live_var);
+      last_live++;
+    }
+
+    // if we reach a binding check the conditions
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+
+    if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        if (!OpSupportsInplace(GetRef<Op>(op_node))) {
+          continue;
+        }
+
+        std::unordered_set<int> candidates;
+        std::unordered_set<int> exact_match_candidates;
+
+        auto target_sinfo = GatherCandidateSinfo(GetStructInfo(defined_var));
+        // can't be done in-place, ignore
+        if (target_sinfo.empty()) {
+          continue;
+        }
+
+        // Check that at least one argument matches size with the result
+        for (size_t j = 0; j < call_node->args.size(); j++) {
+          auto arg = call_node->args[j];
+          for (auto target : target_sinfo) {
+            std::pair<bool, bool> match = SizeMatches(target, GetStructInfo(arg));
+            if (match.first) {
+              candidates.insert(static_cast<int>(j));
+              if (match.second) {
+                exact_match_candidates.insert(static_cast<int>(j));
+              }
+            }
+          }
+        }
+        if (!candidates.size()) {
+          continue;
+        }
+
+        // Make sure at least one candidate is not live past this point and does not have an alias
+        // live past this point
+        std::unordered_set<int> remove_candidates;
+        for (auto candidate : candidates) {
+          if (!InplaceConditionsMet(live_ranges, alias_sets, tuple_map, currently_live,
+                                    call_node->args[candidate], i)) {
+            remove_candidates.insert(candidate);
+          }
+        }
+        // (remove now to avoid modifying the list as we iterate on it)
+        for (auto candidate : remove_candidates) {
+          candidates.erase(candidate);
+        }
+
+        // if we have a candidate, then this can be made in-place. Report the appropriate candidates
+        if (!candidates.size()) {

Review Comment:
   Nit: Prefer `if(candidates.empty())` instead of `if(!canidates.size())`



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {
+      auto target_var = GetRef<Var>(target_var_node);
+      if (alias_map_.count(target_var)) {
+        ret.insert(alias_map_[target_var].begin(), alias_map_[target_var].end());
+      } else {
+        ret.insert(-1);
+      }
+    } else if (auto* target_tuple = value.as<TupleNode>()) {
+      // fresh idx but we update the tuple map
+      int tup_idx = get_fresh_idx();
+      ret.insert(tup_idx);
+      std::vector<std::unordered_set<int>> new_tuple_map;
+      for (auto field : target_tuple->fields) {
+        new_tuple_map.push_back(GetAliasSet(field, bound_var));
+      }
+      tuple_map_[tup_idx] = new_tuple_map;
+    } else if (auto* target_tgi = value.as<TupleGetItemNode>()) {
+      std::unordered_set<int> tuple_set = GetAliasSet(target_tgi->tuple, bound_var);
+      // if -1 is a member of the tuple set, then we have to assume the result is -1
+      if (tuple_set.count(-1)) {
+        ret.insert(-1);
+      } else {
+        // otherwise, consider all members that are tuples of appropriate size and index into them
+        // (this is safe because the type system will ensure we're not indexing into a tuple
+        // of the wrong size)
+        for (int member : tuple_set) {
+          if (tuple_map_.count(member) &&
+              static_cast<int>(tuple_map_[member].size()) > target_tgi->index) {
+            auto member_set = tuple_map_[member][target_tgi->index];
+            ret.insert(member_set.begin(), member_set.end());
+          }
+        }
+      }
+    } else if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        // call_pure_packed: treat as non-op call
+        if (op_node->name == "relax.call_pure_packed") {
+          return HandleMysteryCall(call_node, bound_var, true);
+        } else if (op_node->name == "relax.split") {
+          // split: Returns a tuple, treat as allocation
+
+          // tuple is freshly allocated, but also add components to the tuple map
+          int tup_idx = get_fresh_idx();
+          ret.insert(tup_idx);
+          // the LHS (the bound var) will definitely have a tuple struct info
+          InsertFreshTuple(tup_idx, GetStructInfoAs<TupleStructInfoNode>(bound_var));
+        } else if (op_node->name == "relax.call_tir") {
+          // call_tir: can potentially return a tuple
+          if (auto* tuple_struct_info = call_node->sinfo_args[0].as<TupleStructInfoNode>()) {
+            int tup_idx = get_fresh_idx();
+            ret.insert(tup_idx);
+            InsertFreshTuple(tup_idx, tuple_struct_info);
+          } else {
+            ret.insert(get_fresh_idx());
+          }
+        } else {
+          // We are assuming most op calls return a single fresh allocation.
+          // We may have to track more exceptions
+          ret.insert(get_fresh_idx());
+        }
+      } else {
+        // assume any non-op call can be extremely dangerous and do anything
+        return HandleMysteryCall(call_node, bound_var);
+      }
+    }
+
+    return ret;
+  }
+
+  std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual> alias_map_;
+  std::unordered_map<int, std::vector<std::unordered_set<int>>> tuple_map_;
+  int mem_idx_;
+};
+
+// given a shape, return the allocation size corresponding to it (product of elements)
+int ShapeSize(const ShapeExpr& shape) {
+  int ret = 1;
+  for (auto dim : shape->values) {
+    if (auto int_dim = dim.as<IntImmNode>()) {
+      ret *= static_cast<int>(int_dim->value);
+    } else {
+      return -1;
+    }
+  }
+  return ret;
+}
+
+// Given the struct info of the result, return any struct info nested in it
+// that is eleigible to be used for in-place computations (tensors are eligible
+// only if all their dimensions are integer constants, tuples are eligible if
+// all members are eligible though we can consider only individual members separately)
+std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> GatherCandidateSinfo(
+    const StructInfo& result_sinfo) {
+  if (auto* tensor_info = result_sinfo.as<TensorStructInfoNode>()) {
+    // don't consider void dtype (don't know the size at compile time)
+    if (tensor_info->dtype.is_void()) {
+      return {};
+    }
+    // don't consider cases where we don't know the shape at compile time
+    // TODO(@slyubomirsky): variables might be okay if we use the arithmetic analyzer
+    if (auto* shape_node = tensor_info->shape.as<ShapeExprNode>()) {
+      for (auto dim : shape_node->values) {
+        if (!dim.as<IntImmNode>()) {
+          return {};
+        }
+      }
+      return {GetRef<TensorStructInfo>(tensor_info)};
+    } else {
+      return {};
+    }
+  } else if (auto* tuple_info = result_sinfo.as<TupleStructInfoNode>()) {
+    // we can see if the whole tuple matches or go for any of the components
+    std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> ret;
+    for (auto field : tuple_info->fields) {
+      auto field_candidates = GatherCandidateSinfo(field);
+      ret.insert(field_candidates.begin(), field_candidates.end());
+    }
+    // at least one field should be eligible to be done in-place
+    if (!ret.empty()) {
+      ret.insert(GetRef<StructInfo>(tuple_info));
+    }
+    return ret;
+  } else {
+    // don't consider any other types
+    return {};
+  }
+}
+
+// Given the two struct info, return a pair of bools where the first element is true if
+// the two struct info are the same _size_ in memory and the second element is true
+// if the shapes match _exactly_. Performs this check recursively and ensures the
+// stated condition is true for all tensor members of the struct info (return false
+// if a single pair of corresponding tensors does not meet the condition).
+std::pair<bool, bool> SizeMatches(const StructInfo& target_info, const StructInfo& arg_info) {
+  if (target_info.as<TensorStructInfoNode>() && arg_info.as<TensorStructInfoNode>()) {
+    auto target_tensor = Downcast<TensorStructInfo>(target_info);
+    auto arg_tensor = Downcast<TensorStructInfo>(arg_info);
+    if (target_tensor->shape.defined() && target_tensor->shape.as<ShapeExprNode>() &&
+        arg_tensor->shape.defined() && arg_tensor->shape.as<ShapeExprNode>()) {
+      if (target_tensor->dtype != arg_tensor->dtype) {
+        return {false, false};
+      }
+      auto target_shape = Downcast<ShapeExpr>(target_tensor->shape);
+      auto arg_shape = Downcast<ShapeExpr>(arg_tensor->shape);
+      int target_size = ShapeSize(target_shape);
+      int arg_size = ShapeSize(arg_shape);
+      if (target_size == -1 || arg_size == -1 || target_size < arg_size) {
+        return {false, false};
+      }
+      // exact match: number of dims and each dim matches
+      if (target_shape->values.size() == arg_shape->values.size()) {
+        for (size_t i = 0; i < target_shape->values.size(); i++) {
+          if (!arg_shape->values[i].as<IntImmNode>()) {
+            return {false, false};
+          }
+          if (Downcast<IntImm>(target_shape->values[i])->value !=
+              Downcast<IntImm>(arg_shape->values[i])->value) {
+            return {true, false};
+          }
+        }
+        return {true, true};
+      }
+      return {true, false};
+    } else {
+      return {false, false};
+    }
+  } else if (target_info.as<TupleStructInfoNode>() && arg_info.as<TupleStructInfoNode>()) {
+    auto target_tup = Downcast<TupleStructInfo>(target_info);
+    auto arg_tup = Downcast<TupleStructInfo>(arg_info);
+    if (target_tup->fields.size() != arg_tup->fields.size()) {
+      return {false, false};
+    }
+    bool all_exact = true;
+    for (size_t i = 0; i < target_tup->fields.size(); i++) {
+      auto element_match = SizeMatches(target_tup->fields[i], arg_tup->fields[i]);
+      if (!element_match.first) {
+        return {false, false};
+      }
+      if (!element_match.second) {
+        all_exact = false;
+      }
+    }
+    return {true, all_exact};
+  } else if (target_info.as<PrimStructInfoNode>() && arg_info.as<PrimStructInfoNode>()) {
+    return {true, true};
+  } else {
+    return {false, false};
+  }
+}
+
+// Given an alias index, check if it's a tuple and gather the sets of aliases for the tuple
+// members if so (apply recursively if any of those members are tuples).
+// Return false if the alias set contains -1, meaning a reference to an unknown or
+// possibly dangerous value (no checking we can do for that).
+bool GatherSetsToCheckForLiveness(
+    const std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>&
+        alias_sets,
+    const std::unordered_map<int, std::vector<std::unordered_set<int>>>& tuple_map,
+    std::vector<std::unordered_set<int>>* sets_to_check, int alias_idx) {
+  if (tuple_map.count(alias_idx)) {
+    for (auto member_set : tuple_map.at(alias_idx)) {
+      // contains -1 -> unknown and dangerous, we can short-circuit
+      if (member_set.count(-1)) {
+        return false;
+      }
+      sets_to_check->push_back(member_set);
+
+      // if a member can be a tuple, check it recursively
+      for (int member : member_set) {
+        if (tuple_map.count(member)) {
+          if (!GatherSetsToCheckForLiveness(alias_sets, tuple_map, sets_to_check, member)) {
+            return false;
+          }
+        }
+      }
+    }
+  }
+  return true;
+}
+
+// Check that the target is not live past the index and that no alias of it is live past the
+// binding index (if the target is a tuple, check the conditions recursively for the members)
+bool InplaceConditionsMet(
+    const std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual>& live_ranges,
+    const std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>&
+        alias_sets,
+    const std::unordered_map<int, std::vector<std::unordered_set<int>>>& tuple_map,
+    const std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual>& currently_live,
+    const Expr& target, int binding_idx) {
+  if (auto* var_node = target.as<VarNode>()) {
+    auto current_var = GetRef<Var>(var_node);
+    // if the var is live past this point, we can't use it for in-place computations anyway
+    if (live_ranges.count(current_var)) {
+      auto live_range = live_ranges.at(current_var);
+      if (live_range.second > binding_idx) {
+        return false;
+      }
+    }
+
+    // no entry for the current var -> it must be something external and we have to assume the worst
+    if (!alias_sets.count(current_var)) {
+      return false;
+    }
+    auto alias_set = alias_sets.at(current_var);
+    // -1 -> an external value and we must assume the worst
+    if (alias_set.count(-1)) {
+      return false;
+    }
+    std::vector<std::unordered_set<int>> sets_to_check = {alias_set};
+    std::unordered_set<int> indices_checked;
+    // If a possible alias is a tuple, we will also check for aliases of the members
+    // (possibly recursively)
+    for (int alias_idx : alias_set) {
+      if (!GatherSetsToCheckForLiveness(alias_sets, tuple_map, &sets_to_check, alias_idx)) {
+        return false;
+      }
+    }
+
+    for (Var other_var : currently_live) {
+      if (other_var.same_as(target)) {
+        continue;
+      }
+      // not represented = spooky unknown value that should be modeled by -1
+      if (!alias_sets.count(other_var) || !live_ranges.count(other_var)) {
+        continue;
+      }
+      // var is not live past this point => don't need to worry
+      if (live_ranges.at(other_var).second <= binding_idx) {
+        continue;
+      }
+      auto other_alias_set = alias_sets.at(other_var);
+      for (int alias_idx : other_alias_set) {
+        for (auto check_set : sets_to_check) {
+          if (check_set.count(alias_idx)) {
+            return false;
+          }
+        }
+      }
+    }
+    return true;
+  } else if (auto* tup_node = target.as<TupleNode>()) {
+    for (auto field : tup_node->fields) {
+      if (!InplaceConditionsMet(live_ranges, alias_sets, tuple_map, currently_live, field,
+                                binding_idx)) {
+        return false;
+      }
+    }
+    return true;
+  } else {
+    return true;
+  }
+}
+
+// this is obviously not a complete list
+static std::unordered_set<std::string> SUPPORTED_OPS = {"relax.add",      "relax.subtract",
+                                                        "relax.multiply", "relax.divide",
+                                                        "relax.nn.silu",  "relax.nn.relu"};
+bool OpSupportsInplace(const Op& op) { return SUPPORTED_OPS.count(op->name); }
+
+// Check for in-place eligibility:
+//  1. see if there's an arg big enough to hold the result
+//  2. see if the arg is live past the call
+//  3. see if the arg has an alias that's live past the call
+// If the conditions are met, record the index of that binding.
+// Returns two lists of lists:
+// 1. A list of bindings where at least one argument meets the in-place conditions and the *size*
+//    matches the size of the result.
+// 2. A list of bindings where at least one argument meets the in-place conditions
+//    and *exactly* matches the shape of the result.
+// For both lists, each element is a list of ints of the following format:
+//   The first element is the index of the *binding* in the block.
+//   All remaining elements are the indices of *eligible arguments* in that call.
+std::pair<std::vector<std::vector<int>>, std::vector<std::vector<int>>> FindInplaceOpportunities(
+    const DataflowBlock& block, const Array<Var>& inputs) {
+  auto live_ranges = AnalyzeLiveness(block);
+  AliasAnalyzer analyzer;
+  auto alias_info = analyzer.Analyze(block, inputs);
+  auto alias_sets = alias_info.first;
+  auto tuple_map = alias_info.second;
+
+  std::vector<std::vector<int>> size_match_list;
+  std::vector<std::vector<int>> exact_match_list;
+
+  // sort the live ranges by starting index
+  std::vector<Var> live_order;
+  for (auto kv : live_ranges) {
+    live_order.push_back(kv.first);
+  }
+  std::sort(live_order.begin(), live_order.end(),
+            [&live_ranges](const Var& var1, const Var& var2) -> bool {
+              return live_ranges[var1].first < live_ranges[var2].first;
+            });
+
+  std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual> currently_live;
+  int last_live = 0;
+
+  for (size_t i = 0; i < block->bindings.size(); i++) {
+    // include all vars that are currently live
+    for (int j = last_live; j < static_cast<int>(live_order.size()); j++) {
+      auto live_var = live_order[j];
+      auto live_range = live_ranges[live_var];
+      if (live_range.first > static_cast<int>(i)) {
+        break;
+      }
+      currently_live.insert(live_var);
+      last_live++;
+    }
+
+    // if we reach a binding check the conditions
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+
+    if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        if (!OpSupportsInplace(GetRef<Op>(op_node))) {
+          continue;
+        }
+
+        std::unordered_set<int> candidates;
+        std::unordered_set<int> exact_match_candidates;
+
+        auto target_sinfo = GatherCandidateSinfo(GetStructInfo(defined_var));
+        // can't be done in-place, ignore
+        if (target_sinfo.empty()) {
+          continue;
+        }
+
+        // Check that at least one argument matches size with the result
+        for (size_t j = 0; j < call_node->args.size(); j++) {
+          auto arg = call_node->args[j];
+          for (auto target : target_sinfo) {
+            std::pair<bool, bool> match = SizeMatches(target, GetStructInfo(arg));
+            if (match.first) {
+              candidates.insert(static_cast<int>(j));
+              if (match.second) {
+                exact_match_candidates.insert(static_cast<int>(j));
+              }
+            }
+          }
+        }
+        if (!candidates.size()) {
+          continue;
+        }
+
+        // Make sure at least one candidate is not live past this point and does not have an alias
+        // live past this point
+        std::unordered_set<int> remove_candidates;
+        for (auto candidate : candidates) {
+          if (!InplaceConditionsMet(live_ranges, alias_sets, tuple_map, currently_live,
+                                    call_node->args[candidate], i)) {
+            remove_candidates.insert(candidate);
+          }
+        }
+        // (remove now to avoid modifying the list as we iterate on it)
+        for (auto candidate : remove_candidates) {
+          candidates.erase(candidate);
+        }
+
+        // if we have a candidate, then this can be made in-place. Report the appropriate candidates
+        if (!candidates.size()) {
+          continue;
+        }
+
+        // produce a list of candidates for this index
+        std::vector<int> size_match_indices = {static_cast<int>(i)};
+        size_match_indices.insert(size_match_indices.end(), candidates.begin(), candidates.end());
+        size_match_list.push_back(size_match_indices);
+
+        // also gather up the exact match candidates if there are any
+        std::vector<int> exact_match_indices = {static_cast<int>(i)};
+        for (auto candidate : candidates) {
+          if (exact_match_candidates.count(candidate)) {
+            exact_match_indices.push_back(candidate);
+          }
+        }
+        if (exact_match_indices.size() > 1) {
+          exact_match_list.push_back(exact_match_indices);
+        }
+      }
+    }
+
+    // remove vars whose range has come to an end
+    // (keep a separate set to avoid changing the sit while iterating on it)
+    std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual> remove;
+    for (auto var : currently_live) {
+      auto live_range = live_ranges[var];
+      if (live_range.second <= static_cast<int>(i)) {
+        remove.insert(var);
+      }
+    }
+    for (auto var : remove) {
+      currently_live.erase(var);
+    }
+  }
+
+  return {size_match_list, exact_match_list};
+}
+
+// Replace buffers in a PrimFunc according to the mapping.
+tir::Stmt RemapBuffers(const tir::Stmt& stmt, const Map<tir::Buffer, tir::Buffer>& buffer_map) {
+  class BufferMapper : public tir::StmtExprMutator {
+   public:
+    explicit BufferMapper(const Map<tir::Buffer, tir::Buffer>& buffer_map)
+        : buffer_map_(buffer_map) {}
+
+    tir::Stmt Remap(const tir::Stmt& stmt) { return VisitStmt(stmt); }
+
+    PrimExpr VisitExpr_(const tir::BufferLoadNode* op) final {
+      auto node = Downcast<tir::BufferLoad>(tir::StmtExprMutator::VisitExpr_(op));
+      auto* node_cow = node.CopyOnWrite();
+      node_cow->buffer = AttemptRemap(node->buffer);
+      return node;
+    }
+
+    tir::Stmt VisitStmt_(const tir::BufferStoreNode* op) final {
+      auto node = Downcast<tir::BufferStore>(tir::StmtExprMutator::VisitStmt_(op));
+      auto* node_cow = node.CopyOnWrite();
+      node_cow->buffer = AttemptRemap(node->buffer);
+      return node;
+    }
+
+    tir::Stmt VisitStmt_(const tir::BufferRealizeNode* op) final {
+      auto node = Downcast<tir::BufferRealize>(tir::StmtExprMutator::VisitStmt_(op));
+      auto* node_cow = node.CopyOnWrite();
+      node_cow->buffer = AttemptRemap(node->buffer);
+      return node;
+    }
+
+    tir::Stmt VisitStmt_(const tir::DeclBufferNode* op) final {
+      auto node = Downcast<tir::DeclBuffer>(tir::StmtExprMutator::VisitStmt_(op));
+      auto* node_cow = node.CopyOnWrite();
+      node_cow->buffer = AttemptRemap(node->buffer);
+      return node;
+    }
+
+    tir::Stmt VisitStmt_(const tir::BlockNode* op) final {
+      auto node = Downcast<tir::Block>(tir::StmtExprMutator::VisitStmt_(op));
+      auto* node_cow = node.CopyOnWrite();
+      // need the lambdas because class methods are not first-class (how ironic)
+      node_cow->alloc_buffers =
+          node->alloc_buffers.Map([this](const tir::Buffer& b) { return AttemptRemap(b); });
+      node_cow->reads =
+          node->reads.Map([this](const tir::BufferRegion& br) { return VisitBufferRegion(br); });
+      node_cow->writes =
+          node->writes.Map([this](const tir::BufferRegion& br) { return VisitBufferRegion(br); });
+      node_cow->match_buffers = node->match_buffers.Map(
+          [this](const tir::MatchBufferRegion& mbr) { return VisitMatchBufferRegion(mbr); });
+      return node;
+    }
+
+   private:
+    tir::Buffer AttemptRemap(const tir::Buffer& buffer) {
+      if (buffer_map_.count(buffer)) {
+        return buffer_map_.at(buffer);
+      }
+      return buffer;
+    }
+
+    tir::BufferRegion VisitBufferRegion(tir::BufferRegion region) {
+      auto* region_cow = region.CopyOnWrite();
+      region_cow->buffer = AttemptRemap(region_cow->buffer);
+      return region;
+    }
+
+    tir::MatchBufferRegion VisitMatchBufferRegion(tir::MatchBufferRegion region) {
+      auto* region_cow = region.CopyOnWrite();
+      region_cow->buffer = AttemptRemap(region_cow->buffer);
+      return region;
+    }
+
+    const Map<tir::Buffer, tir::Buffer>& buffer_map_;
+  };
+
+  BufferMapper mapper(buffer_map);
+  auto ret = mapper.Remap(stmt);
+  return ret;
+}
+
+class ModuleInplaceTransformer : public ExprMutator {
+ public:
+  explicit ModuleInplaceTransformer(const IRModule& mod) : mod_(mod) {
+    builder_ = BlockBuilder::Create(mod);
+  }
+
+  IRModule Transform() {
+    // visit every Relax function in the module
+    for (auto kv : mod_->functions) {
+      if (auto* func_node = kv.second.as<FunctionNode>()) {
+        auto gv = kv.first;
+        auto func_params = func_node->params;
+        auto function = GetRef<Function>(func_node);
+        auto* function_cow = function.CopyOnWrite();
+        auto new_body = VisitExpr(function->body);
+        function_cow->body = new_body;
+        builder_->UpdateFunction(gv, function);
+      }
+    }
+
+    auto ret = builder_->GetContextIRModule();
+    // clean up to avoid polluting the IRModule
+    for (auto gv : legalizers_added) {
+      ret->Remove(gv);
+    }
+    return ret;
+  }
+
+  // for handling inner functions
+  Expr VisitExpr_(const FunctionNode* op) override {
+    auto old_func_params = func_params;
+    func_params = op->params;
+    auto ret = ExprMutator::VisitExpr(GetRef<Function>(op));
+    func_params = old_func_params;
+    return ret;
+  }
+
+  // the only case we will override: we will visit all binding blocks
+  // and replace any valid calls in them
+  BindingBlock VisitBindingBlock_(const DataflowBlockNode* op) override {
+    auto block = GetRef<DataflowBlock>(op);
+    // std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual> free_var_set;
+    // for (auto binding : block->bindings) {
+    //   auto binding_free_vars = FreeVars(GetBoundValue(binding));
+    //   free_var_set.insert(binding_free_vars.begin(), binding_free_vars.end());
+    // }
+    // Array<Var> free_var_list(free_var_set.begin(), free_var_set.end());
+
+    // For now, only handle exact match cases.
+    // Note: Not passing any input values for now, as we can't make any assumptions
+    // about them.
+    auto matches_found = FindInplaceOpportunities(block, {});
+    auto exact_matches = matches_found.second;
+
+    Array<Binding> new_bindings;
+    int current_match_index = 0;
+    for (size_t i = 0; i < block->bindings.size(); i++) {
+      if (current_match_index >= static_cast<int>(exact_matches.size()) ||
+          exact_matches[current_match_index][0] != static_cast<int>(i)) {
+        new_bindings.push_back(block->bindings[i]);
+        continue;
+      }
+
+      auto target_binding = block->bindings[i];
+      auto target_call = Downcast<Call>(GetBoundValue(target_binding));
+      // can just pick the first index arbitrarily (only using one output for now too)
+      auto new_call = CreateInplaceCall(target_call, {exact_matches[current_match_index][1]});
+      // now replace the binding appropriately
+      if (auto* var_binding_node = target_binding.as<VarBindingNode>()) {
+        auto var_binding = GetRef<VarBinding>(var_binding_node);
+        auto* var_binding_cow = var_binding.CopyOnWrite();
+        var_binding_cow->value = new_call;
+        new_bindings.push_back(var_binding);
+      } else if (auto* match_cast_node = target_binding.as<MatchCastNode>()) {
+        auto match_cast = GetRef<MatchCast>(match_cast_node);
+        auto* match_cast_cow = match_cast.CopyOnWrite();
+        match_cast_cow->value = new_call;
+        new_bindings.push_back(match_cast);
+      } else {
+        CHECK(false) << "Invalid binding type";
+      }
+      current_match_index++;
+    }
+    return DataflowBlock(new_bindings, block->span);
+  }
+
+  // Given the call and indices of arguments that could be done in-place,
+  // replace the call with a call to an in-place PrimFunc.
+  // (Made public for testing.)
+  Call CreateInplaceCall(const Call& call, const Array<Integer>& inplace_indices) {
+    static const auto& legalize_map = Op::GetAttrMap<FLegalize>("FLegalize");
+    static const auto& call_tir_inplace_op = Op::Get("relax.call_tir_inplace");
+
+    auto op = Downcast<Op>(call->op);
+    auto legalized_call = Downcast<Call>(legalize_map[op](builder_, call));
+    auto* legalized_call_cow = legalized_call.CopyOnWrite();
+
+    // The legalized call should be call_tir. We will replace it with call_tir_inplace
+    // and replace the called PrimFunc with an inplace version
+    auto legal_op = Downcast<GlobalVar>(legalized_call->args[0]);
+    legalizers_added.push_back(legal_op);
+    auto inline_legal_op_name = legal_op->name_hint + "_inplace";
+
+    auto mod = builder_->GetContextIRModule();
+    auto legal_primfunc = Downcast<tir::PrimFunc>(mod->Lookup(legal_op));
+    auto* legal_primfunc_cow = legal_primfunc.CopyOnWrite();
+    size_t num_outs = inplace_indices.size();
+    size_t num_params = legal_primfunc->params.size();
+
+    // the replacement we must make:
+    // 1. For each output var, replace its corresponding buffers with the corresponding inplace
+    // index
+    //    var's buffers
+    // 2. For each output var, replace its instances with the corresponding inplace index var
+    // 3. Do the same for the *buffer vars* corresponding to the output vars
+    // 4. Remove the output vars from the param list and buffer map
+    Map<tir::Buffer, tir::Buffer> buffer_subst_map;
+    Map<tir::Var, tir::Var> var_subst_map;
+    for (size_t i = 0; i < num_outs; i++) {
+      // we will substitute output i with the corresponding param indicated by inplace indices
+      auto output_var = legal_primfunc->params[num_params - num_outs + i];
+      auto inplace_var = legal_primfunc->params[inplace_indices[i].IntValue()];
+      var_subst_map.Set(output_var, inplace_var);
+
+      // also do the same with the buffer vars
+      auto output_buffer = legal_primfunc->buffer_map.at(output_var);
+      auto inplace_buffer = legal_primfunc->buffer_map.at(inplace_var);
+      var_subst_map.Set(output_buffer->data, inplace_buffer->data);
+      buffer_subst_map.Set(output_buffer, inplace_buffer);
+    }
+
+    // apply substitutions
+    legal_primfunc_cow->body = RemapBuffers(legal_primfunc->body, buffer_subst_map);
+    legal_primfunc_cow->body = tir::Substitute(
+        legal_primfunc->body, [&var_subst_map](const tir::Var& v) -> Optional<PrimExpr> {
+          if (var_subst_map.count(v)) {
+            return var_subst_map.at(v);
+          }
+          return Optional<PrimExpr>();
+        });
+
+    // remove the now-unused outputs from the buffer map
+    auto buffer_map = legal_primfunc->buffer_map;
+    for (size_t i = 0; i < num_outs; i++) {
+      buffer_map.erase(legal_primfunc->params[num_params - num_outs + i]);
+    }
+    legal_primfunc_cow->buffer_map = buffer_map;
+
+    // now get rid of the last num_outputs arguments
+    // (couldn't do earlier or else it would have thrown off the indexing)
+    legal_primfunc_cow->params = Array<tir::Var>(
+        legal_primfunc->params.begin(), legal_primfunc->params.begin() + (num_params - num_outs));
+
+    // note: this might be a good time to get rid of the old legalized function, but we don't do it
+    // now because later ops might need the same one. Instead, we will clean up at the end
+    auto new_gv = builder_->AddFunction(legal_primfunc, inline_legal_op_name);
+
+    // update the call (change the op, update the argument, change the attrs)
+    legalized_call_cow->op = call_tir_inplace_op;
+
+    Array<Expr> new_args(legalized_call->args.begin(), legalized_call->args.end());
+    new_args.Set(0, new_gv);
+    legalized_call_cow->args = new_args;
+
+    ObjectPtr<CallTIRInplaceAttrs> attrs = make_object<CallTIRInplaceAttrs>();
+    attrs->inplace_indices = inplace_indices;
+    legalized_call_cow->attrs = Attrs(attrs);
+
+    return legalized_call;
+  }
+
+  // Made public for testing.
+  IRModule CurrentMod() { return builder_->GetContextIRModule(); }
+
+ private:
+  const IRModule& mod_;
+  Array<GlobalVar>
+      legalizers_added;    // Keep track of legalizers we add so we can clean up at the end.
+  Array<Var> func_params;  // The current function's params will be treated as non-aliased

Review Comment:
   Is this assumption required?  As a user of the pass, I would assume that any arguments for externally-exposed functions would be ineligible for overwriting.  With that behavior, it wouldn't matter if they are aliased.



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {
+      auto target_var = GetRef<Var>(target_var_node);
+      if (alias_map_.count(target_var)) {
+        ret.insert(alias_map_[target_var].begin(), alias_map_[target_var].end());
+      } else {
+        ret.insert(-1);
+      }
+    } else if (auto* target_tuple = value.as<TupleNode>()) {
+      // fresh idx but we update the tuple map
+      int tup_idx = get_fresh_idx();
+      ret.insert(tup_idx);
+      std::vector<std::unordered_set<int>> new_tuple_map;
+      for (auto field : target_tuple->fields) {
+        new_tuple_map.push_back(GetAliasSet(field, bound_var));
+      }
+      tuple_map_[tup_idx] = new_tuple_map;
+    } else if (auto* target_tgi = value.as<TupleGetItemNode>()) {
+      std::unordered_set<int> tuple_set = GetAliasSet(target_tgi->tuple, bound_var);
+      // if -1 is a member of the tuple set, then we have to assume the result is -1
+      if (tuple_set.count(-1)) {
+        ret.insert(-1);
+      } else {
+        // otherwise, consider all members that are tuples of appropriate size and index into them
+        // (this is safe because the type system will ensure we're not indexing into a tuple
+        // of the wrong size)
+        for (int member : tuple_set) {
+          if (tuple_map_.count(member) &&
+              static_cast<int>(tuple_map_[member].size()) > target_tgi->index) {
+            auto member_set = tuple_map_[member][target_tgi->index];
+            ret.insert(member_set.begin(), member_set.end());
+          }
+        }
+      }
+    } else if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        // call_pure_packed: treat as non-op call
+        if (op_node->name == "relax.call_pure_packed") {
+          return HandleMysteryCall(call_node, bound_var, true);
+        } else if (op_node->name == "relax.split") {
+          // split: Returns a tuple, treat as allocation
+
+          // tuple is freshly allocated, but also add components to the tuple map
+          int tup_idx = get_fresh_idx();
+          ret.insert(tup_idx);
+          // the LHS (the bound var) will definitely have a tuple struct info
+          InsertFreshTuple(tup_idx, GetStructInfoAs<TupleStructInfoNode>(bound_var));
+        } else if (op_node->name == "relax.call_tir") {
+          // call_tir: can potentially return a tuple
+          if (auto* tuple_struct_info = call_node->sinfo_args[0].as<TupleStructInfoNode>()) {
+            int tup_idx = get_fresh_idx();
+            ret.insert(tup_idx);
+            InsertFreshTuple(tup_idx, tuple_struct_info);
+          } else {
+            ret.insert(get_fresh_idx());
+          }
+        } else {
+          // We are assuming most op calls return a single fresh allocation.
+          // We may have to track more exceptions
+          ret.insert(get_fresh_idx());
+        }
+      } else {
+        // assume any non-op call can be extremely dangerous and do anything
+        return HandleMysteryCall(call_node, bound_var);
+      }
+    }
+
+    return ret;
+  }
+
+  std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual> alias_map_;
+  std::unordered_map<int, std::vector<std::unordered_set<int>>> tuple_map_;
+  int mem_idx_;
+};
+
+// given a shape, return the allocation size corresponding to it (product of elements)
+int ShapeSize(const ShapeExpr& shape) {

Review Comment:
   Can this be `PrimExpr` instead of `int` return type?  That would avoid needing to use `-1` as a special value.
   
   ```c++
   PrimExpr NumElements(const ShapeExpr& shape) {
      PrimExpr ret = IntImm(DataType::Int(64), 1);
      for(auto dim: shape->values) {
          ret *= dim;
      }
      return ret;
   }
   ```



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {
+      auto target_var = GetRef<Var>(target_var_node);
+      if (alias_map_.count(target_var)) {
+        ret.insert(alias_map_[target_var].begin(), alias_map_[target_var].end());
+      } else {
+        ret.insert(-1);
+      }
+    } else if (auto* target_tuple = value.as<TupleNode>()) {
+      // fresh idx but we update the tuple map
+      int tup_idx = get_fresh_idx();
+      ret.insert(tup_idx);
+      std::vector<std::unordered_set<int>> new_tuple_map;
+      for (auto field : target_tuple->fields) {
+        new_tuple_map.push_back(GetAliasSet(field, bound_var));
+      }
+      tuple_map_[tup_idx] = new_tuple_map;
+    } else if (auto* target_tgi = value.as<TupleGetItemNode>()) {
+      std::unordered_set<int> tuple_set = GetAliasSet(target_tgi->tuple, bound_var);
+      // if -1 is a member of the tuple set, then we have to assume the result is -1
+      if (tuple_set.count(-1)) {
+        ret.insert(-1);
+      } else {
+        // otherwise, consider all members that are tuples of appropriate size and index into them
+        // (this is safe because the type system will ensure we're not indexing into a tuple
+        // of the wrong size)
+        for (int member : tuple_set) {
+          if (tuple_map_.count(member) &&
+              static_cast<int>(tuple_map_[member].size()) > target_tgi->index) {
+            auto member_set = tuple_map_[member][target_tgi->index];
+            ret.insert(member_set.begin(), member_set.end());
+          }
+        }
+      }
+    } else if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        // call_pure_packed: treat as non-op call
+        if (op_node->name == "relax.call_pure_packed") {
+          return HandleMysteryCall(call_node, bound_var, true);
+        } else if (op_node->name == "relax.split") {
+          // split: Returns a tuple, treat as allocation
+
+          // tuple is freshly allocated, but also add components to the tuple map
+          int tup_idx = get_fresh_idx();
+          ret.insert(tup_idx);
+          // the LHS (the bound var) will definitely have a tuple struct info
+          InsertFreshTuple(tup_idx, GetStructInfoAs<TupleStructInfoNode>(bound_var));
+        } else if (op_node->name == "relax.call_tir") {
+          // call_tir: can potentially return a tuple
+          if (auto* tuple_struct_info = call_node->sinfo_args[0].as<TupleStructInfoNode>()) {
+            int tup_idx = get_fresh_idx();
+            ret.insert(tup_idx);
+            InsertFreshTuple(tup_idx, tuple_struct_info);
+          } else {
+            ret.insert(get_fresh_idx());
+          }
+        } else {
+          // We are assuming most op calls return a single fresh allocation.
+          // We may have to track more exceptions
+          ret.insert(get_fresh_idx());
+        }
+      } else {
+        // assume any non-op call can be extremely dangerous and do anything
+        return HandleMysteryCall(call_node, bound_var);
+      }
+    }
+
+    return ret;
+  }
+
+  std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual> alias_map_;
+  std::unordered_map<int, std::vector<std::unordered_set<int>>> tuple_map_;
+  int mem_idx_;
+};
+
+// given a shape, return the allocation size corresponding to it (product of elements)
+int ShapeSize(const ShapeExpr& shape) {
+  int ret = 1;
+  for (auto dim : shape->values) {
+    if (auto int_dim = dim.as<IntImmNode>()) {
+      ret *= static_cast<int>(int_dim->value);
+    } else {
+      return -1;
+    }
+  }
+  return ret;
+}
+
+// Given the struct info of the result, return any struct info nested in it
+// that is eleigible to be used for in-place computations (tensors are eligible
+// only if all their dimensions are integer constants, tuples are eligible if
+// all members are eligible though we can consider only individual members separately)
+std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> GatherCandidateSinfo(
+    const StructInfo& result_sinfo) {
+  if (auto* tensor_info = result_sinfo.as<TensorStructInfoNode>()) {
+    // don't consider void dtype (don't know the size at compile time)
+    if (tensor_info->dtype.is_void()) {
+      return {};
+    }
+    // don't consider cases where we don't know the shape at compile time
+    // TODO(@slyubomirsky): variables might be okay if we use the arithmetic analyzer
+    if (auto* shape_node = tensor_info->shape.as<ShapeExprNode>()) {
+      for (auto dim : shape_node->values) {
+        if (!dim.as<IntImmNode>()) {
+          return {};
+        }
+      }
+      return {GetRef<TensorStructInfo>(tensor_info)};
+    } else {
+      return {};
+    }
+  } else if (auto* tuple_info = result_sinfo.as<TupleStructInfoNode>()) {
+    // we can see if the whole tuple matches or go for any of the components
+    std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> ret;
+    for (auto field : tuple_info->fields) {
+      auto field_candidates = GatherCandidateSinfo(field);
+      ret.insert(field_candidates.begin(), field_candidates.end());
+    }
+    // at least one field should be eligible to be done in-place
+    if (!ret.empty()) {
+      ret.insert(GetRef<StructInfo>(tuple_info));
+    }
+    return ret;
+  } else {
+    // don't consider any other types
+    return {};
+  }
+}
+
+// Given the two struct info, return a pair of bools where the first element is true if
+// the two struct info are the same _size_ in memory and the second element is true
+// if the shapes match _exactly_. Performs this check recursively and ensures the
+// stated condition is true for all tensor members of the struct info (return false
+// if a single pair of corresponding tensors does not meet the condition).
+std::pair<bool, bool> SizeMatches(const StructInfo& target_info, const StructInfo& arg_info) {
+  if (target_info.as<TensorStructInfoNode>() && arg_info.as<TensorStructInfoNode>()) {
+    auto target_tensor = Downcast<TensorStructInfo>(target_info);
+    auto arg_tensor = Downcast<TensorStructInfo>(arg_info);
+    if (target_tensor->shape.defined() && target_tensor->shape.as<ShapeExprNode>() &&
+        arg_tensor->shape.defined() && arg_tensor->shape.as<ShapeExprNode>()) {
+      if (target_tensor->dtype != arg_tensor->dtype) {
+        return {false, false};
+      }
+      auto target_shape = Downcast<ShapeExpr>(target_tensor->shape);
+      auto arg_shape = Downcast<ShapeExpr>(arg_tensor->shape);
+      int target_size = ShapeSize(target_shape);
+      int arg_size = ShapeSize(arg_shape);
+      if (target_size == -1 || arg_size == -1 || target_size < arg_size) {

Review Comment:
   I think this condition is a bit too strong.  If there is an element-wise operator applied to a dynamic shape, the `target_size == -1` condition would treat it as a mismatch.
   
   ```python
   A: R.Tensor([m,n], "float16")
   B: R.Tensor([m,n], "float16")
   C: R.Tensor([m,n], "float16") = R.add(A, B)
   ```
   
   With `ShapeSize` replaced by `NumElements`, I think we should instead have the condition below.  This would handle the integer case, and would also handle symbolic shapes that are the same.  The double-negative of `!analyzer.CanProve( ! (arg_size < target_size) )` ensures that we have the correct fallback behavior, since an improvable check should not result in in-place usage.
   
   ```c++
   PrimExpr target_size = NumElements(target_shape);
   PrimExpr arg_size = NumElements(arg_shape);
   if( !analyzer.CanProve(arg_size >= target_size) ) {
       return {false, false};
   }
   ```



##########
tests/python/relax/test_dataflow_inplace.py:
##########
@@ -0,0 +1,464 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from typing import List, Set, Tuple
+import tvm
+from tvm import relax, testing
+from tvm.relax.transform import DataflowUseInplaceCalls
+from tvm.relax.testing.transform import (
+    dataflow_liveness_analysis,
+    dataflow_alias_analysis,
+    dataflow_inplace_analysis,
+    dataflow_single_inplace_call,
+)
+from tvm.script.parser import ir as I, relax as R, tir as T
+
+import numpy as np
+
+
+def test_liveness_analysis():
+    @I.ir_module
+    class BasicLiveness:
+        @R.function
+        def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            with R.dataflow():
+                y = R.const(1, dtype="int32")
+                z = R.add(x, y)
+                q = R.multiply(z, y)
+                p = R.add(z, q)
+                n = R.multiply(p, p)
+                R.output(n)
+            return n
+
+    block = BasicLiveness["main"].body.blocks[0]
+    live_ranges = dataflow_liveness_analysis(block)
+    expected_ranges = {
+        "x": (-1, 1),

Review Comment:
   I think `"x"` should have a liveness range of `(-1,5)`, because local analysis cannot determine whether `x` is needed again within the calling scope of the function.  Rather than tracking the function parameters separate from the liveness, this would track the function parameters as being live for the entire duration.  That would avoid potential pitfalls in the future, as the liveness results would be the only thing required to track.



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {

Review Comment:
   This looks very similar to the alias tracking in `CanonicalizeBindings`.  Could this be used to replace the analysis step in `CanonicalizeBindings`?



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {
+      auto target_var = GetRef<Var>(target_var_node);
+      if (alias_map_.count(target_var)) {
+        ret.insert(alias_map_[target_var].begin(), alias_map_[target_var].end());
+      } else {
+        ret.insert(-1);
+      }
+    } else if (auto* target_tuple = value.as<TupleNode>()) {
+      // fresh idx but we update the tuple map
+      int tup_idx = get_fresh_idx();
+      ret.insert(tup_idx);
+      std::vector<std::unordered_set<int>> new_tuple_map;
+      for (auto field : target_tuple->fields) {
+        new_tuple_map.push_back(GetAliasSet(field, bound_var));
+      }
+      tuple_map_[tup_idx] = new_tuple_map;
+    } else if (auto* target_tgi = value.as<TupleGetItemNode>()) {
+      std::unordered_set<int> tuple_set = GetAliasSet(target_tgi->tuple, bound_var);
+      // if -1 is a member of the tuple set, then we have to assume the result is -1
+      if (tuple_set.count(-1)) {
+        ret.insert(-1);
+      } else {
+        // otherwise, consider all members that are tuples of appropriate size and index into them
+        // (this is safe because the type system will ensure we're not indexing into a tuple
+        // of the wrong size)
+        for (int member : tuple_set) {
+          if (tuple_map_.count(member) &&
+              static_cast<int>(tuple_map_[member].size()) > target_tgi->index) {
+            auto member_set = tuple_map_[member][target_tgi->index];
+            ret.insert(member_set.begin(), member_set.end());
+          }
+        }
+      }
+    } else if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        // call_pure_packed: treat as non-op call
+        if (op_node->name == "relax.call_pure_packed") {
+          return HandleMysteryCall(call_node, bound_var, true);
+        } else if (op_node->name == "relax.split") {
+          // split: Returns a tuple, treat as allocation
+
+          // tuple is freshly allocated, but also add components to the tuple map
+          int tup_idx = get_fresh_idx();
+          ret.insert(tup_idx);
+          // the LHS (the bound var) will definitely have a tuple struct info
+          InsertFreshTuple(tup_idx, GetStructInfoAs<TupleStructInfoNode>(bound_var));
+        } else if (op_node->name == "relax.call_tir") {
+          // call_tir: can potentially return a tuple
+          if (auto* tuple_struct_info = call_node->sinfo_args[0].as<TupleStructInfoNode>()) {
+            int tup_idx = get_fresh_idx();
+            ret.insert(tup_idx);
+            InsertFreshTuple(tup_idx, tuple_struct_info);
+          } else {
+            ret.insert(get_fresh_idx());
+          }
+        } else {
+          // We are assuming most op calls return a single fresh allocation.
+          // We may have to track more exceptions
+          ret.insert(get_fresh_idx());
+        }
+      } else {
+        // assume any non-op call can be extremely dangerous and do anything
+        return HandleMysteryCall(call_node, bound_var);
+      }
+    }
+
+    return ret;
+  }
+
+  std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual> alias_map_;
+  std::unordered_map<int, std::vector<std::unordered_set<int>>> tuple_map_;
+  int mem_idx_;
+};
+
+// given a shape, return the allocation size corresponding to it (product of elements)
+int ShapeSize(const ShapeExpr& shape) {
+  int ret = 1;
+  for (auto dim : shape->values) {
+    if (auto int_dim = dim.as<IntImmNode>()) {
+      ret *= static_cast<int>(int_dim->value);
+    } else {
+      return -1;
+    }
+  }
+  return ret;
+}
+
+// Given the struct info of the result, return any struct info nested in it
+// that is eleigible to be used for in-place computations (tensors are eligible
+// only if all their dimensions are integer constants, tuples are eligible if
+// all members are eligible though we can consider only individual members separately)
+std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> GatherCandidateSinfo(
+    const StructInfo& result_sinfo) {
+  if (auto* tensor_info = result_sinfo.as<TensorStructInfoNode>()) {
+    // don't consider void dtype (don't know the size at compile time)
+    if (tensor_info->dtype.is_void()) {
+      return {};
+    }
+    // don't consider cases where we don't know the shape at compile time
+    // TODO(@slyubomirsky): variables might be okay if we use the arithmetic analyzer
+    if (auto* shape_node = tensor_info->shape.as<ShapeExprNode>()) {
+      for (auto dim : shape_node->values) {
+        if (!dim.as<IntImmNode>()) {
+          return {};
+        }
+      }
+      return {GetRef<TensorStructInfo>(tensor_info)};
+    } else {
+      return {};
+    }
+  } else if (auto* tuple_info = result_sinfo.as<TupleStructInfoNode>()) {
+    // we can see if the whole tuple matches or go for any of the components
+    std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> ret;
+    for (auto field : tuple_info->fields) {
+      auto field_candidates = GatherCandidateSinfo(field);
+      ret.insert(field_candidates.begin(), field_candidates.end());
+    }
+    // at least one field should be eligible to be done in-place
+    if (!ret.empty()) {
+      ret.insert(GetRef<StructInfo>(tuple_info));
+    }
+    return ret;
+  } else {
+    // don't consider any other types
+    return {};
+  }
+}
+
+// Given the two struct info, return a pair of bools where the first element is true if
+// the two struct info are the same _size_ in memory and the second element is true
+// if the shapes match _exactly_. Performs this check recursively and ensures the
+// stated condition is true for all tensor members of the struct info (return false
+// if a single pair of corresponding tensors does not meet the condition).
+std::pair<bool, bool> SizeMatches(const StructInfo& target_info, const StructInfo& arg_info) {
+  if (target_info.as<TensorStructInfoNode>() && arg_info.as<TensorStructInfoNode>()) {
+    auto target_tensor = Downcast<TensorStructInfo>(target_info);
+    auto arg_tensor = Downcast<TensorStructInfo>(arg_info);
+    if (target_tensor->shape.defined() && target_tensor->shape.as<ShapeExprNode>() &&
+        arg_tensor->shape.defined() && arg_tensor->shape.as<ShapeExprNode>()) {
+      if (target_tensor->dtype != arg_tensor->dtype) {
+        return {false, false};
+      }
+      auto target_shape = Downcast<ShapeExpr>(target_tensor->shape);
+      auto arg_shape = Downcast<ShapeExpr>(arg_tensor->shape);
+      int target_size = ShapeSize(target_shape);
+      int arg_size = ShapeSize(arg_shape);
+      if (target_size == -1 || arg_size == -1 || target_size < arg_size) {
+        return {false, false};
+      }
+      // exact match: number of dims and each dim matches
+      if (target_shape->values.size() == arg_shape->values.size()) {
+        for (size_t i = 0; i < target_shape->values.size(); i++) {
+          if (!arg_shape->values[i].as<IntImmNode>()) {
+            return {false, false};
+          }
+          if (Downcast<IntImm>(target_shape->values[i])->value !=
+              Downcast<IntImm>(arg_shape->values[i])->value) {
+            return {true, false};
+          }
+        }
+        return {true, true};
+      }
+      return {true, false};
+    } else {
+      return {false, false};
+    }
+  } else if (target_info.as<TupleStructInfoNode>() && arg_info.as<TupleStructInfoNode>()) {
+    auto target_tup = Downcast<TupleStructInfo>(target_info);
+    auto arg_tup = Downcast<TupleStructInfo>(arg_info);
+    if (target_tup->fields.size() != arg_tup->fields.size()) {
+      return {false, false};
+    }
+    bool all_exact = true;
+    for (size_t i = 0; i < target_tup->fields.size(); i++) {
+      auto element_match = SizeMatches(target_tup->fields[i], arg_tup->fields[i]);
+      if (!element_match.first) {
+        return {false, false};
+      }
+      if (!element_match.second) {
+        all_exact = false;
+      }
+    }
+    return {true, all_exact};
+  } else if (target_info.as<PrimStructInfoNode>() && arg_info.as<PrimStructInfoNode>()) {
+    return {true, true};
+  } else {
+    return {false, false};
+  }
+}
+
+// Given an alias index, check if it's a tuple and gather the sets of aliases for the tuple
+// members if so (apply recursively if any of those members are tuples).
+// Return false if the alias set contains -1, meaning a reference to an unknown or
+// possibly dangerous value (no checking we can do for that).
+bool GatherSetsToCheckForLiveness(
+    const std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>&
+        alias_sets,
+    const std::unordered_map<int, std::vector<std::unordered_set<int>>>& tuple_map,
+    std::vector<std::unordered_set<int>>* sets_to_check, int alias_idx) {
+  if (tuple_map.count(alias_idx)) {
+    for (auto member_set : tuple_map.at(alias_idx)) {
+      // contains -1 -> unknown and dangerous, we can short-circuit
+      if (member_set.count(-1)) {
+        return false;
+      }
+      sets_to_check->push_back(member_set);
+
+      // if a member can be a tuple, check it recursively
+      for (int member : member_set) {
+        if (tuple_map.count(member)) {
+          if (!GatherSetsToCheckForLiveness(alias_sets, tuple_map, sets_to_check, member)) {
+            return false;
+          }
+        }
+      }
+    }
+  }
+  return true;
+}
+
+// Check that the target is not live past the index and that no alias of it is live past the
+// binding index (if the target is a tuple, check the conditions recursively for the members)
+bool InplaceConditionsMet(
+    const std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual>& live_ranges,
+    const std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>&
+        alias_sets,
+    const std::unordered_map<int, std::vector<std::unordered_set<int>>>& tuple_map,
+    const std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual>& currently_live,
+    const Expr& target, int binding_idx) {
+  if (auto* var_node = target.as<VarNode>()) {
+    auto current_var = GetRef<Var>(var_node);
+    // if the var is live past this point, we can't use it for in-place computations anyway
+    if (live_ranges.count(current_var)) {
+      auto live_range = live_ranges.at(current_var);
+      if (live_range.second > binding_idx) {
+        return false;
+      }
+    }
+
+    // no entry for the current var -> it must be something external and we have to assume the worst
+    if (!alias_sets.count(current_var)) {
+      return false;
+    }
+    auto alias_set = alias_sets.at(current_var);
+    // -1 -> an external value and we must assume the worst
+    if (alias_set.count(-1)) {
+      return false;
+    }
+    std::vector<std::unordered_set<int>> sets_to_check = {alias_set};
+    std::unordered_set<int> indices_checked;
+    // If a possible alias is a tuple, we will also check for aliases of the members
+    // (possibly recursively)
+    for (int alias_idx : alias_set) {
+      if (!GatherSetsToCheckForLiveness(alias_sets, tuple_map, &sets_to_check, alias_idx)) {
+        return false;
+      }
+    }
+
+    for (Var other_var : currently_live) {
+      if (other_var.same_as(target)) {
+        continue;
+      }
+      // not represented = spooky unknown value that should be modeled by -1
+      if (!alias_sets.count(other_var) || !live_ranges.count(other_var)) {
+        continue;
+      }
+      // var is not live past this point => don't need to worry
+      if (live_ranges.at(other_var).second <= binding_idx) {
+        continue;
+      }
+      auto other_alias_set = alias_sets.at(other_var);
+      for (int alias_idx : other_alias_set) {
+        for (auto check_set : sets_to_check) {
+          if (check_set.count(alias_idx)) {
+            return false;
+          }
+        }
+      }
+    }
+    return true;
+  } else if (auto* tup_node = target.as<TupleNode>()) {
+    for (auto field : tup_node->fields) {
+      if (!InplaceConditionsMet(live_ranges, alias_sets, tuple_map, currently_live, field,
+                                binding_idx)) {
+        return false;
+      }
+    }
+    return true;
+  } else {
+    return true;
+  }
+}
+
+// this is obviously not a complete list
+static std::unordered_set<std::string> SUPPORTED_OPS = {"relax.add",      "relax.subtract",
+                                                        "relax.multiply", "relax.divide",
+                                                        "relax.nn.silu",  "relax.nn.relu"};
+bool OpSupportsInplace(const Op& op) { return SUPPORTED_OPS.count(op->name); }
+
+// Check for in-place eligibility:
+//  1. see if there's an arg big enough to hold the result
+//  2. see if the arg is live past the call
+//  3. see if the arg has an alias that's live past the call
+// If the conditions are met, record the index of that binding.
+// Returns two lists of lists:
+// 1. A list of bindings where at least one argument meets the in-place conditions and the *size*
+//    matches the size of the result.
+// 2. A list of bindings where at least one argument meets the in-place conditions
+//    and *exactly* matches the shape of the result.
+// For both lists, each element is a list of ints of the following format:
+//   The first element is the index of the *binding* in the block.
+//   All remaining elements are the indices of *eligible arguments* in that call.
+std::pair<std::vector<std::vector<int>>, std::vector<std::vector<int>>> FindInplaceOpportunities(
+    const DataflowBlock& block, const Array<Var>& inputs) {
+  auto live_ranges = AnalyzeLiveness(block);
+  AliasAnalyzer analyzer;
+  auto alias_info = analyzer.Analyze(block, inputs);
+  auto alias_sets = alias_info.first;
+  auto tuple_map = alias_info.second;
+
+  std::vector<std::vector<int>> size_match_list;
+  std::vector<std::vector<int>> exact_match_list;
+
+  // sort the live ranges by starting index
+  std::vector<Var> live_order;
+  for (auto kv : live_ranges) {
+    live_order.push_back(kv.first);
+  }
+  std::sort(live_order.begin(), live_order.end(),
+            [&live_ranges](const Var& var1, const Var& var2) -> bool {
+              return live_ranges[var1].first < live_ranges[var2].first;
+            });
+
+  std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual> currently_live;
+  int last_live = 0;
+
+  for (size_t i = 0; i < block->bindings.size(); i++) {
+    // include all vars that are currently live
+    for (int j = last_live; j < static_cast<int>(live_order.size()); j++) {
+      auto live_var = live_order[j];
+      auto live_range = live_ranges[live_var];
+      if (live_range.first > static_cast<int>(i)) {
+        break;
+      }
+      currently_live.insert(live_var);
+      last_live++;
+    }
+
+    // if we reach a binding check the conditions
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+
+    if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        if (!OpSupportsInplace(GetRef<Op>(op_node))) {
+          continue;
+        }
+
+        std::unordered_set<int> candidates;
+        std::unordered_set<int> exact_match_candidates;
+
+        auto target_sinfo = GatherCandidateSinfo(GetStructInfo(defined_var));
+        // can't be done in-place, ignore
+        if (target_sinfo.empty()) {
+          continue;
+        }
+
+        // Check that at least one argument matches size with the result
+        for (size_t j = 0; j < call_node->args.size(); j++) {
+          auto arg = call_node->args[j];
+          for (auto target : target_sinfo) {
+            std::pair<bool, bool> match = SizeMatches(target, GetStructInfo(arg));
+            if (match.first) {
+              candidates.insert(static_cast<int>(j));
+              if (match.second) {
+                exact_match_candidates.insert(static_cast<int>(j));
+              }
+            }
+          }
+        }
+        if (!candidates.size()) {
+          continue;
+        }
+
+        // Make sure at least one candidate is not live past this point and does not have an alias
+        // live past this point
+        std::unordered_set<int> remove_candidates;
+        for (auto candidate : candidates) {
+          if (!InplaceConditionsMet(live_ranges, alias_sets, tuple_map, currently_live,
+                                    call_node->args[candidate], i)) {
+            remove_candidates.insert(candidate);
+          }
+        }
+        // (remove now to avoid modifying the list as we iterate on it)
+        for (auto candidate : remove_candidates) {
+          candidates.erase(candidate);
+        }
+
+        // if we have a candidate, then this can be made in-place. Report the appropriate candidates
+        if (!candidates.size()) {
+          continue;
+        }
+
+        // produce a list of candidates for this index
+        std::vector<int> size_match_indices = {static_cast<int>(i)};
+        size_match_indices.insert(size_match_indices.end(), candidates.begin(), candidates.end());
+        size_match_list.push_back(size_match_indices);
+
+        // also gather up the exact match candidates if there are any
+        std::vector<int> exact_match_indices = {static_cast<int>(i)};
+        for (auto candidate : candidates) {
+          if (exact_match_candidates.count(candidate)) {
+            exact_match_indices.push_back(candidate);
+          }
+        }
+        if (exact_match_indices.size() > 1) {
+          exact_match_list.push_back(exact_match_indices);
+        }
+      }
+    }
+
+    // remove vars whose range has come to an end
+    // (keep a separate set to avoid changing the sit while iterating on it)
+    std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual> remove;
+    for (auto var : currently_live) {
+      auto live_range = live_ranges[var];
+      if (live_range.second <= static_cast<int>(i)) {
+        remove.insert(var);
+      }
+    }
+    for (auto var : remove) {
+      currently_live.erase(var);
+    }
+  }
+
+  return {size_match_list, exact_match_list};
+}
+
+// Replace buffers in a PrimFunc according to the mapping.
+tir::Stmt RemapBuffers(const tir::Stmt& stmt, const Map<tir::Buffer, tir::Buffer>& buffer_map) {

Review Comment:
   I think this could be handled by calling [`tir::Specialize`](https://github.com/apache/tvm/blob/main/include/tvm/tir/function.h#L267).



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {
+      auto target_var = GetRef<Var>(target_var_node);
+      if (alias_map_.count(target_var)) {
+        ret.insert(alias_map_[target_var].begin(), alias_map_[target_var].end());
+      } else {
+        ret.insert(-1);
+      }
+    } else if (auto* target_tuple = value.as<TupleNode>()) {
+      // fresh idx but we update the tuple map
+      int tup_idx = get_fresh_idx();
+      ret.insert(tup_idx);
+      std::vector<std::unordered_set<int>> new_tuple_map;
+      for (auto field : target_tuple->fields) {
+        new_tuple_map.push_back(GetAliasSet(field, bound_var));
+      }
+      tuple_map_[tup_idx] = new_tuple_map;
+    } else if (auto* target_tgi = value.as<TupleGetItemNode>()) {
+      std::unordered_set<int> tuple_set = GetAliasSet(target_tgi->tuple, bound_var);
+      // if -1 is a member of the tuple set, then we have to assume the result is -1
+      if (tuple_set.count(-1)) {
+        ret.insert(-1);
+      } else {
+        // otherwise, consider all members that are tuples of appropriate size and index into them
+        // (this is safe because the type system will ensure we're not indexing into a tuple
+        // of the wrong size)
+        for (int member : tuple_set) {
+          if (tuple_map_.count(member) &&
+              static_cast<int>(tuple_map_[member].size()) > target_tgi->index) {
+            auto member_set = tuple_map_[member][target_tgi->index];
+            ret.insert(member_set.begin(), member_set.end());
+          }
+        }
+      }
+    } else if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        // call_pure_packed: treat as non-op call
+        if (op_node->name == "relax.call_pure_packed") {
+          return HandleMysteryCall(call_node, bound_var, true);
+        } else if (op_node->name == "relax.split") {
+          // split: Returns a tuple, treat as allocation
+
+          // tuple is freshly allocated, but also add components to the tuple map
+          int tup_idx = get_fresh_idx();
+          ret.insert(tup_idx);
+          // the LHS (the bound var) will definitely have a tuple struct info
+          InsertFreshTuple(tup_idx, GetStructInfoAs<TupleStructInfoNode>(bound_var));
+        } else if (op_node->name == "relax.call_tir") {
+          // call_tir: can potentially return a tuple
+          if (auto* tuple_struct_info = call_node->sinfo_args[0].as<TupleStructInfoNode>()) {
+            int tup_idx = get_fresh_idx();
+            ret.insert(tup_idx);
+            InsertFreshTuple(tup_idx, tuple_struct_info);
+          } else {
+            ret.insert(get_fresh_idx());
+          }
+        } else {
+          // We are assuming most op calls return a single fresh allocation.
+          // We may have to track more exceptions
+          ret.insert(get_fresh_idx());
+        }
+      } else {
+        // assume any non-op call can be extremely dangerous and do anything
+        return HandleMysteryCall(call_node, bound_var);
+      }
+    }
+
+    return ret;
+  }
+
+  std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual> alias_map_;
+  std::unordered_map<int, std::vector<std::unordered_set<int>>> tuple_map_;
+  int mem_idx_;
+};
+
+// given a shape, return the allocation size corresponding to it (product of elements)

Review Comment:
   Calling this function "size" makes it sound like it is the size in bytes.  Can we change this to be `NumElements` instead?



##########
tests/python/relax/test_dataflow_inplace.py:
##########
@@ -0,0 +1,464 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from typing import List, Set, Tuple
+import tvm
+from tvm import relax, testing
+from tvm.relax.transform import DataflowUseInplaceCalls
+from tvm.relax.testing.transform import (
+    dataflow_liveness_analysis,
+    dataflow_alias_analysis,
+    dataflow_inplace_analysis,
+    dataflow_single_inplace_call,
+)
+from tvm.script.parser import ir as I, relax as R, tir as T
+
+import numpy as np
+
+
+def test_liveness_analysis():
+    @I.ir_module
+    class BasicLiveness:
+        @R.function
+        def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            with R.dataflow():
+                y = R.const(1, dtype="int32")
+                z = R.add(x, y)
+                q = R.multiply(z, y)
+                p = R.add(z, q)
+                n = R.multiply(p, p)
+                R.output(n)
+            return n
+
+    block = BasicLiveness["main"].body.blocks[0]
+    live_ranges = dataflow_liveness_analysis(block)
+    expected_ranges = {
+        "x": (-1, 1),
+        "y": (0, 2),
+        "z": (1, 3),
+        "q": (2, 3),
+        "p": (3, 4),
+        "n": (4, 5),
+    }
+    for var, live_range in live_ranges.items():
+        assert live_range == expected_ranges[var.name_hint]
+
+
+def test_alias_analysis_basic():
+    @I.ir_module
+    class BasicAliasAnalysis:
+        @R.function
+        def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            with R.dataflow():
+                y = x  # y is an alias of x
+                z = R.add(y, y)  # fresh value
+                n = z  # alias of z
+                R.output(n)
+            return n
+
+    block = BasicAliasAnalysis["main"].body.blocks[0]
+    alias_sets, tuple_map = dataflow_alias_analysis(block, BasicAliasAnalysis["main"].params)
+    expected = {
+        "x": {0},
+        "y": {0},
+        "z": {1},
+        "n": {1},
+    }
+
+    for var, alias_set in alias_sets.items():
+        assert alias_set == expected[var.name_hint]
+    assert tuple_map == {}
+
+
+def test_alias_analysis_tuple():
+    @I.ir_module
+    class AliasesWithTuples:
+        @R.function
+        def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            with R.dataflow():
+                y = R.const(1, dtype="int32")
+                t = (x, y)
+                a = t[0]
+                b = t[1]
+                c = t[0]
+                d = t[1]
+                u = t
+                e = t[0]
+                f = t[1]
+                z = R.add(c, d)
+                n = z
+                R.output(n)
+            return n
+
+    block = AliasesWithTuples["main"].body.blocks[0]
+    alias_sets, tuple_map = dataflow_alias_analysis(block, AliasesWithTuples["main"].params)
+    expected = {
+        "x": {0},
+        "y": {1},
+        "t": {2},
+        "a": {0},
+        "b": {1},
+        "c": {0},
+        "d": {1},
+        "u": {2},
+        "e": {0},
+        "f": {1},
+        "z": {3},
+        "n": {3},
+    }
+
+    for var, alias_set in alias_sets.items():
+        assert alias_set == expected[var.name_hint]
+    assert 2 in tuple_map
+    assert tuple_map[2] == [{0}, {1}]
+
+
+def test_alias_split():
+    @I.ir_module
+    class AliasSplit:
+        @R.function
+        def main(x: R.Tensor((60,), "int32")) -> R.Tensor((15,), "int32"):
+            with R.dataflow():
+                t = R.split(x, 4)
+                y = t[0]
+                z = t[1]
+                q = t[2]
+                p = t[3]
+                n = z
+                R.output(n)
+            return n
+
+    block = AliasSplit["main"].body.blocks[0]
+    alias_sets, tuple_map = dataflow_alias_analysis(block, AliasSplit["main"].params)
+    expected = {
+        "x": {0},
+        "t": {1},
+        "y": {2},
+        "z": {3},
+        "q": {4},
+        "p": {5},
+        "n": {3},
+    }
+
+    for var, alias_set in alias_sets.items():
+        assert alias_set == expected[var.name_hint]
+    assert len(tuple_map) == 1
+    assert 1 in tuple_map
+    assert tuple_map[1] == [{2}, {3}, {4}, {5}]
+
+
+def test_alias_call_tir():
+    # call TIR can yield either a single tensor or a tuple
+    @I.ir_module
+    class AliasCallTir:
+        @T.prim_func
+        def tir_id(x: T.handle, y: T.handle) -> None:
+            T.func_attr({"global_symbol": "tir_id"})
+            m = T.int32()
+            n = T.int32()
+            A = T.match_buffer(x, (m, n))
+            B = T.match_buffer(y, (m, n))
+
+            for i, j in T.grid(m, n):
+                with T.block("id"):
+                    vi, vj = T.axis.remap("SS", [i, j])
+                    B[vi, vj] = A[vi, vj]
+
+        @T.prim_func
+        def tir_id2(x: T.handle, y: T.handle, z: T.handle) -> None:
+            T.func_attr({"global_symbol": "tir_id"})
+            m = T.int32()
+            n = T.int32()
+            A = T.match_buffer(x, (m, n))
+            B = T.match_buffer(y, (m, n))
+            C = T.match_buffer(z, (m, n))
+
+            for i, j in T.grid(m, n):
+                with T.block("id"):
+                    vi, vj = T.axis.remap("SS", [i, j])
+                    B[vi, vj] = A[vi, vj]
+                    C[vi, vj] = A[vi, vj]
+
+        @R.function
+        def main(x: R.Tensor((10, 10), "int32")) -> R.Tensor((10, 10), "int32"):
+            with R.dataflow():
+                cls = AliasCallTir
+                y = R.call_tir(cls.tir_id, (x,), out_sinfo=R.Tensor((10, 10), "int32"))
+                t = R.call_tir(
+                    cls.tir_id2,
+                    (y,),
+                    out_sinfo=[R.Tensor((10, 10), "int32"), R.Tensor((10, 10), "int32")],
+                )
+                z = y
+                p = t[0]
+                q = t[1]
+                u = t
+                m = u[0]
+                n = u[1]
+                v = n
+                R.output(v)
+            return v
+
+    block = AliasCallTir["main"].body.blocks[0]
+    alias_sets, tuple_map = dataflow_alias_analysis(block, AliasCallTir["main"].params)
+    expected = {
+        "x": {0},
+        "y": {1},
+        "t": {2},
+        "z": {1},
+        "p": {3},
+        "q": {4},
+        "u": {2},
+        "m": {3},
+        "n": {4},
+        "v": {4},
+    }
+
+    for var, alias_set in alias_sets.items():
+        assert alias_set == expected[var.name_hint]
+    assert len(tuple_map) == 1
+    assert 2 in tuple_map
+    assert tuple_map[2] == [{3}, {4}]
+
+
+def test_mystery_calls():
+    @I.ir_module
+    class AliasChaosCalls:
+        @R.function
+        def identity(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            return x
+
+        @R.function
+        def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            with R.dataflow():
+                cls = AliasChaosCalls
+                y = cls.identity(x)
+                z = cls.identity(y)
+                m = R.const(1, dtype="int32")
+                n = R.const(2, dtype="int32")
+                t = (m, n)
+                a = R.call_pure_packed(
+                    "chaos", t, sinfo_args=R.Tuple(R.Tensor((), "int32"), R.Tensor((), "int32"))
+                )
+                b = a[0]
+                c = a[1]
+                R.output(c)
+            return c
+
+    block = AliasChaosCalls["main"].body.blocks[0]
+    alias_sets, tuple_map = dataflow_alias_analysis(block, AliasChaosCalls["main"].params)
+    expected = {
+        "x": {0},
+        "y": {0, 1},
+        "z": {0, 1, 2},
+        "m": {3},
+        "n": {4},
+        "t": {5},
+        "a": {3, 4, 5, 6, 7, 8},  # either t or a fresh tuple
+        "b": {3, 4, 5, 6, 7, 8},  # the tuple components can be aliased to any member...
+        "c": {3, 4, 5, 6, 7, 8},  # the tuple components can be aliased to any member...
+        # (in principle, we can use type information to narrow down the aliasing)
+    }
+
+    for var, alias_set in alias_sets.items():
+        assert alias_set == expected[var.name_hint]
+    assert len(tuple_map) == 2
+    assert 5 in tuple_map
+    assert tuple_map[5] == [{3}, {4}]
+    assert 6 in tuple_map
+    assert tuple_map[6] == [{3, 4, 5, 6, 7, 8}, {3, 4, 5, 6, 7, 8}]
+
+
+def test_alias_external_value():
+    @I.ir_module
+    class AliasExternalValue:
+        @R.function
+        def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            y = R.const(1, dtype="int32")  # not in DF block, treated as external
+            t1 = (y, y)  # not in DF block, treated as external
+            with R.dataflow():
+                z = y  # mystery value
+                a = R.const(2, dtype="int32")
+                t2 = (z, a)
+                b = t2[0]
+                c = t1[1]  # tuple index into external value
+                R.output(b)
+            return b
+
+    block = AliasExternalValue["main"].body.blocks[1]
+    alias_sets, tuple_map = dataflow_alias_analysis(block, AliasExternalValue["main"].params)
+    expected = {
+        "x": {0},
+        "z": {-1},
+        "a": {1},
+        "t2": {2},
+        "b": {-1},
+        "c": {-1},
+    }
+
+    for var, alias_set in alias_sets.items():
+        assert alias_set == expected[var.name_hint]
+    assert len(tuple_map) == 1
+    assert 2 in tuple_map
+    assert tuple_map[2] == [{-1}, {1}]
+
+
+def test_inplace_simple_case():
+    @I.ir_module
+    class InplaceBasic:
+        @R.function
+        def main(
+            x: R.Tensor((2, 3), "int32"), y: R.Tensor((2, 3), "int32")
+        ) -> R.Tensor((2, 3), "int32"):
+            with R.dataflow():
+                z = R.add(x, y)  # cannot be done inplace: x and y are live later
+                q = R.multiply(x, y)  # can be done inplace: neither x nor y is used later
+                p = R.add(z, q)  # can be done inplace: neither z nor q is used later
+                r = p  # alias of p
+                m = R.multiply(p, p)  # p is not used later but r is, so can't do inplace
+                n = R.add(m, r)  # can be done inplace: r is not used again
+                ret = R.subtract(n, m)  # can be done inplace: neither is used again
+                R.output(ret)
+            return ret
+
+    block = InplaceBasic["main"].body.blocks[0]
+    size_match, exact_match = dataflow_inplace_analysis(block, InplaceBasic["main"].params)
+
+    # order does not matter for the listing of candidates, so we have to implement as sets
+    def assert_candidate_list(
+        actual: List[List[int]], expected: List[Tuple[int, Set[int]]]
+    ) -> None:
+        assert len(actual) == len(expected)
+        for i in range(len(actual)):
+            assert actual[i][0] == expected[i][0]
+            assert len(expected[i][1]) == len(actual[i]) - 1
+            for j in range(len(expected[i][1])):
+                assert actual[i][j + 1] in expected[i][1]
+
+    assert_candidate_list(size_match, [(1, {0, 1}), (2, {0, 1}), (5, {1}), (6, {0, 1})])
+    # TODO(@slyubomirsky): I couldn't think of an easy example where sizes don't match,
+    # but broadcasting might cause it to happen
+    assert_candidate_list(exact_match, [(1, {0, 1}), (2, {0, 1}), (5, {1}), (6, {0, 1})])
+
+
+def test_inplace_single_call():
+    @I.ir_module
+    class TestModule:
+        @R.function
+        def main(
+            x: R.Tensor((2, 3), dtype="float32"), y: R.Tensor((2, 3), dtype="float32")
+        ) -> R.Tensor((2, 3), dtype="float32"):
+            z = R.add(x, y)
+            q = R.nn.silu(z)
+            return q
+
+    add_call = TestModule["main"].body.blocks[0].bindings[0].value
+    new_add, new_mod = dataflow_single_inplace_call(TestModule, add_call, [0])
+
+    @T.prim_func(private=True)
+    def expected_add(
+        A: T.Buffer((T.int64(2), T.int64(3)), "float32"),
+        B: T.Buffer((T.int64(2), T.int64(3)), "float32"),
+    ):
+        T.func_attr({"tir.noalias": T.bool(True)})
+        for ax0, ax1 in T.grid(T.int64(2), T.int64(3)):
+            with T.block("T_add"):
+                v_ax0, v_ax1 = T.axis.remap("SS", [ax0, ax1])
+                T.reads(A[v_ax0, v_ax1], B[v_ax0, v_ax1])
+                T.writes(A[v_ax0, v_ax1])
+                A[v_ax0, v_ax1] = A[v_ax0, v_ax1] + B[v_ax0, v_ax1]
+
+    tvm.ir.assert_structural_equal(new_mod["add_inplace"], expected_add)
+    assert new_add.op.name == "relax.call_tir_inplace"
+    assert new_add.args[0].name_hint == "add_inplace"
+    for i, arg in enumerate(new_add.args[1].fields):
+        arg == add_call.args[i]
+    new_add.attrs.inplace_indices == [0]
+
+    @T.prim_func(private=True)
+    def expected_silu(A: T.Buffer((T.int64(2), T.int64(3)), "float32")):
+        T.func_attr({"tir.noalias": T.bool(True)})
+        compute = T.alloc_buffer((T.int64(2), T.int64(3)))
+        for i0, i1 in T.grid(T.int64(2), T.int64(3)):
+            with T.block("compute"):
+                v_i0, v_i1 = T.axis.remap("SS", [i0, i1])
+                T.reads(A[v_i0, v_i1])
+                T.writes(compute[v_i0, v_i1])
+                compute[v_i0, v_i1] = T.sigmoid(A[v_i0, v_i1])
+        for ax0, ax1 in T.grid(T.int64(2), T.int64(3)):
+            with T.block("T_multiply"):
+                v_ax0, v_ax1 = T.axis.remap("SS", [ax0, ax1])
+                T.reads(A[v_ax0, v_ax1], compute[v_ax0, v_ax1])
+                T.writes(A[v_ax0, v_ax1])
+                A[v_ax0, v_ax1] = A[v_ax0, v_ax1] * compute[v_ax0, v_ax1]
+
+    silu_call = TestModule["main"].body.blocks[0].bindings[1].value
+    new_silu, new_mod = dataflow_single_inplace_call(TestModule, silu_call, [0])
+
+    tvm.ir.assert_structural_equal(new_mod["silu_inplace"], expected_silu)
+    assert new_silu.op.name == "relax.call_tir_inplace"
+    assert new_silu.args[0].name_hint == "silu_inplace"
+    for i, arg in enumerate(new_silu.args[1].fields):
+        arg == silu_call.args[i]
+    new_silu.attrs.inplace_indices == [0]
+
+
+def test_insert_inplace_calls():
+    @I.ir_module
+    class EndToEndTest:
+        @R.function
+        def main(
+            x: R.Tensor((2, 3), dtype="float32"), y: R.Tensor((1, 3), dtype="float32")
+        ) -> R.Tensor((2, 3), dtype="float32"):
+            with R.dataflow():
+                z = R.add(x, y)  # broadcast happens here
+                # Cannot be done in-place because x is an argument.
+                a = R.add(z, y)  # this one can be done in-place
+                q = R.multiply(a, y)  # broadcast again, a is eligible
+                r = R.subtract(y, y)  # cannot be done in-place because y is an argument
+                s = R.subtract(r, r)  # No broadcast. Can be done in-place
+                m = R.multiply(q, s)  # should give us all zeros
+                R.output(m)
+            return m
+
+    transform_pass = DataflowUseInplaceCalls()
+    new_mod = transform_pass(EndToEndTest)

Review Comment:
   Can we add an expected output as well, with `assert_structural_equal`?



##########
src/relax/transform/dataflow_inplace.cc:
##########
@@ -0,0 +1,1017 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ *
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*!
+ * \file src/relax/transform/dataflow_inplace.cc
+ * \brief Pass that converts eligible operator calls in dataflow blocks
+ *   into in-place versions.
+ */
+
+#include <tvm/ir/transform.h>
+#include <tvm/relax/analysis.h>
+#include <tvm/relax/attrs/op.h>
+#include <tvm/relax/expr.h>
+#include <tvm/relax/expr_functor.h>
+#include <tvm/relax/transform.h>
+#include <tvm/relax/utils.h>
+#include <tvm/tir/stmt_functor.h>
+
+#include "utils.h"
+
+namespace tvm {
+namespace relax {
+
+// Perform liveness analysis on a dataflow block, returning a map of vars to
+// pairs of indices (the liveness interval, from the starting index to the end index).
+// A starting index of -1 means the var is defined before the block starts and an end index
+// of block->bindings.size() (one past the last index) means it is live after the block ends.
+std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> AnalyzeLiveness(
+    const DataflowBlock& block) {
+  std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual> ret;
+  for (int i = block->bindings.size() - 1; i >= 0; i--) {
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+    Array<Var> used_vars;
+    // for a function literal, we consider only the free vars
+    // (those captured from the outer scope)
+    if (value.as<FunctionNode>()) {
+      used_vars = FreeVars(value);
+    } else if (value.as<TupleGetItemNode>()) {
+      // Special case: we do not consider a tuple index to be a "use."
+      // This is a bit of a hack but allows us to do operations that
+      // create tuples to be done in-place (otherwise, any index of the tuple
+      // would be considered a use and so the tuple would be live later).
+      // Hence we keep the array empty.
+    } else {
+      used_vars = AllVars(value);
+    }
+
+    for (auto var : used_vars) {
+      if (!ret.count(var)) {
+        ret[var] = {-1, i};
+      }
+    }
+
+    if (!ret.count(defined_var)) {
+      // if it's an output, then it lives past the end of the block
+      if (!defined_var.as<DataflowVarNode>()) {
+        ret[defined_var] = {i, block->bindings.size()};
+      } else {
+        // otherwise, it's live only here
+        ret[defined_var] = {i, i};
+      }
+    } else {
+      // this means the var is used later but we encountered its definition now
+      auto last_range = ret[defined_var];
+      CHECK_EQ(last_range.first, -1);
+      std::pair<int, int> new_range = {i, last_range.second};
+      ret[defined_var] = new_range;
+    }
+  }
+  return ret;
+}
+
+class AliasAnalyzer {
+ public:
+  AliasAnalyzer() : alias_map_(), tuple_map_(), mem_idx_(0) {}
+
+  // The analysis returns a map of vars to memory locations that it *could* map to
+  // (any unique allocation = one memory location), plus a map of memory locations
+  // that correspond to tuples (this maps to sets of memory locations for each tuple element).
+  // Note: inputs are values that should be assumed not to be aliased and are therefore
+  // (in the case of in-place ops) safe to overwrite. This may not be true of function args.
+  std::pair<std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>,
+            std::unordered_map<int, std::vector<std::unordered_set<int>>>>
+  Analyze(const DataflowBlock& block, const Array<Var>& inputs) {
+    for (auto input : inputs) {
+      int curr_idx = get_fresh_idx();
+      alias_map_[input] = {curr_idx};
+      if (auto* tup_info = GetStructInfoAs<TupleStructInfoNode>(input)) {
+        InsertFreshTuple(curr_idx, tup_info);
+      }
+    }
+
+    for (const Binding& binding : block->bindings) {
+      Var current_var = binding->var;
+      Expr value = GetBoundValue(binding);
+      alias_map_[current_var] = GetAliasSet(value, current_var);
+    }
+
+    return {alias_map_, tuple_map_};
+  }
+
+ private:
+  int get_fresh_idx() {
+    int ret = mem_idx_;
+    mem_idx_++;
+    return ret;
+  }
+
+  // Fresh tuple = each element is assumed to be a unique allocation
+  void InsertFreshTuple(int tup_idx, const TupleStructInfoNode* tup_info) {
+    std::vector<std::unordered_set<int>> tuple_set;
+    for (int i = 0; i < static_cast<int>(tup_info->fields.size()); i++) {
+      int curr_field = get_fresh_idx();
+      tuple_set.push_back({curr_field});
+      if (auto* nested_tup_info = tup_info->fields[i].as<TupleStructInfoNode>()) {
+        InsertFreshTuple(curr_field, nested_tup_info);
+      }
+    }
+    tuple_map_[tup_idx] = tuple_set;
+  }
+
+  // given a tuple index, add the given memory location indices to each component's
+  // alias set
+  void UpdateTupleComponents(int tup_idx, const std::unordered_set<int>& insert_idxs) {
+    if (tuple_map_.count(tup_idx)) {
+      auto tuple_comps = tuple_map_[tup_idx];
+      for (size_t i = 0; i < tuple_comps.size(); i++) {
+        auto comp_set = tuple_comps[i];
+
+        // if a member is a tuple, update its components as well
+        for (int member : comp_set) {
+          if (tuple_map_.count(member)) {
+            UpdateTupleComponents(member, insert_idxs);
+          }
+        }
+
+        // update after iterating to avoid iterating over the inserted elements
+        tuple_map_[tup_idx][i].insert(insert_idxs.begin(), insert_idxs.end());
+      }
+    }
+  }
+
+  // capture the given index and also its tuple components (including recursively)
+  // if they exist
+  void AddCapturedIndices(std::unordered_set<int>* captured_set, int idx) {
+    captured_set->insert(idx);
+    if (tuple_map_.count(idx)) {
+      for (auto comp_set : tuple_map_[idx]) {
+        for (auto tup_comp_idx : comp_set) {
+          AddCapturedIndices(captured_set, tup_comp_idx);
+        }
+      }
+    }
+  }
+
+  // Conservative extremely pessimistic assumption:
+  // assume that the result of a non-op call can be aliased to any argument
+  // or that it could be a newly allocated value.
+  // For tuples, assume all members are aliased. Yeah, it's bad.
+  // (Skip first arg is for handling call_pure_packed, where the first arg is an ExternFunc that we
+  // should ignore)
+  std::unordered_set<int> HandleMysteryCall(const CallNode* call_node, const Var& bound_var,
+                                            bool skip_first_arg = false) {
+    // the result may or may not be newly allocated
+    std::unordered_set<int> ret;
+    int res_idx = get_fresh_idx();
+    // the result may be a tuple
+    if (auto* tup_info_node = GetStructInfoAs<TupleStructInfoNode>(bound_var)) {
+      InsertFreshTuple(res_idx, tup_info_node);
+    }
+    AddCapturedIndices(&ret, res_idx);
+
+    for (size_t i = (skip_first_arg) ? 1 : 0; i < call_node->args.size(); i++) {
+      auto arg = call_node->args[i];
+      auto arg_alias_set = GetAliasSet(arg, bound_var);
+      for (int alias_idx : arg_alias_set) {
+        AddCapturedIndices(&ret, alias_idx);
+      }
+    }
+    // if the result is a tuple, the components can also potentially be aliased to any arg
+    // or, in fact, to each other
+    UpdateTupleComponents(res_idx, ret);
+    return ret;
+  }
+
+  // given the expression value, return the set of memory locations corresponding to it
+  // (the var the expression is being bound to is needed for struct info)
+  std::unordered_set<int> GetAliasSet(const Expr& value, const Var& bound_var) {
+    std::unordered_set<int> ret;
+
+    // cases for value:
+    // constant: it's a fresh index
+    // var: look up in alias map (-1 if not present)
+    // op call: assume it's fresh (may need to make list of exceptions)
+    // tuple: fresh entry in tuple index, recurse to determine indices for values
+    // function/packed call: chaos reigns, alias with any other argument
+    //   (if tuple is passed, assume also aliased with all members of the tuple)
+    // tuple index: -1 if tuple is not in tuple map, otherwise look up corresponding entry
+    // function constant: give them a fresh index (TODO: we can handle in more detail if this is a
+    // case we need to support) prim value: fresh index if node: should not happen inside dataflow
+    // block
+    if (value.as<ConstantNode>() || value.as<PrimValueNode>() || value.as<FunctionNode>()) {
+      // TODO(@slyubomirsky): We will probably want special handling for closures
+      ret.insert(get_fresh_idx());
+    } else if (auto* target_var_node = value.as<VarNode>()) {
+      auto target_var = GetRef<Var>(target_var_node);
+      if (alias_map_.count(target_var)) {
+        ret.insert(alias_map_[target_var].begin(), alias_map_[target_var].end());
+      } else {
+        ret.insert(-1);
+      }
+    } else if (auto* target_tuple = value.as<TupleNode>()) {
+      // fresh idx but we update the tuple map
+      int tup_idx = get_fresh_idx();
+      ret.insert(tup_idx);
+      std::vector<std::unordered_set<int>> new_tuple_map;
+      for (auto field : target_tuple->fields) {
+        new_tuple_map.push_back(GetAliasSet(field, bound_var));
+      }
+      tuple_map_[tup_idx] = new_tuple_map;
+    } else if (auto* target_tgi = value.as<TupleGetItemNode>()) {
+      std::unordered_set<int> tuple_set = GetAliasSet(target_tgi->tuple, bound_var);
+      // if -1 is a member of the tuple set, then we have to assume the result is -1
+      if (tuple_set.count(-1)) {
+        ret.insert(-1);
+      } else {
+        // otherwise, consider all members that are tuples of appropriate size and index into them
+        // (this is safe because the type system will ensure we're not indexing into a tuple
+        // of the wrong size)
+        for (int member : tuple_set) {
+          if (tuple_map_.count(member) &&
+              static_cast<int>(tuple_map_[member].size()) > target_tgi->index) {
+            auto member_set = tuple_map_[member][target_tgi->index];
+            ret.insert(member_set.begin(), member_set.end());
+          }
+        }
+      }
+    } else if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        // call_pure_packed: treat as non-op call
+        if (op_node->name == "relax.call_pure_packed") {
+          return HandleMysteryCall(call_node, bound_var, true);
+        } else if (op_node->name == "relax.split") {
+          // split: Returns a tuple, treat as allocation
+
+          // tuple is freshly allocated, but also add components to the tuple map
+          int tup_idx = get_fresh_idx();
+          ret.insert(tup_idx);
+          // the LHS (the bound var) will definitely have a tuple struct info
+          InsertFreshTuple(tup_idx, GetStructInfoAs<TupleStructInfoNode>(bound_var));
+        } else if (op_node->name == "relax.call_tir") {
+          // call_tir: can potentially return a tuple
+          if (auto* tuple_struct_info = call_node->sinfo_args[0].as<TupleStructInfoNode>()) {
+            int tup_idx = get_fresh_idx();
+            ret.insert(tup_idx);
+            InsertFreshTuple(tup_idx, tuple_struct_info);
+          } else {
+            ret.insert(get_fresh_idx());
+          }
+        } else {
+          // We are assuming most op calls return a single fresh allocation.
+          // We may have to track more exceptions
+          ret.insert(get_fresh_idx());
+        }
+      } else {
+        // assume any non-op call can be extremely dangerous and do anything
+        return HandleMysteryCall(call_node, bound_var);
+      }
+    }
+
+    return ret;
+  }
+
+  std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual> alias_map_;
+  std::unordered_map<int, std::vector<std::unordered_set<int>>> tuple_map_;
+  int mem_idx_;
+};
+
+// given a shape, return the allocation size corresponding to it (product of elements)
+int ShapeSize(const ShapeExpr& shape) {
+  int ret = 1;
+  for (auto dim : shape->values) {
+    if (auto int_dim = dim.as<IntImmNode>()) {
+      ret *= static_cast<int>(int_dim->value);
+    } else {
+      return -1;
+    }
+  }
+  return ret;
+}
+
+// Given the struct info of the result, return any struct info nested in it
+// that is eleigible to be used for in-place computations (tensors are eligible
+// only if all their dimensions are integer constants, tuples are eligible if
+// all members are eligible though we can consider only individual members separately)
+std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> GatherCandidateSinfo(
+    const StructInfo& result_sinfo) {
+  if (auto* tensor_info = result_sinfo.as<TensorStructInfoNode>()) {
+    // don't consider void dtype (don't know the size at compile time)
+    if (tensor_info->dtype.is_void()) {
+      return {};
+    }
+    // don't consider cases where we don't know the shape at compile time
+    // TODO(@slyubomirsky): variables might be okay if we use the arithmetic analyzer
+    if (auto* shape_node = tensor_info->shape.as<ShapeExprNode>()) {
+      for (auto dim : shape_node->values) {
+        if (!dim.as<IntImmNode>()) {
+          return {};
+        }
+      }
+      return {GetRef<TensorStructInfo>(tensor_info)};
+    } else {
+      return {};
+    }
+  } else if (auto* tuple_info = result_sinfo.as<TupleStructInfoNode>()) {
+    // we can see if the whole tuple matches or go for any of the components
+    std::unordered_set<StructInfo, ObjectPtrHash, ObjectPtrEqual> ret;
+    for (auto field : tuple_info->fields) {
+      auto field_candidates = GatherCandidateSinfo(field);
+      ret.insert(field_candidates.begin(), field_candidates.end());
+    }
+    // at least one field should be eligible to be done in-place
+    if (!ret.empty()) {
+      ret.insert(GetRef<StructInfo>(tuple_info));
+    }
+    return ret;
+  } else {
+    // don't consider any other types
+    return {};
+  }
+}
+
+// Given the two struct info, return a pair of bools where the first element is true if
+// the two struct info are the same _size_ in memory and the second element is true
+// if the shapes match _exactly_. Performs this check recursively and ensures the
+// stated condition is true for all tensor members of the struct info (return false
+// if a single pair of corresponding tensors does not meet the condition).
+std::pair<bool, bool> SizeMatches(const StructInfo& target_info, const StructInfo& arg_info) {
+  if (target_info.as<TensorStructInfoNode>() && arg_info.as<TensorStructInfoNode>()) {
+    auto target_tensor = Downcast<TensorStructInfo>(target_info);
+    auto arg_tensor = Downcast<TensorStructInfo>(arg_info);
+    if (target_tensor->shape.defined() && target_tensor->shape.as<ShapeExprNode>() &&
+        arg_tensor->shape.defined() && arg_tensor->shape.as<ShapeExprNode>()) {
+      if (target_tensor->dtype != arg_tensor->dtype) {
+        return {false, false};
+      }
+      auto target_shape = Downcast<ShapeExpr>(target_tensor->shape);
+      auto arg_shape = Downcast<ShapeExpr>(arg_tensor->shape);
+      int target_size = ShapeSize(target_shape);
+      int arg_size = ShapeSize(arg_shape);
+      if (target_size == -1 || arg_size == -1 || target_size < arg_size) {
+        return {false, false};
+      }
+      // exact match: number of dims and each dim matches
+      if (target_shape->values.size() == arg_shape->values.size()) {
+        for (size_t i = 0; i < target_shape->values.size(); i++) {
+          if (!arg_shape->values[i].as<IntImmNode>()) {
+            return {false, false};
+          }
+          if (Downcast<IntImm>(target_shape->values[i])->value !=
+              Downcast<IntImm>(arg_shape->values[i])->value) {
+            return {true, false};
+          }
+        }
+        return {true, true};
+      }
+      return {true, false};
+    } else {
+      return {false, false};
+    }
+  } else if (target_info.as<TupleStructInfoNode>() && arg_info.as<TupleStructInfoNode>()) {
+    auto target_tup = Downcast<TupleStructInfo>(target_info);
+    auto arg_tup = Downcast<TupleStructInfo>(arg_info);
+    if (target_tup->fields.size() != arg_tup->fields.size()) {
+      return {false, false};
+    }
+    bool all_exact = true;
+    for (size_t i = 0; i < target_tup->fields.size(); i++) {
+      auto element_match = SizeMatches(target_tup->fields[i], arg_tup->fields[i]);
+      if (!element_match.first) {
+        return {false, false};
+      }
+      if (!element_match.second) {
+        all_exact = false;
+      }
+    }
+    return {true, all_exact};
+  } else if (target_info.as<PrimStructInfoNode>() && arg_info.as<PrimStructInfoNode>()) {
+    return {true, true};
+  } else {
+    return {false, false};
+  }
+}
+
+// Given an alias index, check if it's a tuple and gather the sets of aliases for the tuple
+// members if so (apply recursively if any of those members are tuples).
+// Return false if the alias set contains -1, meaning a reference to an unknown or
+// possibly dangerous value (no checking we can do for that).
+bool GatherSetsToCheckForLiveness(
+    const std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>&
+        alias_sets,
+    const std::unordered_map<int, std::vector<std::unordered_set<int>>>& tuple_map,
+    std::vector<std::unordered_set<int>>* sets_to_check, int alias_idx) {
+  if (tuple_map.count(alias_idx)) {
+    for (auto member_set : tuple_map.at(alias_idx)) {
+      // contains -1 -> unknown and dangerous, we can short-circuit
+      if (member_set.count(-1)) {
+        return false;
+      }
+      sets_to_check->push_back(member_set);
+
+      // if a member can be a tuple, check it recursively
+      for (int member : member_set) {
+        if (tuple_map.count(member)) {
+          if (!GatherSetsToCheckForLiveness(alias_sets, tuple_map, sets_to_check, member)) {
+            return false;
+          }
+        }
+      }
+    }
+  }
+  return true;
+}
+
+// Check that the target is not live past the index and that no alias of it is live past the
+// binding index (if the target is a tuple, check the conditions recursively for the members)
+bool InplaceConditionsMet(
+    const std::unordered_map<Var, std::pair<int, int>, ObjectPtrHash, ObjectPtrEqual>& live_ranges,
+    const std::unordered_map<Var, std::unordered_set<int>, ObjectPtrHash, ObjectPtrEqual>&
+        alias_sets,
+    const std::unordered_map<int, std::vector<std::unordered_set<int>>>& tuple_map,
+    const std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual>& currently_live,
+    const Expr& target, int binding_idx) {
+  if (auto* var_node = target.as<VarNode>()) {
+    auto current_var = GetRef<Var>(var_node);
+    // if the var is live past this point, we can't use it for in-place computations anyway
+    if (live_ranges.count(current_var)) {
+      auto live_range = live_ranges.at(current_var);
+      if (live_range.second > binding_idx) {
+        return false;
+      }
+    }
+
+    // no entry for the current var -> it must be something external and we have to assume the worst
+    if (!alias_sets.count(current_var)) {
+      return false;
+    }
+    auto alias_set = alias_sets.at(current_var);
+    // -1 -> an external value and we must assume the worst
+    if (alias_set.count(-1)) {
+      return false;
+    }
+    std::vector<std::unordered_set<int>> sets_to_check = {alias_set};
+    std::unordered_set<int> indices_checked;
+    // If a possible alias is a tuple, we will also check for aliases of the members
+    // (possibly recursively)
+    for (int alias_idx : alias_set) {
+      if (!GatherSetsToCheckForLiveness(alias_sets, tuple_map, &sets_to_check, alias_idx)) {
+        return false;
+      }
+    }
+
+    for (Var other_var : currently_live) {
+      if (other_var.same_as(target)) {
+        continue;
+      }
+      // not represented = spooky unknown value that should be modeled by -1
+      if (!alias_sets.count(other_var) || !live_ranges.count(other_var)) {
+        continue;
+      }
+      // var is not live past this point => don't need to worry
+      if (live_ranges.at(other_var).second <= binding_idx) {
+        continue;
+      }
+      auto other_alias_set = alias_sets.at(other_var);
+      for (int alias_idx : other_alias_set) {
+        for (auto check_set : sets_to_check) {
+          if (check_set.count(alias_idx)) {
+            return false;
+          }
+        }
+      }
+    }
+    return true;
+  } else if (auto* tup_node = target.as<TupleNode>()) {
+    for (auto field : tup_node->fields) {
+      if (!InplaceConditionsMet(live_ranges, alias_sets, tuple_map, currently_live, field,
+                                binding_idx)) {
+        return false;
+      }
+    }
+    return true;
+  } else {
+    return true;
+  }
+}
+
+// this is obviously not a complete list
+static std::unordered_set<std::string> SUPPORTED_OPS = {"relax.add",      "relax.subtract",
+                                                        "relax.multiply", "relax.divide",
+                                                        "relax.nn.silu",  "relax.nn.relu"};
+bool OpSupportsInplace(const Op& op) { return SUPPORTED_OPS.count(op->name); }
+
+// Check for in-place eligibility:
+//  1. see if there's an arg big enough to hold the result
+//  2. see if the arg is live past the call
+//  3. see if the arg has an alias that's live past the call
+// If the conditions are met, record the index of that binding.
+// Returns two lists of lists:
+// 1. A list of bindings where at least one argument meets the in-place conditions and the *size*
+//    matches the size of the result.
+// 2. A list of bindings where at least one argument meets the in-place conditions
+//    and *exactly* matches the shape of the result.
+// For both lists, each element is a list of ints of the following format:
+//   The first element is the index of the *binding* in the block.
+//   All remaining elements are the indices of *eligible arguments* in that call.
+std::pair<std::vector<std::vector<int>>, std::vector<std::vector<int>>> FindInplaceOpportunities(
+    const DataflowBlock& block, const Array<Var>& inputs) {
+  auto live_ranges = AnalyzeLiveness(block);
+  AliasAnalyzer analyzer;
+  auto alias_info = analyzer.Analyze(block, inputs);
+  auto alias_sets = alias_info.first;
+  auto tuple_map = alias_info.second;
+
+  std::vector<std::vector<int>> size_match_list;
+  std::vector<std::vector<int>> exact_match_list;
+
+  // sort the live ranges by starting index
+  std::vector<Var> live_order;
+  for (auto kv : live_ranges) {
+    live_order.push_back(kv.first);
+  }
+  std::sort(live_order.begin(), live_order.end(),
+            [&live_ranges](const Var& var1, const Var& var2) -> bool {
+              return live_ranges[var1].first < live_ranges[var2].first;
+            });
+
+  std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual> currently_live;
+  int last_live = 0;
+
+  for (size_t i = 0; i < block->bindings.size(); i++) {
+    // include all vars that are currently live
+    for (int j = last_live; j < static_cast<int>(live_order.size()); j++) {
+      auto live_var = live_order[j];
+      auto live_range = live_ranges[live_var];
+      if (live_range.first > static_cast<int>(i)) {
+        break;
+      }
+      currently_live.insert(live_var);
+      last_live++;
+    }
+
+    // if we reach a binding check the conditions
+    Binding b = block->bindings[i];
+    Var defined_var = b->var;
+    Expr value = GetBoundValue(b);
+
+    if (auto* call_node = value.as<CallNode>()) {
+      if (auto* op_node = call_node->op.as<OpNode>()) {
+        if (!OpSupportsInplace(GetRef<Op>(op_node))) {
+          continue;
+        }
+
+        std::unordered_set<int> candidates;
+        std::unordered_set<int> exact_match_candidates;
+
+        auto target_sinfo = GatherCandidateSinfo(GetStructInfo(defined_var));
+        // can't be done in-place, ignore
+        if (target_sinfo.empty()) {
+          continue;
+        }
+
+        // Check that at least one argument matches size with the result
+        for (size_t j = 0; j < call_node->args.size(); j++) {
+          auto arg = call_node->args[j];
+          for (auto target : target_sinfo) {
+            std::pair<bool, bool> match = SizeMatches(target, GetStructInfo(arg));
+            if (match.first) {
+              candidates.insert(static_cast<int>(j));
+              if (match.second) {
+                exact_match_candidates.insert(static_cast<int>(j));
+              }
+            }
+          }
+        }
+        if (!candidates.size()) {
+          continue;
+        }
+
+        // Make sure at least one candidate is not live past this point and does not have an alias
+        // live past this point
+        std::unordered_set<int> remove_candidates;
+        for (auto candidate : candidates) {
+          if (!InplaceConditionsMet(live_ranges, alias_sets, tuple_map, currently_live,
+                                    call_node->args[candidate], i)) {
+            remove_candidates.insert(candidate);
+          }
+        }
+        // (remove now to avoid modifying the list as we iterate on it)
+        for (auto candidate : remove_candidates) {
+          candidates.erase(candidate);
+        }
+
+        // if we have a candidate, then this can be made in-place. Report the appropriate candidates
+        if (!candidates.size()) {
+          continue;
+        }
+
+        // produce a list of candidates for this index
+        std::vector<int> size_match_indices = {static_cast<int>(i)};
+        size_match_indices.insert(size_match_indices.end(), candidates.begin(), candidates.end());
+        size_match_list.push_back(size_match_indices);
+
+        // also gather up the exact match candidates if there are any
+        std::vector<int> exact_match_indices = {static_cast<int>(i)};
+        for (auto candidate : candidates) {
+          if (exact_match_candidates.count(candidate)) {
+            exact_match_indices.push_back(candidate);
+          }
+        }
+        if (exact_match_indices.size() > 1) {
+          exact_match_list.push_back(exact_match_indices);
+        }
+      }
+    }
+
+    // remove vars whose range has come to an end
+    // (keep a separate set to avoid changing the sit while iterating on it)
+    std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual> remove;
+    for (auto var : currently_live) {
+      auto live_range = live_ranges[var];
+      if (live_range.second <= static_cast<int>(i)) {
+        remove.insert(var);
+      }
+    }
+    for (auto var : remove) {
+      currently_live.erase(var);
+    }
+  }
+
+  return {size_match_list, exact_match_list};
+}
+
+// Replace buffers in a PrimFunc according to the mapping.
+tir::Stmt RemapBuffers(const tir::Stmt& stmt, const Map<tir::Buffer, tir::Buffer>& buffer_map) {
+  class BufferMapper : public tir::StmtExprMutator {
+   public:
+    explicit BufferMapper(const Map<tir::Buffer, tir::Buffer>& buffer_map)
+        : buffer_map_(buffer_map) {}
+
+    tir::Stmt Remap(const tir::Stmt& stmt) { return VisitStmt(stmt); }
+
+    PrimExpr VisitExpr_(const tir::BufferLoadNode* op) final {
+      auto node = Downcast<tir::BufferLoad>(tir::StmtExprMutator::VisitExpr_(op));
+      auto* node_cow = node.CopyOnWrite();
+      node_cow->buffer = AttemptRemap(node->buffer);
+      return node;
+    }
+
+    tir::Stmt VisitStmt_(const tir::BufferStoreNode* op) final {
+      auto node = Downcast<tir::BufferStore>(tir::StmtExprMutator::VisitStmt_(op));
+      auto* node_cow = node.CopyOnWrite();
+      node_cow->buffer = AttemptRemap(node->buffer);
+      return node;
+    }
+
+    tir::Stmt VisitStmt_(const tir::BufferRealizeNode* op) final {
+      auto node = Downcast<tir::BufferRealize>(tir::StmtExprMutator::VisitStmt_(op));
+      auto* node_cow = node.CopyOnWrite();
+      node_cow->buffer = AttemptRemap(node->buffer);
+      return node;
+    }
+
+    tir::Stmt VisitStmt_(const tir::DeclBufferNode* op) final {
+      auto node = Downcast<tir::DeclBuffer>(tir::StmtExprMutator::VisitStmt_(op));
+      auto* node_cow = node.CopyOnWrite();
+      node_cow->buffer = AttemptRemap(node->buffer);
+      return node;
+    }
+
+    tir::Stmt VisitStmt_(const tir::BlockNode* op) final {
+      auto node = Downcast<tir::Block>(tir::StmtExprMutator::VisitStmt_(op));
+      auto* node_cow = node.CopyOnWrite();
+      // need the lambdas because class methods are not first-class (how ironic)
+      node_cow->alloc_buffers =
+          node->alloc_buffers.Map([this](const tir::Buffer& b) { return AttemptRemap(b); });
+      node_cow->reads =
+          node->reads.Map([this](const tir::BufferRegion& br) { return VisitBufferRegion(br); });
+      node_cow->writes =
+          node->writes.Map([this](const tir::BufferRegion& br) { return VisitBufferRegion(br); });
+      node_cow->match_buffers = node->match_buffers.Map(
+          [this](const tir::MatchBufferRegion& mbr) { return VisitMatchBufferRegion(mbr); });
+      return node;
+    }
+
+   private:
+    tir::Buffer AttemptRemap(const tir::Buffer& buffer) {
+      if (buffer_map_.count(buffer)) {
+        return buffer_map_.at(buffer);
+      }
+      return buffer;
+    }
+
+    tir::BufferRegion VisitBufferRegion(tir::BufferRegion region) {
+      auto* region_cow = region.CopyOnWrite();
+      region_cow->buffer = AttemptRemap(region_cow->buffer);
+      return region;
+    }
+
+    tir::MatchBufferRegion VisitMatchBufferRegion(tir::MatchBufferRegion region) {
+      auto* region_cow = region.CopyOnWrite();
+      region_cow->buffer = AttemptRemap(region_cow->buffer);
+      return region;
+    }
+
+    const Map<tir::Buffer, tir::Buffer>& buffer_map_;
+  };
+
+  BufferMapper mapper(buffer_map);
+  auto ret = mapper.Remap(stmt);
+  return ret;
+}
+
+class ModuleInplaceTransformer : public ExprMutator {
+ public:
+  explicit ModuleInplaceTransformer(const IRModule& mod) : mod_(mod) {
+    builder_ = BlockBuilder::Create(mod);
+  }
+
+  IRModule Transform() {
+    // visit every Relax function in the module
+    for (auto kv : mod_->functions) {
+      if (auto* func_node = kv.second.as<FunctionNode>()) {
+        auto gv = kv.first;
+        auto func_params = func_node->params;
+        auto function = GetRef<Function>(func_node);
+        auto* function_cow = function.CopyOnWrite();
+        auto new_body = VisitExpr(function->body);
+        function_cow->body = new_body;
+        builder_->UpdateFunction(gv, function);
+      }
+    }
+
+    auto ret = builder_->GetContextIRModule();
+    // clean up to avoid polluting the IRModule
+    for (auto gv : legalizers_added) {
+      ret->Remove(gv);
+    }
+    return ret;
+  }
+
+  // for handling inner functions
+  Expr VisitExpr_(const FunctionNode* op) override {
+    auto old_func_params = func_params;
+    func_params = op->params;
+    auto ret = ExprMutator::VisitExpr(GetRef<Function>(op));
+    func_params = old_func_params;
+    return ret;
+  }
+
+  // the only case we will override: we will visit all binding blocks
+  // and replace any valid calls in them
+  BindingBlock VisitBindingBlock_(const DataflowBlockNode* op) override {
+    auto block = GetRef<DataflowBlock>(op);
+    // std::unordered_set<Var, ObjectPtrHash, ObjectPtrEqual> free_var_set;
+    // for (auto binding : block->bindings) {
+    //   auto binding_free_vars = FreeVars(GetBoundValue(binding));
+    //   free_var_set.insert(binding_free_vars.begin(), binding_free_vars.end());
+    // }
+    // Array<Var> free_var_list(free_var_set.begin(), free_var_set.end());
+
+    // For now, only handle exact match cases.
+    // Note: Not passing any input values for now, as we can't make any assumptions
+    // about them.
+    auto matches_found = FindInplaceOpportunities(block, {});
+    auto exact_matches = matches_found.second;
+
+    Array<Binding> new_bindings;
+    int current_match_index = 0;
+    for (size_t i = 0; i < block->bindings.size(); i++) {
+      if (current_match_index >= static_cast<int>(exact_matches.size()) ||
+          exact_matches[current_match_index][0] != static_cast<int>(i)) {
+        new_bindings.push_back(block->bindings[i]);
+        continue;
+      }
+
+      auto target_binding = block->bindings[i];
+      auto target_call = Downcast<Call>(GetBoundValue(target_binding));
+      // can just pick the first index arbitrarily (only using one output for now too)
+      auto new_call = CreateInplaceCall(target_call, {exact_matches[current_match_index][1]});
+      // now replace the binding appropriately
+      if (auto* var_binding_node = target_binding.as<VarBindingNode>()) {
+        auto var_binding = GetRef<VarBinding>(var_binding_node);
+        auto* var_binding_cow = var_binding.CopyOnWrite();
+        var_binding_cow->value = new_call;
+        new_bindings.push_back(var_binding);
+      } else if (auto* match_cast_node = target_binding.as<MatchCastNode>()) {
+        auto match_cast = GetRef<MatchCast>(match_cast_node);
+        auto* match_cast_cow = match_cast.CopyOnWrite();
+        match_cast_cow->value = new_call;
+        new_bindings.push_back(match_cast);
+      } else {
+        CHECK(false) << "Invalid binding type";
+      }
+      current_match_index++;
+    }
+    return DataflowBlock(new_bindings, block->span);
+  }
+
+  // Given the call and indices of arguments that could be done in-place,
+  // replace the call with a call to an in-place PrimFunc.
+  // (Made public for testing.)
+  Call CreateInplaceCall(const Call& call, const Array<Integer>& inplace_indices) {
+    static const auto& legalize_map = Op::GetAttrMap<FLegalize>("FLegalize");

Review Comment:
   Should this use a separate legalization map?  Even if they map to the same function, the per-operator support could then be checked by seeing if `"FLegalizeInPlace"` is defined for an operator.



##########
tests/python/relax/test_dataflow_inplace.py:
##########
@@ -0,0 +1,464 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from typing import List, Set, Tuple
+import tvm
+from tvm import relax, testing
+from tvm.relax.transform import DataflowUseInplaceCalls
+from tvm.relax.testing.transform import (
+    dataflow_liveness_analysis,
+    dataflow_alias_analysis,
+    dataflow_inplace_analysis,
+    dataflow_single_inplace_call,
+)
+from tvm.script.parser import ir as I, relax as R, tir as T
+
+import numpy as np
+
+
+def test_liveness_analysis():
+    @I.ir_module
+    class BasicLiveness:
+        @R.function
+        def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            with R.dataflow():
+                y = R.const(1, dtype="int32")
+                z = R.add(x, y)
+                q = R.multiply(z, y)
+                p = R.add(z, q)
+                n = R.multiply(p, p)
+                R.output(n)
+            return n
+
+    block = BasicLiveness["main"].body.blocks[0]
+    live_ranges = dataflow_liveness_analysis(block)
+    expected_ranges = {
+        "x": (-1, 1),
+        "y": (0, 2),
+        "z": (1, 3),
+        "q": (2, 3),
+        "p": (3, 4),
+        "n": (4, 5),
+    }
+    for var, live_range in live_ranges.items():
+        assert live_range == expected_ranges[var.name_hint]
+
+
+def test_alias_analysis_basic():
+    @I.ir_module
+    class BasicAliasAnalysis:
+        @R.function
+        def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            with R.dataflow():
+                y = x  # y is an alias of x
+                z = R.add(y, y)  # fresh value
+                n = z  # alias of z
+                R.output(n)
+            return n
+
+    block = BasicAliasAnalysis["main"].body.blocks[0]
+    alias_sets, tuple_map = dataflow_alias_analysis(block, BasicAliasAnalysis["main"].params)
+    expected = {
+        "x": {0},
+        "y": {0},
+        "z": {1},
+        "n": {1},
+    }
+
+    for var, alias_set in alias_sets.items():
+        assert alias_set == expected[var.name_hint]
+    assert tuple_map == {}
+
+
+def test_alias_analysis_tuple():
+    @I.ir_module
+    class AliasesWithTuples:
+        @R.function
+        def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            with R.dataflow():
+                y = R.const(1, dtype="int32")
+                t = (x, y)
+                a = t[0]
+                b = t[1]
+                c = t[0]
+                d = t[1]
+                u = t
+                e = t[0]
+                f = t[1]
+                z = R.add(c, d)
+                n = z
+                R.output(n)
+            return n
+
+    block = AliasesWithTuples["main"].body.blocks[0]
+    alias_sets, tuple_map = dataflow_alias_analysis(block, AliasesWithTuples["main"].params)
+    expected = {
+        "x": {0},
+        "y": {1},
+        "t": {2},
+        "a": {0},
+        "b": {1},
+        "c": {0},
+        "d": {1},
+        "u": {2},
+        "e": {0},
+        "f": {1},
+        "z": {3},
+        "n": {3},
+    }
+
+    for var, alias_set in alias_sets.items():
+        assert alias_set == expected[var.name_hint]
+    assert 2 in tuple_map
+    assert tuple_map[2] == [{0}, {1}]
+
+
+def test_alias_split():
+    @I.ir_module
+    class AliasSplit:
+        @R.function
+        def main(x: R.Tensor((60,), "int32")) -> R.Tensor((15,), "int32"):
+            with R.dataflow():
+                t = R.split(x, 4)
+                y = t[0]
+                z = t[1]
+                q = t[2]
+                p = t[3]
+                n = z
+                R.output(n)
+            return n
+
+    block = AliasSplit["main"].body.blocks[0]
+    alias_sets, tuple_map = dataflow_alias_analysis(block, AliasSplit["main"].params)
+    expected = {
+        "x": {0},
+        "t": {1},
+        "y": {2},
+        "z": {3},
+        "q": {4},
+        "p": {5},
+        "n": {3},
+    }
+
+    for var, alias_set in alias_sets.items():
+        assert alias_set == expected[var.name_hint]
+    assert len(tuple_map) == 1
+    assert 1 in tuple_map
+    assert tuple_map[1] == [{2}, {3}, {4}, {5}]
+
+
+def test_alias_call_tir():
+    # call TIR can yield either a single tensor or a tuple
+    @I.ir_module
+    class AliasCallTir:
+        @T.prim_func
+        def tir_id(x: T.handle, y: T.handle) -> None:
+            T.func_attr({"global_symbol": "tir_id"})
+            m = T.int32()
+            n = T.int32()
+            A = T.match_buffer(x, (m, n))
+            B = T.match_buffer(y, (m, n))
+
+            for i, j in T.grid(m, n):
+                with T.block("id"):
+                    vi, vj = T.axis.remap("SS", [i, j])
+                    B[vi, vj] = A[vi, vj]
+
+        @T.prim_func
+        def tir_id2(x: T.handle, y: T.handle, z: T.handle) -> None:
+            T.func_attr({"global_symbol": "tir_id"})
+            m = T.int32()
+            n = T.int32()
+            A = T.match_buffer(x, (m, n))
+            B = T.match_buffer(y, (m, n))
+            C = T.match_buffer(z, (m, n))
+
+            for i, j in T.grid(m, n):
+                with T.block("id"):
+                    vi, vj = T.axis.remap("SS", [i, j])
+                    B[vi, vj] = A[vi, vj]
+                    C[vi, vj] = A[vi, vj]
+
+        @R.function
+        def main(x: R.Tensor((10, 10), "int32")) -> R.Tensor((10, 10), "int32"):
+            with R.dataflow():
+                cls = AliasCallTir
+                y = R.call_tir(cls.tir_id, (x,), out_sinfo=R.Tensor((10, 10), "int32"))
+                t = R.call_tir(
+                    cls.tir_id2,
+                    (y,),
+                    out_sinfo=[R.Tensor((10, 10), "int32"), R.Tensor((10, 10), "int32")],
+                )
+                z = y
+                p = t[0]
+                q = t[1]
+                u = t
+                m = u[0]
+                n = u[1]
+                v = n
+                R.output(v)
+            return v
+
+    block = AliasCallTir["main"].body.blocks[0]
+    alias_sets, tuple_map = dataflow_alias_analysis(block, AliasCallTir["main"].params)
+    expected = {
+        "x": {0},
+        "y": {1},
+        "t": {2},
+        "z": {1},
+        "p": {3},
+        "q": {4},
+        "u": {2},
+        "m": {3},
+        "n": {4},
+        "v": {4},
+    }
+
+    for var, alias_set in alias_sets.items():
+        assert alias_set == expected[var.name_hint]
+    assert len(tuple_map) == 1
+    assert 2 in tuple_map
+    assert tuple_map[2] == [{3}, {4}]
+
+
+def test_mystery_calls():
+    @I.ir_module
+    class AliasChaosCalls:
+        @R.function
+        def identity(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            return x
+
+        @R.function
+        def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            with R.dataflow():
+                cls = AliasChaosCalls
+                y = cls.identity(x)
+                z = cls.identity(y)
+                m = R.const(1, dtype="int32")
+                n = R.const(2, dtype="int32")
+                t = (m, n)
+                a = R.call_pure_packed(
+                    "chaos", t, sinfo_args=R.Tuple(R.Tensor((), "int32"), R.Tensor((), "int32"))
+                )
+                b = a[0]
+                c = a[1]
+                R.output(c)
+            return c
+
+    block = AliasChaosCalls["main"].body.blocks[0]
+    alias_sets, tuple_map = dataflow_alias_analysis(block, AliasChaosCalls["main"].params)
+    expected = {
+        "x": {0},
+        "y": {0, 1},
+        "z": {0, 1, 2},
+        "m": {3},
+        "n": {4},
+        "t": {5},
+        "a": {3, 4, 5, 6, 7, 8},  # either t or a fresh tuple
+        "b": {3, 4, 5, 6, 7, 8},  # the tuple components can be aliased to any member...
+        "c": {3, 4, 5, 6, 7, 8},  # the tuple components can be aliased to any member...
+        # (in principle, we can use type information to narrow down the aliasing)
+    }
+
+    for var, alias_set in alias_sets.items():
+        assert alias_set == expected[var.name_hint]
+    assert len(tuple_map) == 2
+    assert 5 in tuple_map
+    assert tuple_map[5] == [{3}, {4}]
+    assert 6 in tuple_map
+    assert tuple_map[6] == [{3, 4, 5, 6, 7, 8}, {3, 4, 5, 6, 7, 8}]
+
+
+def test_alias_external_value():
+    @I.ir_module
+    class AliasExternalValue:
+        @R.function
+        def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
+            y = R.const(1, dtype="int32")  # not in DF block, treated as external
+            t1 = (y, y)  # not in DF block, treated as external
+            with R.dataflow():
+                z = y  # mystery value
+                a = R.const(2, dtype="int32")
+                t2 = (z, a)
+                b = t2[0]
+                c = t1[1]  # tuple index into external value
+                R.output(b)
+            return b
+
+    block = AliasExternalValue["main"].body.blocks[1]
+    alias_sets, tuple_map = dataflow_alias_analysis(block, AliasExternalValue["main"].params)
+    expected = {
+        "x": {0},
+        "z": {-1},
+        "a": {1},
+        "t2": {2},
+        "b": {-1},
+        "c": {-1},
+    }
+
+    for var, alias_set in alias_sets.items():
+        assert alias_set == expected[var.name_hint]
+    assert len(tuple_map) == 1
+    assert 2 in tuple_map
+    assert tuple_map[2] == [{-1}, {1}]
+
+
+def test_inplace_simple_case():
+    @I.ir_module
+    class InplaceBasic:
+        @R.function
+        def main(
+            x: R.Tensor((2, 3), "int32"), y: R.Tensor((2, 3), "int32")
+        ) -> R.Tensor((2, 3), "int32"):
+            with R.dataflow():
+                z = R.add(x, y)  # cannot be done inplace: x and y are live later
+                q = R.multiply(x, y)  # can be done inplace: neither x nor y is used later
+                p = R.add(z, q)  # can be done inplace: neither z nor q is used later
+                r = p  # alias of p
+                m = R.multiply(p, p)  # p is not used later but r is, so can't do inplace
+                n = R.add(m, r)  # can be done inplace: r is not used again
+                ret = R.subtract(n, m)  # can be done inplace: neither is used again
+                R.output(ret)
+            return ret
+
+    block = InplaceBasic["main"].body.blocks[0]
+    size_match, exact_match = dataflow_inplace_analysis(block, InplaceBasic["main"].params)
+
+    # order does not matter for the listing of candidates, so we have to implement as sets
+    def assert_candidate_list(
+        actual: List[List[int]], expected: List[Tuple[int, Set[int]]]
+    ) -> None:
+        assert len(actual) == len(expected)
+        for i in range(len(actual)):
+            assert actual[i][0] == expected[i][0]
+            assert len(expected[i][1]) == len(actual[i]) - 1
+            for j in range(len(expected[i][1])):
+                assert actual[i][j + 1] in expected[i][1]
+
+    assert_candidate_list(size_match, [(1, {0, 1}), (2, {0, 1}), (5, {1}), (6, {0, 1})])
+    # TODO(@slyubomirsky): I couldn't think of an easy example where sizes don't match,
+    # but broadcasting might cause it to happen
+    assert_candidate_list(exact_match, [(1, {0, 1}), (2, {0, 1}), (5, {1}), (6, {0, 1})])
+
+
+def test_inplace_single_call():
+    @I.ir_module
+    class TestModule:
+        @R.function
+        def main(
+            x: R.Tensor((2, 3), dtype="float32"), y: R.Tensor((2, 3), dtype="float32")
+        ) -> R.Tensor((2, 3), dtype="float32"):
+            z = R.add(x, y)
+            q = R.nn.silu(z)
+            return q
+
+    add_call = TestModule["main"].body.blocks[0].bindings[0].value
+    new_add, new_mod = dataflow_single_inplace_call(TestModule, add_call, [0])
+
+    @T.prim_func(private=True)
+    def expected_add(
+        A: T.Buffer((T.int64(2), T.int64(3)), "float32"),
+        B: T.Buffer((T.int64(2), T.int64(3)), "float32"),
+    ):
+        T.func_attr({"tir.noalias": T.bool(True)})
+        for ax0, ax1 in T.grid(T.int64(2), T.int64(3)):
+            with T.block("T_add"):
+                v_ax0, v_ax1 = T.axis.remap("SS", [ax0, ax1])
+                T.reads(A[v_ax0, v_ax1], B[v_ax0, v_ax1])
+                T.writes(A[v_ax0, v_ax1])
+                A[v_ax0, v_ax1] = A[v_ax0, v_ax1] + B[v_ax0, v_ax1]
+
+    tvm.ir.assert_structural_equal(new_mod["add_inplace"], expected_add)
+    assert new_add.op.name == "relax.call_tir_inplace"
+    assert new_add.args[0].name_hint == "add_inplace"
+    for i, arg in enumerate(new_add.args[1].fields):
+        arg == add_call.args[i]
+    new_add.attrs.inplace_indices == [0]
+
+    @T.prim_func(private=True)
+    def expected_silu(A: T.Buffer((T.int64(2), T.int64(3)), "float32")):
+        T.func_attr({"tir.noalias": T.bool(True)})
+        compute = T.alloc_buffer((T.int64(2), T.int64(3)))
+        for i0, i1 in T.grid(T.int64(2), T.int64(3)):
+            with T.block("compute"):
+                v_i0, v_i1 = T.axis.remap("SS", [i0, i1])
+                T.reads(A[v_i0, v_i1])
+                T.writes(compute[v_i0, v_i1])
+                compute[v_i0, v_i1] = T.sigmoid(A[v_i0, v_i1])
+        for ax0, ax1 in T.grid(T.int64(2), T.int64(3)):
+            with T.block("T_multiply"):
+                v_ax0, v_ax1 = T.axis.remap("SS", [ax0, ax1])
+                T.reads(A[v_ax0, v_ax1], compute[v_ax0, v_ax1])
+                T.writes(A[v_ax0, v_ax1])
+                A[v_ax0, v_ax1] = A[v_ax0, v_ax1] * compute[v_ax0, v_ax1]
+
+    silu_call = TestModule["main"].body.blocks[0].bindings[1].value
+    new_silu, new_mod = dataflow_single_inplace_call(TestModule, silu_call, [0])
+
+    tvm.ir.assert_structural_equal(new_mod["silu_inplace"], expected_silu)
+    assert new_silu.op.name == "relax.call_tir_inplace"
+    assert new_silu.args[0].name_hint == "silu_inplace"
+    for i, arg in enumerate(new_silu.args[1].fields):
+        arg == silu_call.args[i]
+    new_silu.attrs.inplace_indices == [0]
+
+
+def test_insert_inplace_calls():
+    @I.ir_module
+    class EndToEndTest:
+        @R.function
+        def main(
+            x: R.Tensor((2, 3), dtype="float32"), y: R.Tensor((1, 3), dtype="float32")
+        ) -> R.Tensor((2, 3), dtype="float32"):
+            with R.dataflow():
+                z = R.add(x, y)  # broadcast happens here
+                # Cannot be done in-place because x is an argument.
+                a = R.add(z, y)  # this one can be done in-place
+                q = R.multiply(a, y)  # broadcast again, a is eligible
+                r = R.subtract(y, y)  # cannot be done in-place because y is an argument
+                s = R.subtract(r, r)  # No broadcast. Can be done in-place
+                m = R.multiply(q, s)  # should give us all zeros
+                R.output(m)
+            return m
+
+    transform_pass = DataflowUseInplaceCalls()
+    new_mod = transform_pass(EndToEndTest)
+
+    # check that all operations are done in-place
+    assert new_mod["add_inplace"]
+    assert new_mod["subtract_inplace"]
+    assert new_mod["multiply_inplace"]
+    expected_ops = ["add_inplace", "multiply_inplace", "subtract_inplace", "multiply_inplace"]
+    inplace_binding_idxs = [1, 2, 4, 5]
+    for i, idx in enumerate(inplace_binding_idxs):
+        binding = new_mod["main"].body.blocks[0].bindings[idx]
+        assert binding.value.op.name == "relax.call_tir_inplace"
+        assert binding.value.args[0].name_hint == expected_ops[i]
+
+    x = tvm.nd.array(np.random.rand(2, 3).astype("float32"))
+    y = tvm.nd.array(np.random.rand(1, 3).astype("float32"))
+    expected = np.zeros((2, 3), dtype="float32")
+
+    target = tvm.target.Target("llvm")
+    ex = relax.build(new_mod, target)
+    vm = relax.VirtualMachine(ex, tvm.cpu())
+    res = vm["main"](x, y)
+    assert (expected == res.numpy()).all()
+

Review Comment:
   Do we want a `pytest.xfail` case for internal functions, to indicate that they are not yet supported?  I'm thinking of cases like the following, where the current local analysis would conclude that `Module.func` cannot do in-place operations, but non-local analysis could perform `Module.func` in-place for some cases, depending on the liveness analysis in the calling scope.
   
   ```python
   @I.ir_module
   class Module:
       @R.function
       def main(x: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
           cls = Module
           with R.dataflow():
               # Cannot be in-place, uses a function argument
               y = cls.subroutine(x)
               
               # Can be in-place, uses a local binding
               z = cls.subroutine(y)
               R.output(z)
           return z
       
       @R.function(private=True)
       def subroutine(x: R.Tensor((), "int32"), y: R.Tensor((), "int32")) -> R.Tensor((), "int32"):
           with R.dataflow():
               z = R.add(x,R.const(1, "int32"))
               R.output(z)
           return z
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org