You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "guan404ming (via GitHub)" <gi...@apache.org> on 2025/11/10 12:41:34 UTC
[PR] [Relax][Pytorch] Support basic range constraints [tvm]
guan404ming opened a new pull request, #18429:
URL: https://github.com/apache/tvm/pull/18429
As per tile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2514941567
##########
python/tvm/relax/frontend/torch/exported_program_translator.py:
##########
@@ -1149,14 +1170,22 @@ def from_exported_program(
from torch import fx # type: ignore
# Create input variables.
- parameter_buffer_constant_vars, user_input_vars = self.create_input_vars(exported_program)
+ (
+ parameter_buffer_constant_vars,
+ user_input_vars,
+ range_constraints,
+ ) = self.create_input_vars(exported_program)
inputs_vars = user_input_vars.copy()
inputs_vars.update(parameter_buffer_constant_vars)
# Initialize the block builder with a function and a dataflow block.
self.block_builder = relax.BlockBuilder()
func_name = "main"
func_attrs = {"num_input": len(user_input_vars)} if keep_params_as_input else None
+ if range_constraints:
+ if func_attrs is None:
+ func_attrs = {}
+ func_attrs["shape_var_constraints"] = range_constraints
Review Comment:
Thanks for letting me know, I've updated to use `tir_var_upper_bound` to annotate upper bound.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "mshr-h (via GitHub)" <gi...@apache.org>.
mshr-h commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2513619153
##########
python/tvm/relax/frontend/torch/exported_program_translator.py:
##########
@@ -1149,14 +1170,22 @@ def from_exported_program(
from torch import fx # type: ignore
# Create input variables.
- parameter_buffer_constant_vars, user_input_vars = self.create_input_vars(exported_program)
+ (
+ parameter_buffer_constant_vars,
+ user_input_vars,
+ range_constraints,
+ ) = self.create_input_vars(exported_program)
inputs_vars = user_input_vars.copy()
inputs_vars.update(parameter_buffer_constant_vars)
# Initialize the block builder with a function and a dataflow block.
self.block_builder = relax.BlockBuilder()
func_name = "main"
func_attrs = {"num_input": len(user_input_vars)} if keep_params_as_input else None
+ if range_constraints:
+ if func_attrs is None:
+ func_attrs = {}
+ func_attrs["shape_var_constraints"] = range_constraints
Review Comment:
Please use `tir_var_upper_bound` to annotate upper bound.
I grepped the tvm code base and I found that there's no lower bound annotation. So I don't think we need to keep it at the moment. If we have a real use case for it, it's fine to keep it in the Relax module.
https://github.com/apache/tvm/blob/main/src/relax/transform/static_plan_block_memory.cc#L62-L66
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "mshr-h (via GitHub)" <gi...@apache.org>.
mshr-h commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2513601966
##########
python/tvm/relax/frontend/torch/exported_program_translator.py:
##########
@@ -1121,13 +1136,19 @@ def create_input_vars(
torch_shape = exported_program.state_dict[spec.target].shape
torch_dtype = exported_program.state_dict[spec.target].dtype
- # TODO(mshr-h): Support range constraints
- relax_shape = [
- torch_symbol_to_relax_var.setdefault(str(s), tvm.tir.SizeVar(str(s), "int64"))
- if isinstance(s, torch.SymInt)
- else s
- for s in torch_shape
- ]
+ # Create TIR variables for symbolic dimensions
Review Comment:
Looks like there's no functional changes. Any reason for the change?
##########
tests/python/relax/test_frontend_from_exported_program.py:
##########
@@ -6663,5 +6527,29 @@ def forward(self, x):
np.testing.assert_allclose(pytorch_output2.numpy(), tvm_output2_np, rtol=1e-4, atol=1e-5)
+def test_dynamic_shape_with_range_constraints():
+ class DynamicModel(torch.nn.Module):
+ def forward(self, x1, x2):
+ return torch.ops.aten.add.Tensor(x1, x2)
+
+ example_args = (torch.randn(8, 4), torch.randn(8, 4))
+ batch = torch.export.Dim("batch", min=1, max=64)
+ dynamic_shapes = {"x1": {0: batch}, "x2": {0: batch}}
+ exported_program = export(DynamicModel(), args=example_args, dynamic_shapes=dynamic_shapes)
+
+ mod = from_exported_program(exported_program)
+
+ main_func = mod["main"]
+ assert hasattr(main_func, "attrs"), "Function should have attributes"
Review Comment:
Please use structual equality instead of manually checking the attributes.
##########
python/tvm/relax/frontend/torch/exported_program_translator.py:
##########
@@ -1149,14 +1170,22 @@ def from_exported_program(
from torch import fx # type: ignore
# Create input variables.
- parameter_buffer_constant_vars, user_input_vars = self.create_input_vars(exported_program)
+ (
+ parameter_buffer_constant_vars,
+ user_input_vars,
+ range_constraints,
+ ) = self.create_input_vars(exported_program)
inputs_vars = user_input_vars.copy()
inputs_vars.update(parameter_buffer_constant_vars)
# Initialize the block builder with a function and a dataflow block.
self.block_builder = relax.BlockBuilder()
func_name = "main"
func_attrs = {"num_input": len(user_input_vars)} if keep_params_as_input else None
+ if range_constraints:
+ if func_attrs is None:
+ func_attrs = {}
+ func_attrs["shape_var_constraints"] = range_constraints
Review Comment:
Please use `tir_var_upper_bound` to annotate upper bound.
I don't think we need to keep the lower bound at the moment. If we have a real use case for it, it's fine to keep it in the Relax module.
https://github.com/apache/tvm/blob/main/src/relax/transform/static_plan_block_memory.cc#L62-L66
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "tlopex (via GitHub)" <gi...@apache.org>.
tlopex commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2515816043
##########
tests/python/relax/test_frontend_from_exported_program.py:
##########
@@ -6663,5 +6527,29 @@ def forward(self, x):
np.testing.assert_allclose(pytorch_output2.numpy(), tvm_output2_np, rtol=1e-4, atol=1e-5)
+def test_dynamic_shape_with_range_constraints():
+ class DynamicModel(torch.nn.Module):
+ def forward(self, x1, x2):
+ return torch.ops.aten.add.Tensor(x1, x2)
+
+ example_args = (torch.randn(8, 4), torch.randn(8, 4))
+ batch = torch.export.Dim("batch", min=1, max=64)
+ dynamic_shapes = {"x1": {0: batch}, "x2": {0: batch}}
+ exported_program = export(DynamicModel(), args=example_args, dynamic_shapes=dynamic_shapes)
+
+ mod = from_exported_program(exported_program)
Review Comment:
Of course you can help update it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #18429:
URL: https://github.com/apache/tvm/pull/18429#issuecomment-3520774220
Thanks for your detailed comments and reviews!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2514904102
##########
tests/python/relax/test_frontend_from_exported_program.py:
##########
@@ -6663,5 +6527,29 @@ def forward(self, x):
np.testing.assert_allclose(pytorch_output2.numpy(), tvm_output2_np, rtol=1e-4, atol=1e-5)
+def test_dynamic_shape_with_range_constraints():
+ class DynamicModel(torch.nn.Module):
+ def forward(self, x1, x2):
+ return torch.ops.aten.add.Tensor(x1, x2)
+
+ example_args = (torch.randn(8, 4), torch.randn(8, 4))
+ batch = torch.export.Dim("batch", min=1, max=64)
+ dynamic_shapes = {"x1": {0: batch}, "x2": {0: batch}}
+ exported_program = export(DynamicModel(), args=example_args, dynamic_shapes=dynamic_shapes)
+
+ mod = from_exported_program(exported_program)
+
+ main_func = mod["main"]
+ assert hasattr(main_func, "attrs"), "Function should have attributes"
Review Comment:
Sure, I've updated the test to follow the convention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "mshr-h (via GitHub)" <gi...@apache.org>.
mshr-h commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2513731480
##########
tests/python/relax/test_frontend_from_exported_program.py:
##########
@@ -6663,5 +6527,29 @@ def forward(self, x):
np.testing.assert_allclose(pytorch_output2.numpy(), tvm_output2_np, rtol=1e-4, atol=1e-5)
+def test_dynamic_shape_with_range_constraints():
+ class DynamicModel(torch.nn.Module):
+ def forward(self, x1, x2):
+ return torch.ops.aten.add.Tensor(x1, x2)
+
+ example_args = (torch.randn(8, 4), torch.randn(8, 4))
+ batch = torch.export.Dim("batch", min=1, max=64)
+ dynamic_shapes = {"x1": {0: batch}, "x2": {0: batch}}
+ exported_program = export(DynamicModel(), args=example_args, dynamic_shapes=dynamic_shapes)
+
+ mod = from_exported_program(exported_program)
Review Comment:
Should be `from_exported_program(exported_program, run_ep_decomposition=True)`
See #18399
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2514798838
##########
tests/python/relax/test_frontend_from_exported_program.py:
##########
@@ -6663,5 +6527,29 @@ def forward(self, x):
np.testing.assert_allclose(pytorch_output2.numpy(), tvm_output2_np, rtol=1e-4, atol=1e-5)
+def test_dynamic_shape_with_range_constraints():
+ class DynamicModel(torch.nn.Module):
+ def forward(self, x1, x2):
+ return torch.ops.aten.add.Tensor(x1, x2)
+
+ example_args = (torch.randn(8, 4), torch.randn(8, 4))
+ batch = torch.export.Dim("batch", min=1, max=64)
+ dynamic_shapes = {"x1": {0: batch}, "x2": {0: batch}}
+ exported_program = export(DynamicModel(), args=example_args, dynamic_shapes=dynamic_shapes)
+
+ mod = from_exported_program(exported_program)
Review Comment:
Thanks for providing useful info!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2514703870
##########
python/tvm/relax/frontend/torch/exported_program_translator.py:
##########
@@ -1097,11 +1097,28 @@ def create_convert_map(
def create_input_vars(
self, exported_program: torch.export.ExportedProgram
- ) -> Tuple[Dict[str, relax.Var], Dict[str, relax.Var]]:
+ ) -> Tuple[Dict[str, relax.Var], Dict[str, relax.Var], Dict[str, Tuple[int, int]]]:
"""Create relax input vars."""
parameters_buffers_constants = OrderedDict()
user_inputs = OrderedDict()
torch_symbol_to_relax_var: Dict[str, tvm.tir.Var] = {}
+ range_constraints = {}
+
+ if hasattr(exported_program, "range_constraints"):
+ for symbol, value_range in exported_program.range_constraints.items():
+ symbol_name = str(symbol)
+ if hasattr(value_range, "lower") and hasattr(value_range, "upper"):
+ try:
+ lower = int(value_range.lower)
+ except (OverflowError, AttributeError, TypeError):
+ continue
+
+ try:
+ upper = int(value_range.upper)
+ except (OverflowError, AttributeError, TypeError):
+ continue
+
+ range_constraints[symbol_name] = (lower, upper)
Review Comment:
Sure, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2514952876
##########
tests/python/relax/test_frontend_from_exported_program.py:
##########
@@ -6663,5 +6527,29 @@ def forward(self, x):
np.testing.assert_allclose(pytorch_output2.numpy(), tvm_output2_np, rtol=1e-4, atol=1e-5)
+def test_dynamic_shape_with_range_constraints():
+ class DynamicModel(torch.nn.Module):
+ def forward(self, x1, x2):
+ return torch.ops.aten.add.Tensor(x1, x2)
+
+ example_args = (torch.randn(8, 4), torch.randn(8, 4))
+ batch = torch.export.Dim("batch", min=1, max=64)
+ dynamic_shapes = {"x1": {0: batch}, "x2": {0: batch}}
+ exported_program = export(DynamicModel(), args=example_args, dynamic_shapes=dynamic_shapes)
+
+ mod = from_exported_program(exported_program)
Review Comment:
Should we also need to update tests in https://github.com/apache/tvm/blob/f574031657faa035ffd1875050316b11821187cf/tests/python/relax/test_frontend_from_exported_program.py?
I could help with this if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "mshr-h (via GitHub)" <gi...@apache.org>.
mshr-h commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2513731480
##########
tests/python/relax/test_frontend_from_exported_program.py:
##########
@@ -6663,5 +6527,29 @@ def forward(self, x):
np.testing.assert_allclose(pytorch_output2.numpy(), tvm_output2_np, rtol=1e-4, atol=1e-5)
+def test_dynamic_shape_with_range_constraints():
+ class DynamicModel(torch.nn.Module):
+ def forward(self, x1, x2):
+ return torch.ops.aten.add.Tensor(x1, x2)
+
+ example_args = (torch.randn(8, 4), torch.randn(8, 4))
+ batch = torch.export.Dim("batch", min=1, max=64)
+ dynamic_shapes = {"x1": {0: batch}, "x2": {0: batch}}
+ exported_program = export(DynamicModel(), args=example_args, dynamic_shapes=dynamic_shapes)
+
+ mod = from_exported_program(exported_program)
Review Comment:
Please include `run_ep_decomposition=True`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #18429:
URL: https://github.com/apache/tvm/pull/18429#issuecomment-3512731711
cc @mshr-h
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2516534364
##########
tests/python/relax/test_frontend_from_exported_program.py:
##########
@@ -6663,5 +6527,29 @@ def forward(self, x):
np.testing.assert_allclose(pytorch_output2.numpy(), tvm_output2_np, rtol=1e-4, atol=1e-5)
+def test_dynamic_shape_with_range_constraints():
+ class DynamicModel(torch.nn.Module):
+ def forward(self, x1, x2):
+ return torch.ops.aten.add.Tensor(x1, x2)
+
+ example_args = (torch.randn(8, 4), torch.randn(8, 4))
+ batch = torch.export.Dim("batch", min=1, max=64)
+ dynamic_shapes = {"x1": {0: batch}, "x2": {0: batch}}
+ exported_program = export(DynamicModel(), args=example_args, dynamic_shapes=dynamic_shapes)
+
+ mod = from_exported_program(exported_program)
Review Comment:
Sure! I would open another PR to handle this, thanks for your reply.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "mshr-h (via GitHub)" <gi...@apache.org>.
mshr-h merged PR #18429:
URL: https://github.com/apache/tvm/pull/18429
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2513643742
##########
python/tvm/relax/frontend/torch/exported_program_translator.py:
##########
@@ -1097,11 +1097,28 @@ def create_convert_map(
def create_input_vars(
self, exported_program: torch.export.ExportedProgram
- ) -> Tuple[Dict[str, relax.Var], Dict[str, relax.Var]]:
+ ) -> Tuple[Dict[str, relax.Var], Dict[str, relax.Var], Dict[str, Tuple[int, int]]]:
"""Create relax input vars."""
parameters_buffers_constants = OrderedDict()
user_inputs = OrderedDict()
torch_symbol_to_relax_var: Dict[str, tvm.tir.Var] = {}
+ range_constraints = {}
+
+ if hasattr(exported_program, "range_constraints"):
+ for symbol, value_range in exported_program.range_constraints.items():
+ symbol_name = str(symbol)
+ if hasattr(value_range, "lower") and hasattr(value_range, "upper"):
+ try:
+ lower = int(value_range.lower)
+ except (OverflowError, AttributeError, TypeError):
+ continue
+
+ try:
+ upper = int(value_range.upper)
+ except (OverflowError, AttributeError, TypeError):
+ continue
+
+ range_constraints[symbol_name] = (lower, upper)
Review Comment:
The two `try-except` blocks for parsing `lower` and `upper` values are repetitive. You can combine them into a single `try-except` block for better readability and conciseness. This would also ensure that `range_constraints` is only updated if both `lower` and `upper` are successfully parsed.
```suggestion
try:
lower = int(value_range.lower)
upper = int(value_range.upper)
range_constraints[symbol_name] = (lower, upper)
except (OverflowError, AttributeError, TypeError):
continue
```
##########
tests/python/relax/test_frontend_from_exported_program.py:
##########
@@ -6527,5 +6527,29 @@ def forward(self, x):
np.testing.assert_allclose(pytorch_output2.numpy(), tvm_output2_np, rtol=1e-4, atol=1e-5)
+def test_dynamic_shape_with_range_constraints():
+ class DynamicModel(torch.nn.Module):
+ def forward(self, x1, x2):
+ return torch.ops.aten.add.Tensor(x1, x2)
+
+ example_args = (torch.randn(8, 4), torch.randn(8, 4))
+ batch = torch.export.Dim("batch", min=1, max=64)
+ dynamic_shapes = {"x1": {0: batch}, "x2": {0: batch}}
+ exported_program = export(DynamicModel(), args=example_args, dynamic_shapes=dynamic_shapes)
+
+ mod = from_exported_program(exported_program)
+
+ main_func = mod["main"]
+ assert hasattr(main_func, "attrs"), "Function should have attributes"
+
+ if "shape_var_constraints" in main_func.attrs:
+ constraints = main_func.attrs["shape_var_constraints"]
+ assert len(constraints) > 0, "Should have at least one constraint"
+
+ for symbol_name, (min_val, max_val) in constraints.items():
+ assert min_val == 1, f"Expected min=1 for {symbol_name}, got {min_val}"
+ assert max_val == 64, f"Expected max=64 for {symbol_name}, got {max_val}"
Review Comment:
Using `if "shape_var_constraints" in main_func.attrs:` makes the test pass silently if the attribute is missing, which would hide a potential regression. To make the test more robust and ensure it actually verifies the presence of range constraints, you should use an `assert` statement instead.
```suggestion
assert "shape_var_constraints" in main_func.attrs
constraints = main_func.attrs["shape_var_constraints"]
assert len(constraints) > 0, "Should have at least one constraint"
for symbol_name, (min_val, max_val) in constraints.items():
assert min_val == 1, f"Expected min=1 for {symbol_name}, got {min_val}"
assert max_val == 64, f"Expected max=64 for {symbol_name}, got {max_val}"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "mshr-h (via GitHub)" <gi...@apache.org>.
mshr-h commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2513731480
##########
tests/python/relax/test_frontend_from_exported_program.py:
##########
@@ -6663,5 +6527,29 @@ def forward(self, x):
np.testing.assert_allclose(pytorch_output2.numpy(), tvm_output2_np, rtol=1e-4, atol=1e-5)
+def test_dynamic_shape_with_range_constraints():
+ class DynamicModel(torch.nn.Module):
+ def forward(self, x1, x2):
+ return torch.ops.aten.add.Tensor(x1, x2)
+
+ example_args = (torch.randn(8, 4), torch.randn(8, 4))
+ batch = torch.export.Dim("batch", min=1, max=64)
+ dynamic_shapes = {"x1": {0: batch}, "x2": {0: batch}}
+ exported_program = export(DynamicModel(), args=example_args, dynamic_shapes=dynamic_shapes)
+
+ mod = from_exported_program(exported_program)
Review Comment:
Please add `run_ep_decomposition=True`.
##########
tests/python/relax/test_frontend_from_exported_program.py:
##########
@@ -6663,5 +6527,29 @@ def forward(self, x):
np.testing.assert_allclose(pytorch_output2.numpy(), tvm_output2_np, rtol=1e-4, atol=1e-5)
+def test_dynamic_shape_with_range_constraints():
+ class DynamicModel(torch.nn.Module):
+ def forward(self, x1, x2):
+ return torch.ops.aten.add.Tensor(x1, x2)
+
+ example_args = (torch.randn(8, 4), torch.randn(8, 4))
+ batch = torch.export.Dim("batch", min=1, max=64)
+ dynamic_shapes = {"x1": {0: batch}, "x2": {0: batch}}
+ exported_program = export(DynamicModel(), args=example_args, dynamic_shapes=dynamic_shapes)
+
+ mod = from_exported_program(exported_program)
Review Comment:
Please add `run_ep_decomposition=True` for `from_exported_program`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "mshr-h (via GitHub)" <gi...@apache.org>.
mshr-h commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2513977745
##########
python/tvm/relax/frontend/torch/exported_program_translator.py:
##########
@@ -1097,11 +1097,28 @@ def create_convert_map(
def create_input_vars(
self, exported_program: torch.export.ExportedProgram
- ) -> Tuple[Dict[str, relax.Var], Dict[str, relax.Var]]:
+ ) -> Tuple[Dict[str, relax.Var], Dict[str, relax.Var], Dict[str, Tuple[int, int]]]:
"""Create relax input vars."""
parameters_buffers_constants = OrderedDict()
user_inputs = OrderedDict()
torch_symbol_to_relax_var: Dict[str, tvm.tir.Var] = {}
+ range_constraints = {}
+
+ if hasattr(exported_program, "range_constraints"):
+ for symbol, value_range in exported_program.range_constraints.items():
+ symbol_name = str(symbol)
+ if hasattr(value_range, "lower") and hasattr(value_range, "upper"):
+ try:
+ lower = int(value_range.lower)
+ except (OverflowError, AttributeError, TypeError):
+ continue
+
+ try:
+ upper = int(value_range.upper)
+ except (OverflowError, AttributeError, TypeError):
+ continue
+
+ range_constraints[symbol_name] = (lower, upper)
Review Comment:
Make sense to me.
@guan404ming Can you apply the change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #18429:
URL: https://github.com/apache/tvm/pull/18429#discussion_r2514765206
##########
python/tvm/relax/frontend/torch/exported_program_translator.py:
##########
@@ -1121,13 +1136,19 @@ def create_input_vars(
torch_shape = exported_program.state_dict[spec.target].shape
torch_dtype = exported_program.state_dict[spec.target].dtype
- # TODO(mshr-h): Support range constraints
- relax_shape = [
- torch_symbol_to_relax_var.setdefault(str(s), tvm.tir.SizeVar(str(s), "int64"))
- if isinstance(s, torch.SymInt)
- else s
- for s in torch_shape
- ]
+ # Create TIR variables for symbolic dimensions
Review Comment:
Sorry about this, it seems like legacy change in #17898. Let me revert this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Relax][Pytorch] Support basic range constraints [tvm]
Posted by "mshr-h (via GitHub)" <gi...@apache.org>.
mshr-h commented on PR #18429:
URL: https://github.com/apache/tvm/pull/18429#issuecomment-3516046883
/gemini review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org