You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/01/24 21:42:12 UTC
[GitHub] [tvm] achalhoubmeaa opened a new issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
achalhoubmeaa opened a new issue #10050:
URL: https://github.com/apache/tvm/issues/10050
I am trying to work with the SSD300_VGG16 object detection model and convert it to a serialized and optimized format to eventually run on an embedded system. The script that I use to create and save the traced model from TorchScript is the following:
```
import torch
import torchvision
def do_trace(model, in_size=500):
model_trace = torch.jit.trace(model, torch.rand(1, 3, in_size, in_size))
model_trace.eval()
return model_trace
def dict_to_tuple(out_dict):
if "masks" in out_dict.keys():
return (out_dict["boxes"], out_dict["scores"], out_dict["labels"], out_dict["masks"])
return (out_dict["boxes"], out_dict["scores"], out_dict["labels"])
class TraceWrapper(torch.nn.Module):
def __init__(self, model):
super().__init__()
self.model = model
def forward(self, inp):
out = self.model(inp)
return dict_to_tuple(out[0])
model_funcs = [torchvision.models.detection.ssd300_vgg16]
names = ["ssd300_vgg16"]
for name, model_func in zip(names, model_funcs):
model = TraceWrapper(model_func(num_classes=50, pretrained_backbone=False))
model.eval()
in_size = 500
inp = torch.rand(1, 3, in_size, in_size)
with torch.no_grad():
out = model(inp)
script_module = do_trace(model)
script_out = script_module(inp)
assert len(out[0]) > 0 and len(script_out[0]) > 0
torch._C._jit_pass_inline(script_module.graph)
torch.jit.save(script_module, name + ".pt")
```
After that I would like to convert the saved model to another format that I am working with, but I am getting the error below:
File "/home/achalhoub/tvm/python/tvm/relay/frontend/pytorch.py", line 3091, in report_missing_conversion
raise NotImplementedError(msg)
NotImplementedError: The following operators are not implemented: ['aten::clamp_min', 'aten::copy_']
From the research that I've done on this, it seems like TVM doesn't support the conversion of some OPs in certain model architectures?
I would appreciate help with this issue!
Thanks.
I am using the following versions:
**torch 1.10.1+cu102
torchvision 0.11.2+cu102
tvm 0.9.dev0**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] achalhoubmeaa commented on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
achalhoubmeaa commented on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1021622659
Thank you for explaining and sending a PR.
What about the **['aten::clamp_min']** error as well? How can I know what section/part of the network caused this error?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1021628997
I've just added `clamp_min` support in https://github.com/apache/tvm/compare/main...masahi:torch-clamp-min?expand=1
With the two patches, the script https://github.com/masahi/torchscript-to-tvm/blob/master/ssd-vgg/test.py should work. But currently the outputs from TVM are very strange, I'll take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1020626046
`aten::copy_` cannot convert to relay since this is an in-place op. So the model needs to be modified to remove mutation on PyTorch side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi edited a comment on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
masahi edited a comment on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1021554357
so `aten::copy_` apparently comes from these lines https://github.com/pytorch/vision/blob/d367a01a18a3ae6bee13d8be3b63fd6a581ea46f/torchvision/models/detection/anchor_utils.py#L267-L268
This is an example of in-place op (values are mutated in-place). We need to replace this with "pure" operation for conversion to TVM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] achalhoubmeaa commented on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
achalhoubmeaa commented on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1023277181
Thank you so much! This seems like a really helpful tool for debugging such issues. I appreciate your help :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] achalhoubmeaa edited a comment on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
achalhoubmeaa edited a comment on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1022229107
@masahi is there any guidance you could give me on how you were able to identify what part of the model code were responsible for producing those errors please? Especially the `aten::copy_` error. I understand them once you explained them but I couldn't have been able to identify them myself, and it would be useful to me to have some knowledge of how to debug for those kinds of errors since I will be working with multiple PyTorch models in the future.
I would really appreciate it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1020626046
`aten::copy_` cannot convert to relay since this is an in-place op. So the model needs to be modified to remove mutation on PyTorch side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] achalhoubmeaa commented on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
achalhoubmeaa commented on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1022229107
@masahi is there any guidance you could give me on how you were able to identify what part of the model code were responsible for producing those errors please? Especially the `aten::copy_` error. I understand them once you explained them but I couldn't have been able to identify them myself, and it would be useful to me to have some knowledge of how to debug for that specific type of error since I will be working with multiple PyTorch models in the future.
I would really appreciate it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] DzAvril edited a comment on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
DzAvril edited a comment on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1049690820
> `aten::copy_` cannot convert to relay since this is an in-place op. So the model needs to be modified to remove mutation on PyTorch side.
@masahi May I ask why the in-place ops cannot be converted to relay? In pytorch front pytorch.py there are in-place ops like aten::add_ has been converted to relay. So what is the difference between aten::copy_ and aten::add_？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] achalhoubmeaa commented on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
achalhoubmeaa commented on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1021417527
Thank you for your response. Would you mind briefly explaining what an in-place op is? Or maybe guide me to some documentation so I can better understand?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1021554357
so `aten::copy_` apparently comes from these lines
https://github.com/pytorch/vision/blob/d367a01a18a3ae6bee13d8be3b63fd6a581ea46f/torchvision/models/detection/anchor_utils.py#L267-L268
This is an example of in-place op (values are mutated in-place). We need to replace this with "pure" operation for conversion to TVM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] achalhoubmeaa commented on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
achalhoubmeaa commented on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1021651800
I will try to convert the model again with these two patches and let you know if I have any other issues. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi closed issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
masahi closed issue #10050:
URL: https://github.com/apache/tvm/issues/10050
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1021577844
FYI I sent a PR to torchvision to remove the in-place op in question https://github.com/pytorch/vision/pull/5279#issuecomment-1021573157
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1022681200
Yeah that's a good question. The way I did it was, first put `print(graph`) after https://github.com/apache/tvm/blob/095b63935efdd42effc0298998d67838086b7b26/python/tvm/relay/frontend/pytorch.py#L3913 and find `aten::copy_`. The dump shows for each `aten::` op its corresponding location in the python source code. That's how I found that code in `anchor_utils.py`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] DzAvril edited a comment on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
DzAvril edited a comment on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1049690820
> `aten::copy_` cannot convert to relay since this is an in-place op. So the model needs to be modified to remove mutation on PyTorch side.
@masahi May I ask why the in-place ops cannot be converted to relay? In pytorch front pytorch.py there are in-place ops like aten::add_ has been converted to relay. So what is the difference between aten::copy_ and aten::add_.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] DzAvril commented on issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
DzAvril commented on issue #10050:
URL: https://github.com/apache/tvm/issues/10050#issuecomment-1049690820
> `aten::copy_` cannot convert to relay since this is an in-place op. So the model needs to be modified to remove mutation on PyTorch side.
May I ask why the in-place ops cannot be converted to relay? In pytorch front pytorch.py there are in-place ops like aten::add_ has been converted to relay. So what is the difference between aten::copy_ and aten::add_.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi closed issue #10050: 'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models
Posted by GitBox <gi...@apache.org>.
masahi closed issue #10050:
URL: https://github.com/apache/tvm/issues/10050
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org