You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/02/03 07:38:54 UTC
[GitHub] [tvm] VertexC opened a new issue #7399: TVM compile pytorch model mutliple times seems to have memory leak
VertexC opened a new issue #7399:
URL: https://github.com/apache/tvm/issues/7399
Hi,
I am doing some perf analysis on TVM and find compile pytorch model multiple time seems to have memory leak.
Here is the script. I am using latest tvm `commit f786` compiled with clang+llvm-11.0.1-x86_64-linux-gnu-ubuntu-16.04.
```python
import os, psutil
import numpy as np
from tvm import relay
from tvm.relay import testing
import tvm
from tvm import te
from tvm.contrib import graph_runtime
import tvm.testing
import torch
import torchvision
def tvm_build():
input_name = "input0"
shape = [1, 3, 224, 224]
data = torch.randn(shape, dtype=torch.float32)
model = torchvision.models.resnet50(pretrained=False, progress=True)
shape_list = [(input_name, data.shape)]
scripted_model = torch.jit.trace(model, data).eval()
mod, params = relay.frontend.from_pytorch(scripted_model, shape_list)
# TODO: how opt_level affects performance
opt_level = 3
with tvm.transform.PassContext(opt_level=opt_level):
lib = relay.build(mod,
target='llvm',
target_host='llvm',
params=params)
ctx = tvm.cpu()
module = graph_runtime.GraphModule(lib["default"](ctx))
module.set_input(input_name, data)
def main():
process = psutil.Process(os.getpid())
print('Used Memory before building:', process.memory_info().rss / 1024 / 1024, 'MB')
total = 10
for i in range(0, total+1):
tvm_build()
print('Used Memory after building {} times:'.format(i+1), process.memory_info().rss / 1024 / 1024, 'MB')
if __name__ == "__main__":
main()
```
And the result show the memory usage continue to increase
```bash
Used Memory after building 1 times: 817.33203125 MB
Used Memory after building 2 times: 943.7734375 MB
Used Memory after building 3 times: 1086.84765625 MB
Used Memory after building 4 times: 1276.97265625 MB
Used Memory after building 5 times: 1503.7109375 MB
Used Memory after building 6 times: 1504.33984375 MB
Used Memory after building 7 times: 1662.96875 MB
Used Memory after building 8 times: 1879.96875 MB
Used Memory after building 9 times: 1880.83984375 MB
Used Memory after building 10 times: 1964.89453125 MB
Used Memory after building 11 times: 2152.046875 MB
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen commented on issue #7399: TVM compile pytorch model mutliple times seems to have memory leak
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #7399:
URL: https://github.com/apache/tvm/issues/7399#issuecomment-796355895
@VertexC can you confirm if it is still a problem of GraphModule? we do need to make sure graphmodule get wrapped in a function, so the destructor get triggered
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] VertexC edited a comment on issue #7399: TVM compile pytorch model mutliple times seems to have memory leak
Posted by GitBox <gi...@apache.org>.
VertexC edited a comment on issue #7399:
URL: https://github.com/apache/tvm/issues/7399#issuecomment-782576070
Hi @masahi
Thansk for the reply.
I tried with the fellowing modified script and memory still increases.
```python3
def main()
process = psutil.Process(os.getpid())
print('Used Memory before building:', process.memory_info().rss / 1024 / 1024, 'MB')
total = 10
input_name = "input0"
shape = [1, 3, 224, 224]
data = torch.randn(shape, dtype=torch.float32)
model = torchvision.models.resnet50(pretrained=False, progress=True)
shape_list = [(input_name, data.shape)]
scripted_model = torch.jit.trace(model, data).eval()
mod, params = relay.frontend.from_pytorch(scripted_model, shape_list)
for i in range(0, total+1):
opt_level = 3
with tvm.transform.PassContext(opt_level=opt_level):
lib = relay.build(mod,
target='llvm',
target_host='llvm',
params=params)
ctx = tvm.cpu()
module = graph_runtime.GraphModule(lib["default"](ctx))
module.set_input(input_name, data)
print('Used Memory after building {} times:'.format(i+1), process.memory_info().rss / 1024 / 1024, 'MB')
```
```bash
Used Memory before building: 235.1640625 MB
...
Used Memory after building 1 times: 907.3828125 MB
Used Memory after building 2 times: 1153.3046875 MB
Used Memory after building 3 times: 1273.3515625 MB
Used Memory after building 4 times: 1386.25390625 MB
Used Memory after building 5 times: 1489.9296875 MB
```
btw, if i comment out
```python3
ctx = tvm.cpu()
module = graph_runtime.GraphModule(lib["default"](ctx))
module.set_input(input_name, data)
```
the memory issue seems to be eased
```bash
Used Memory before building: 236.8046875 MB
...
Used Memory after building 1 times: 816.05859375 MB
Used Memory after building 2 times: 1026.9375 MB
Used Memory after building 3 times: 1054.21875 MB
Used Memory after building 4 times: 1062.2734375 MB
Used Memory after building 5 times: 1079.9296875 MB
Used Memory after building 6 times: 1061.29296875 MB
Used Memory after building 7 times: 1079.0703125 MB
Used Memory after building 8 times: 1096.953125 MB
Used Memory after building 9 times: 1078.56640625 MB
Used Memory after building 10 times: 1105.34375 MB
Used Memory after building 11 times: 1087.70703125 MB
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on issue #7399: TVM compile pytorch model mutliple times seems to have memory leak
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #7399:
URL: https://github.com/apache/tvm/issues/7399#issuecomment-778556686
Note that this is not specific to pytorch, try moving
```
input_name = "input0"
shape = [1, 3, 224, 224]
data = torch.randn(shape, dtype=torch.float32)
model = torchvision.models.resnet50(pretrained=False, progress=True)
shape_list = [(input_name, data.shape)]
scripted_model = torch.jit.trace(model, data).eval()
mod, params = relay.frontend.from_pytorch(scripted_model, shape_list)
```
outside of `tvm_build(...)`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] VertexC commented on issue #7399: TVM compile pytorch model mutliple times seems to have memory leak
Posted by GitBox <gi...@apache.org>.
VertexC commented on issue #7399:
URL: https://github.com/apache/tvm/issues/7399#issuecomment-782576070
Hi @masahi
Thansk for the reply.
I tried with the fellowing modified script and memory still increases.
```python3
def main()
process = psutil.Process(os.getpid())
print('Used Memory before building:', process.memory_info().rss / 1024 / 1024, 'MB')
total = 10
input_name = "input0"
shape = [1, 3, 224, 224]
data = torch.randn(shape, dtype=torch.float32)
model = torchvision.models.resnet50(pretrained=False, progress=True)
shape_list = [(input_name, data.shape)]
scripted_model = torch.jit.trace(model, data).eval()
mod, params = relay.frontend.from_pytorch(scripted_model, shape_list)
for i in range(0, total+1):
opt_level = 3
with tvm.transform.PassContext(opt_level=opt_level):
lib = relay.build(mod,
target='llvm',
target_host='llvm',
params=params)
ctx = tvm.cpu()
module = graph_runtime.GraphModule(lib["default"](ctx))
module.set_input(input_name, data)
print('Used Memory after building {} times:'.format(i+1), process.memory_info().rss / 1024 / 1024, 'MB')
```
```bash
Used Memory before building: 235.1640625 MB
...
Used Memory after building 1 times: 907.3828125 MB
Used Memory after building 2 times: 1153.3046875 MB
Used Memory after building 3 times: 1273.3515625 MB
Used Memory after building 4 times: 1386.25390625 MB
Used Memory after building 5 times: 1489.9296875 MB
```
btw, if i comment out
```python3
#ctx = tvm.cpu()
#module = graph_runtime.GraphModule(lib["default"](ctx))
#module.set_input(input_name, data)
```
the memory issue seems to be eased
```bash
Used Memory before building: 236.8046875 MB
...
Used Memory after building 1 times: 816.05859375 MB
Used Memory after building 2 times: 1026.9375 MB
Used Memory after building 3 times: 1054.21875 MB
Used Memory after building 4 times: 1062.2734375 MB
Used Memory after building 5 times: 1079.9296875 MB
Used Memory after building 6 times: 1061.29296875 MB
Used Memory after building 7 times: 1079.0703125 MB
Used Memory after building 8 times: 1096.953125 MB
Used Memory after building 9 times: 1078.56640625 MB
Used Memory after building 10 times: 1105.34375 MB
Used Memory after building 11 times: 1087.70703125 MB
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org