You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mxnet.apache.org by GitBox <gi...@apache.org> on 2022/06/21 14:48:46 UTC
[GitHub] [incubator-mxnet] matteosal opened a new issue, #21076: NDArrays leak their memory when deleted
matteosal opened a new issue, #21076:
URL: https://github.com/apache/incubator-mxnet/issues/21076
This looks pretty bad:
```
import mxnet as mx
import os, psutil
print('branch: ' + mx.runtime.get_branch())
print('commit: ' + mx.runtime.get_commit_hash())
def get_memory_usage():
return psutil.Process(os.getpid()).memory_info().rss / 1e+6
print(get_memory_usage())
nd = mx.nd.ones([1000000, 1000])
mx.ndarray.waitall()
print(get_memory_usage())
del(nd)
mx.ndarray.waitall()
print(get_memory_usage())
```
output:
```
branch: master
commit: 1dba76998de3584bfa98c05989845ac08db2079a
196.419584
[16:47:01] /home/matteo/Git/mxnet-build/Build/Linux-x86-64/MKL/mxnet/src/storage/storage.cc:202: Using Pooled (Naive) StorageManager for CPU
4195.528704
4195.528704
```
Psutil shows that memory is not released after the NDArray is destroyed (last print)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] matteosal commented on issue #21076: NDArrays leak their memory when deleted
Posted by GitBox <gi...@apache.org>.
matteosal commented on issue #21076:
URL: https://github.com/apache/incubator-mxnet/issues/21076#issuecomment-1179017938
Thanks, I was caught by surprise by the change of behaviour with respect to v1.x and was afraid this was a bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] matteosal closed issue #21076: NDArrays leak their memory when deleted
Posted by GitBox <gi...@apache.org>.
matteosal closed issue #21076: NDArrays leak their memory when deleted
URL: https://github.com/apache/incubator-mxnet/issues/21076
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] bgawrych commented on issue #21076: NDArrays leak their memory when deleted
Posted by GitBox <gi...@apache.org>.
bgawrych commented on issue #21076:
URL: https://github.com/apache/incubator-mxnet/issues/21076#issuecomment-1178702865
Hi @matteosal, it is expected behaviour as MXNet by default uses Pooled StorageManager. So memory buffer is cached and can be reused for other array. See code snippet:
```
import mxnet as mx
import os, psutil
print('branch: ' + mx.runtime.get_branch())
print('commit: ' + mx.runtime.get_commit_hash())
def get_memory_usage():
return psutil.Process(os.getpid()).memory_info().rss / 1e+6
print("before alloc:", get_memory_usage())
nd = mx.nd.ones([1000000, 1000])
mx.ndarray.waitall()
print("after alloc: ", get_memory_usage())
del(nd)
mx.ndarray.waitall()
print("after delete: ", get_memory_usage())
nd2 = mx.nd.ones([1000000, 1000])
mx.ndarray.waitall()
print("after nd2 alloc: ", get_memory_usage())
```
Output:
```
branch: HEAD
commit: e36c9f075aa7ee14e1a8ccf245df5d1e1648515b
before alloc: 178.159616
[08:20:11] ../src/storage/storage.cc:202: Using Pooled (Naive) StorageManager for CPU
after alloc: 4174.946304
after delete: 4174.946304
after nd2 alloc: 4174.946304
```
But with `del` statement commented:
```
branch: HEAD
commit: e36c9f075aa7ee14e1a8ccf245df5d1e1648515b
before alloc: 177.11104
[08:19:03] ../src/storage/storage.cc:202: Using Pooled (Naive) StorageManager for CPU
after alloc: 4174.024704
after delete: 4174.024704
after nd2 alloc: 8176.889856
```
If you don't want to use this feature you can change to unpooled storage engine by env flag: `MXNET_CPU_MEM_POOL_TYPE=Unpooled`
https://mxnet.apache.org/versions/master/api/faq/env_var
Let me know if we can close this issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org