You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "MasterJH5574 (via GitHub)" <gi...@apache.org> on 2023/11/03 15:21:11 UTC
[PR] [Runtime] Support clear global memory allocators [tvm]
MasterJH5574 opened a new pull request, #16066:
URL: https://github.com/apache/tvm/pull/16066
This PR supports clearing up all the allocated memory.
Prior to this PR, all the allocated memory are managed in the pool of memory manager. The allocated memory in the pool is on hold and never freed. Consequently, the pool size always goes up monotonically within a single run in a process.
While good to save time of memory allocation, in some cases (e.g., on mobile phones which may have running memory limit) we need to clear the pool and free all the memory in order to prevent the pool from endlessly growing up and some of allocated memory not being effectively utilized (fragmentation).
Therefore, this PR introduces a PackedFunc that helps clean up the memory manager, releasing all the allocated memory. Runtime apps can decide when to invoke this PackedFunc and clean up the pool. Usually, this will happen at some app "reset" or "reload" stage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Runtime] Support clear global memory allocators [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on code in PR #16066:
URL: https://github.com/apache/tvm/pull/16066#discussion_r1381977764
##########
src/runtime/memory/memory_manager.cc:
##########
@@ -166,6 +167,12 @@ Allocator* MemoryManager::GetAllocator(Device dev, AllocatorType type) {
return it->second.at(type).get();
}
+void MemoryManager::Clear() {
+ MemoryManager* m = MemoryManager::Global();
+ std::lock_guard<std::mutex> lock(m->mu_);
+ m->allocators_.clear();
Review Comment:
let us iterate over each allocator and clear each of the freed entries, as oppose to clear the vector, which can cause issues when some of the allocated storage from the allocators get retained and deleted after clear
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Runtime] Support clear global memory allocators [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on code in PR #16066:
URL: https://github.com/apache/tvm/pull/16066#discussion_r1382607364
##########
src/runtime/memory/memory_manager.cc:
##########
@@ -166,6 +167,18 @@ Allocator* MemoryManager::GetAllocator(Device dev, AllocatorType type) {
return it->second.at(type).get();
}
+void MemoryManager::Clear() {
+ MemoryManager* m = MemoryManager::Global();
Review Comment:
We need to add a Clear interface to base Allocator, and iterate over the allocators, to call Allocator->Clear() without removing any allocators
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Runtime] Support clear global memory allocators [tvm]
Posted by "MasterJH5574 (via GitHub)" <gi...@apache.org>.
MasterJH5574 commented on code in PR #16066:
URL: https://github.com/apache/tvm/pull/16066#discussion_r1382081932
##########
src/runtime/memory/memory_manager.cc:
##########
@@ -166,6 +167,12 @@ Allocator* MemoryManager::GetAllocator(Device dev, AllocatorType type) {
return it->second.at(type).get();
}
+void MemoryManager::Clear() {
+ MemoryManager* m = MemoryManager::Global();
+ std::lock_guard<std::mutex> lock(m->mu_);
+ m->allocators_.clear();
Review Comment:
Thanks for pointing it out. Just pushed a new commit, not sure if it addresses in the exactly same way as you mentioned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Runtime] Support clear global memory allocators [tvm]
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on PR #16066:
URL: https://github.com/apache/tvm/pull/16066#issuecomment-1793649491
Any updates? The latest rebase/merge will depend on this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Runtime] Support clear global memory allocators [tvm]
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao merged PR #16066:
URL: https://github.com/apache/tvm/pull/16066
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org