You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/03/29 15:32:24 UTC
[GitHub] [tvm] MuchToMyDelight opened a new pull request #7763: [vm] add support for customized allocator
MuchToMyDelight opened a new pull request #7763:
URL: https://github.com/apache/tvm/pull/7763
Hello guys!
After trying the VM in TVM, I found that the default memory allocator(PooledAllocator) doesn't work very well: in case of dynamic inputs, the size of used buffers are different from inference to inference and PooledAllocator cached buffers based on the size. Therefore, it ends with out of device memory.
This patch doesn't aim to solve this problem, since memory allocation is a very complicated issue. I believe users should have the choice to implement whatever suits their needs the most. In addition, the cost of allowing users to register and use their customized allocator is little to none: Registery::Get gets called only once.
I suppose @wweic is the reviewer for VM, what's your opinion?
Signed-off-by: JasonL <Mu...@sers.noreply.github.com>
Thanks for contributing to TVM! Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @ them in the pull request thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MuchToMyDelight commented on pull request #7763: [vm] add support for customized allocator
Posted by GitBox <gi...@apache.org>.
MuchToMyDelight commented on pull request #7763:
URL: https://github.com/apache/tvm/pull/7763#issuecomment-810310258
> Hey @MuchToMyDelight the patch looks overall good, I wonder if we should allow users to instead register by name or some other key to support multiple allocators to be loaded?
Hello jroesch, I'm not sure what do you mean by 'some other key'. Do you mean add a C++ api like Set/GetAllocator, instead of by name? I chose register by name because: 1. it make code clean & neat. 2. There are other places using the same pattern in TVM, e.g. DeviceApiManger uses keys like "device_api.gpu"/"device_api.cpu". So I suppose this is the 'TVM way' of doing this kind of thing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MuchToMyDelight commented on pull request #7763: [vm] add support for customized allocator
Posted by GitBox <gi...@apache.org>.
MuchToMyDelight commented on pull request #7763:
URL: https://github.com/apache/tvm/pull/7763#issuecomment-810315066
> Hey @MuchToMyDelight the patch looks overall good, I wonder if we should allow users to instead register by name or some other key to support multiple allocators to be loaded?
Hello jroesch, I'm not sure what do you mean by 'some other key'. Do you mean add a C++ api like Set/GetAllocator, instead of register by name? I chose register by name because: 1. it makes code clean & neat. 2. There are other places using the same pattern in TVM, e.g. DeviceApiManger uses keys like "device_api.gpu"/"device_api.cpu". So I suppose this is the 'TVM way' of doing this kind of thing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jroesch commented on pull request #7763: [vm] add support for customized allocator
Posted by GitBox <gi...@apache.org>.
jroesch commented on pull request #7763:
URL: https://github.com/apache/tvm/pull/7763#issuecomment-809981319
Hey @MuchToMyDelight the patch looks overall good, I wonder if we should allow users to instead register by name or some other key to support multiple allocators to be loaded?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MuchToMyDelight removed a comment on pull request #7763: [vm] add support for customized allocator
Posted by GitBox <gi...@apache.org>.
MuchToMyDelight removed a comment on pull request #7763:
URL: https://github.com/apache/tvm/pull/7763#issuecomment-810310258
> Hey @MuchToMyDelight the patch looks overall good, I wonder if we should allow users to instead register by name or some other key to support multiple allocators to be loaded?
Hello jroesch, I'm not sure what do you mean by 'some other key'. Do you mean add a C++ api like Set/GetAllocator, instead of by name? I chose register by name because: 1. it make code clean & neat. 2. There are other places using the same pattern in TVM, e.g. DeviceApiManger uses keys like "device_api.gpu"/"device_api.cpu". So I suppose this is the 'TVM way' of doing this kind of thing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org