You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "masahi (via GitHub)" <gi...@apache.org> on 2023/05/09 01:18:57 UTC
[GitHub] [tvm] masahi opened a new pull request, #14802: [Unity] Add pass to allocate big workspace and pass it to all functions that need temp storage
masahi opened a new pull request, #14802:
URL: https://github.com/apache/tvm/pull/14802
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen commented on pull request #14802: [Unity] Add pass to allocate big workspace and pass it to all functions that need temp storage
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on PR #14802:
URL: https://github.com/apache/tvm/pull/14802#issuecomment-1540498665
Thanks @masahi ! Some high level notes:
- We don't need to use zeros, can use relax's builtin alloc_tensor
Some ideas on making it even more composable:
- We can have cutlass BYOC to generate TIR functions that contains the allocate and call_extern part into cutlass (the T.allocate here can depend on dyn shape)
- We can then have a pass that lift the workspace allocation into relax
- Finally, the relax memory planning will tie up the reusable memory through memory planning
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #14802: [Unity] Add pass to allocate big workspace and pass it to all functions that need temp storage
Posted by "masahi (via GitHub)" <gi...@apache.org>.
masahi commented on PR #14802:
URL: https://github.com/apache/tvm/pull/14802#issuecomment-1540801232
> We don't need to use zeros, can use relax's builtin alloc_tensor
@tqchen It seems `alloc_storage` requires a device index, which we don't know during this pass but `zeros` automatically takes care of the device issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tvm-bot commented on pull request #14802: [Unity] Add pass to allocate big workspace and pass it to all functions that need temp storage
Posted by "tvm-bot (via GitHub)" <gi...@apache.org>.
tvm-bot commented on PR #14802:
URL: https://github.com/apache/tvm/pull/14802#issuecomment-1539259469
<!---bot-comment-->
Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
<!--bot-comment-ccs-start-->
* cc @quic-sanirudh <sub>See [#10317](https://github.com/apache/tvm/issues/10317) for details</sub><!--bot-comment-ccs-end-->
<sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #14802: [Unity] Add pass to allocate big workspace and pass it to all functions that need temp storage
Posted by "masahi (via GitHub)" <gi...@apache.org>.
masahi merged PR #14802:
URL: https://github.com/apache/tvm/pull/14802
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #14802: [Unity] Add pass to allocate big workspace and pass it to all functions that need temp storage
Posted by "masahi (via GitHub)" <gi...@apache.org>.
masahi commented on PR #14802:
URL: https://github.com/apache/tvm/pull/14802#issuecomment-1540765410
> Does this also work for dynamic shape?
@yelite For now no, since we need to find the maximum of the workspace sizes required by all extern functions. In principle we may be able to specify the size via a symbolic expression and doing max at runtime, but that's for future PR :)
Attention with dynamic seq len sounds interesting. For that, the kernel needs to check the given workspace size and if that's smaller than the actual runtime size it requires, we need to fallback to additional malloc / free. If there is a known upper bound on the dynamic dim size, we can use that when we annotate workspace requirements.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] yelite commented on pull request #14802: [Unity] Add pass to allocate big workspace and pass it to all functions that need temp storage
Posted by "yelite (via GitHub)" <gi...@apache.org>.
yelite commented on PR #14802:
URL: https://github.com/apache/tvm/pull/14802#issuecomment-1540488045
Does this also work for dynamic shape? It looks so but just want to confirm. Although the codegen doesn't support dynamic shape attention yet, it would be great if the workspace allocation works with dynamic shape.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen commented on pull request #14802: [Unity] Add pass to allocate big workspace and pass it to all functions that need temp storage
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on PR #14802:
URL: https://github.com/apache/tvm/pull/14802#issuecomment-1540869511
device_index=0 should be sufficient and is also the default behavior(the default device). Hetro execution would need to have smarter selection of device index anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org