You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/07/06 21:27:12 UTC
[GitHub] [tvm] AndrewZhaoLuo opened a new issue #8412: [Topi] Allow relay.nn.dense support arbitrary number dimensions
AndrewZhaoLuo opened a new issue #8412:
URL: https://github.com/apache/tvm/issues/8412
Right now the schedule assumes two dimensions for the input tensor though the documentation suggests otherwise:
https://tvm.apache.org/docs/api/python/relay/nn.html#tvm.relay.nn.dense
Alternatively we can just adjust the documentation to stipulate relay.nn.dense has an input tensor of only two dimensions. However, other frameworks allow their dense layers to be arbitrary number dimensions. E.g. https://pytorch.org/docs/stable/generated/torch.nn.Linear.html#torch.nn.Linear
For forum discussion see: https://discuss.tvm.apache.org/t/relay-nn-does-relay-nn-dense-supports-multi-dimensional-input/10343/5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on issue #8412: [Topi] Allow relay.nn.dense support arbitrary number dimensions
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #8412:
URL: https://github.com/apache/tvm/issues/8412#issuecomment-943180691
I think a good way to add N-D input support to `topi.dense` is to do `topi.reshape` on the input before doing dense compute, and inside dense schedules we do `compute_inline(...)` on the reshape stage. This way, we can reuse existing schedules that are hardcoded for 2D input at no performance overhead (modulo some indexing math from reshaping). See my PR https://github.com/apache/tvm/pull/9207 that does something similar.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi edited a comment on issue #8412: [Topi] Allow relay.nn.dense support arbitrary number dimensions
Posted by GitBox <gi...@apache.org>.
masahi edited a comment on issue #8412:
URL: https://github.com/apache/tvm/issues/8412#issuecomment-943168765
See https://discuss.tvm.apache.org/t/relay-nn-does-relay-nn-dense-supports-multi-dimensional-input/10343/7. I realized that supporting more than 2D input for `dense` is extremely important for BERT-like model performance. In particular, without fixing this issue, I cannot demonstrate the performance advantage of cutlass BYOC https://github.com/apache/tvm/pull/9261 on transformer models over cublass offload. cc @comaniac @Laurawly
It would also improve TVM numbers for `bert_large` in https://github.com/apache/tvm/issues/8294#issuecomment-926599422, because those numbers were obtained without dense + activation fusion (also no fusion for batch_matmul).
@AndrewZhaoLuo Are you going to work on this one? If not, I need to do it anyway before or after https://github.com/apache/tvm/pull/9261 lands.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on issue #8412: [Topi] Allow relay.nn.dense support arbitrary number dimensions
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #8412:
URL: https://github.com/apache/tvm/issues/8412#issuecomment-943168765
See https://discuss.tvm.apache.org/t/relay-nn-does-relay-nn-dense-supports-multi-dimensional-input/10343/7. I realized that supporting more than 2D input for `dense` is extremely important for BERT-like model performance. In particular, without fixing this issue, I cannot demonstrate the performance advantage of cutlass BYOC https://github.com/apache/tvm/pull/9261 on transformer models over cublass offload. cc @comaniac @Laurawly
@AndrewZhaoLuo Are you going to work on this one? If not, I need to do it anyway before or after https://github.com/apache/tvm/pull/9261 lands.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org