You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "MasterJH5574 (via GitHub)" <gi...@apache.org> on 2023/08/16 01:59:25 UTC

[GitHub] [tvm] MasterJH5574 opened a new pull request, #15571: [Unity][Dlight] GeMV rule skip cases of "outer dim being grouped"

MasterJH5574 opened a new pull request, #15571:
URL: https://github.com/apache/tvm/pull/15571

   Prior to this PR, the GeMV contains an assumption that "the grouped dimension of the largest tensor in GeMV must has the same iter type as the innermost dimension", which is enforced by an assertion.
   
   Since AutoGPTQ quantization only supports KN GeMV layout, the decode-GeMV generated under the AutoGPTQ quantization has the pattern where the reduction dimension is not the innermost and is grouped. This pattern does not follow the assersion above and thus an assertion error will be thrown.
   
   This PR updates the logic to skip transforming such cases, instead of asserting the assumption. One real AutoGPTQ decode-GeMV workload is added as a test case for future awareness.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] jinhongyii merged pull request #15571: [Unity][Dlight] GeMV rule skip cases of "outer dim being grouped"

Posted by "jinhongyii (via GitHub)" <gi...@apache.org>.
jinhongyii merged PR #15571:
URL: https://github.com/apache/tvm/pull/15571


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] LeiWang1999 commented on pull request #15571: [Unity][Dlight] GeMV rule skip cases of "outer dim being grouped"

Posted by "LeiWang1999 (via GitHub)" <gi...@apache.org>.
LeiWang1999 commented on PR #15571:
URL: https://github.com/apache/tvm/pull/15571#issuecomment-1679928795

   if we don't know how the memory access can be friendly if we use block reduction, maybe we can manually do transpose when loading the params from gptq, then we can enhance the performance of dlight default schedule ?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] tvm-bot commented on pull request #15571: [Unity][Dlight] GeMV rule skip cases of "outer dim being grouped"

Posted by "tvm-bot (via GitHub)" <gi...@apache.org>.
tvm-bot commented on PR #15571:
URL: https://github.com/apache/tvm/pull/15571#issuecomment-1679855540

   <!---bot-comment-->
   
   Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
   
   <!--bot-comment-ccs-start-->
    * cc @quic-sanirudh <sub>See [#10317](https://github.com/apache/tvm/issues/10317) for details</sub><!--bot-comment-ccs-end-->
   
   <sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] MasterJH5574 commented on pull request #15571: [Unity][Dlight] GeMV rule skip cases of "outer dim being grouped"

Posted by "MasterJH5574 (via GitHub)" <gi...@apache.org>.
MasterJH5574 commented on PR #15571:
URL: https://github.com/apache/tvm/pull/15571#issuecomment-1679954826

   > if we don't know how the memory access can be friendly if we use block reduction on kxn layout, maybe we can manually do transpose when loading the params from gptq, then we can enhance the performance of dlight default schedule ?
   
   Good point. I’m not sure how the autogptq decode-GeMV can benefit from the GeMV dlight rule. Perhaps we can try out as as follow up. In the meantime, we need to remove the assertion here in the PR to ensure robustness.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org