You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/11/01 20:31:54 UTC

[GitHub] [tvm] masahi opened a new pull request, #13259: [MetaSchedule] Swap the order of RewriteTensorize and VerifyGPUCode to reduce tuning time

masahi opened a new pull request, #13259:
URL: https://github.com/apache/tvm/pull/13259

   I'm working on improving tuning time on quantized models using tensor core. One of the reasons it is slow is that the function below, which is called thousands of times during each evo search initial population sampling, takes more than 50 msec: 
   
   https://github.com/apache/tvm/blob/a4840e7de38c5a2000917f2101f3ec4a374bcd39/src/meta_schedule/search_strategy/evolutionary_search.cc#L506-L517
   
   The time it takes can be broken down into the followings:
   * `ApplyToTrace`: 35 msec
   * `RewriteTensorize`: 7-8 msec
   * `VerifyGPUCode`: 7-8 msec
   
   `RewriteTensorize` is relatively slow because it invokes `sch->Tensorize(...)` which does some heavy lifting. Since it is called thousands of times, it quickly adds up. It is wasteful if the tensorized schedule is rejected by `VerifyGPUCode` immediately after that.
   
   Since `tensorize` doesn't affect the validity of a sample, we can improve tuning time slightly by running `VerifyGPUCode` first and do `RewriteTensorize` only on valid samples.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] tvm-bot commented on pull request #13259: [MetaSchedule] Swap the order of RewriteTensorize and VerifyGPUCode to reduce tuning time

Posted by GitBox <gi...@apache.org>.
tvm-bot commented on PR #13259:
URL: https://github.com/apache/tvm/pull/13259#issuecomment-1299103055

   <!---bot-comment-->
   
   Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @-ing them in a comment.
   
   <!--bot-comment-ccs-start-->
    * cc @Hzfengsy, @elvin-n, @junrushao <sub>See [#10317](https://github.com/apache/tvm/issues/10317) for details</sub><!--bot-comment-ccs-end-->
   
   <sub>Generated by [tvm-bot](https://github.com/apache/tvm/blob/main/ci/README.md#github-actions)</sub>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] masahi commented on pull request #13259: [MetaSchedule] Swap the order of RewriteTensorize and VerifyGPUCode to reduce tuning time

Posted by GitBox <gi...@apache.org>.
masahi commented on PR #13259:
URL: https://github.com/apache/tvm/pull/13259#issuecomment-1299282635

   I don't think it's necessary.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] masahi commented on pull request #13259: [MetaSchedule] Swap the order of RewriteTensorize and VerifyGPUCode to reduce tuning time

Posted by GitBox <gi...@apache.org>.
masahi commented on PR #13259:
URL: https://github.com/apache/tvm/pull/13259#issuecomment-1299210825

   @tvm-bot rerun


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] masahi merged pull request #13259: [MetaSchedule] Swap the order of RewriteTensorize and VerifyGPUCode to reduce tuning time

Posted by GitBox <gi...@apache.org>.
masahi merged PR #13259:
URL: https://github.com/apache/tvm/pull/13259


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org