[GitHub] [tvm] lhutton1 commented on issue #12858: [Bug] Issue with tvmc run with repeat=0

[GitBox <gi...@apache.org>]

lhutton1 commented on issue #12858:
URL: https://github.com/apache/tvm/issues/12858#issuecomment-1254866832

   Hi @f2013519, I submitted #12713 which makes sure the module is only run once when the user is _not_ worried about benchmarking so wanted to share some of my findings here. I also ran into a similar issue when benchmarking was being used (`--print-time=True`) and, as you correctly mention, the additional run is due to a warm-up use to prime caches and lazy optimizations. I found this a bit unintuitive at first so tried to document some of these quirks directly in the PR (#12713).
   
   That said, it seems like a legitimate requirement to me to be able to benchmark just a single execution of the model, which currently is not possible. The additional run happens here: https://github.com/apache/tvm/blob/main/src/runtime/profiling.cc#L880 - we should make this additional run optional and expose the option to TVMC perhaps with a flag like `--disable-benchmark-warmup`.
   
   cc @leandron @tkonolige @areusch would be interested to hear your thoughts


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org