[GitHub] [tvm-rfcs] tkonolige commented on a diff in pull request #59: Remove tophub from being the default configuration when compiling

[GitBox <gi...@apache.org>]

tkonolige commented on code in PR #59:
URL: https://github.com/apache/tvm-rfcs/pull/59#discussion_r852435732


##########
rfcs/0059-remove-tophub-from-default.md:
##########
@@ -0,0 +1,57 @@
+- Feature Name: remove_tophub_from_default
+- Start Date: 2021-03-04
+- RFC PR: [apache/tvm-rfcs#0059](https://github.com/apache/tvm-rfcs/pull/0059)
+- GitHub Issue: [apache/tvm#10474](https://github.com/apache/tvm/pull/10474)
+
+# Summary
+[summary]: #summary
+
+Remove tophub from being the default autotvm configuration when compiling with
+the VM or graph executor. This bring thes VM and graph executor compilation
+workflow in line with all other workflows which do no use tophub by default.
+
+https://github.com/apache/tvm/pull/10474 provides a short discussion of the
+tradeoffs/issues in applying this change.
+
+# Motivation
+[motivation]: #motivation
+
+- Tophub is not maintained and is out of date in some cases. Users should
+  explicitly use it so there is no surprising behavior in using out of date
+  schedules.
+- Tophub is implicitly used only with the VM and graph executor compilation
+  flows. Other flows (like `VMCompiler.optimize`) will return different
+  results because they do not implicitly use tophub.
+- Many default schedules exist in TVM, but they are not used because the
+  tophub schedules override them. Either they should be removed or used.
+- Compilation implicitly depends on scheduling choices located outside of the
+  TVM codebase. This implicit behavior is confusing when debugging and
+  requires us to maintain a consistent state between two different repos.
+
+# Drawbacks
+[drawbacks]: #drawbacks
+
+We may see a performance degradation in models compiled without tuning as the
+tophub schedules may be more optimized then the fallback/untuned schedules. If
+this is an issue, we can port the poorly performing schedules from tophub to
+the fallback (i.e. default) configuration for the op.
+
+# Rationale and alternatives
+[rationale-and-alternatives]: #rationale-and-alternatives
+
+Alternative choices:
+
+- Use tophub as the default context when no other context has been specified.
+  This will make `VMCompiler.optimize` and `VMCompiler.lower` consistent in
+  their results.
+
+- Add tophub to `VMCompiler.optimize`. This doesn't solve the underlying
+  problem. I expect us see more inconsistent results in other parts of the
+  codebase where tophub is not used.
+
+# Unresolved questions
+[unresolved-questions]: #unresolved-questions
+
+Will there be any actual performance impact in removing tophub from being the default?

Review Comment:
   > * right now we have no way to assert that a particular schedule is actually selected or used by TVM. we ran into this with microTVM's original prototype where we had to hack the tuning log lookup to always return one schedule. if we had such a thing, we could write tests that assert that a particular TopHub schedule was applied and then we could document which GPU it's for.
   That would be very nice to have. It'll be a little bit of work to pipe through though.
   
   > * it does seem like, lacking this way of testing (and correct me if i'm missing something--perhaps we do have a way to test this), we will be forced to make a hard cutover to a world where we document which hardware each schedule should compile on via some type of test
   > * whatever we do here, there should be a very easy way to get back the old behavior since it's likely some folks will need to use TopHub
   `with autotvm.tophub.context(target)` is the way to do that (added this to the RFC).
   
   > * the root problem @tkonolige set out to address was differences in compiler output between different entry points. it seems like whatever we do above, we should absolutely make sure we shore up that discrepancy.
   This is one part of the issues I've been trying to solve. The other is behavior that is based on content external to the TVM repository.
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org