[GitHub] [tvm] mbs-octoml opened a new pull request, #12105: [Collage] PruneCandidates and demo_collage_partition.py

[GitBox <gi...@apache.org>]

mbs-octoml opened a new pull request, #12105:
URL: https://github.com/apache/tvm/pull/12105

   See https://github.com/apache/tvm-rfcs/blob/main/rfcs/0062-collage.md.
   
   This completes our checkin of our Collage 'sketch' branch into main. Special thanks
   to Matthew Barrett for his help getting this over the line.
   
   The only C++ functionality added here is for 'pruning' candidates. This is a somewhat
   speculative algorithm (and I've called that out in the comments) which tries to
   elide candidate partitions which will 'obviously' not contribute to the final optimal
   partitioning. For largish models such as GPT2 this can significantly reduce the number of
   candidates we need to actually measure latency on. I beefed up the MockCostEstimator to
   make it possible to assert pruning occured from within the test_pass_collage_partition.py
   unit test.
   
   The rest of this PR adds the demo_collage_partition.py driver file we've been using
   to test and measure perfomance differences against various baseline (though only
   for the CUDA ecosystem). To eliminate loading time the models of interest are directly
   expressed in Relay text form in menangerie.py.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] jwfromm merged pull request #12105: [Collage] PruneCandidates and demo_collage_partition.py

[GitBox <gi...@apache.org>]

jwfromm merged PR #12105:
URL: https://github.com/apache/tvm/pull/12105


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] mbs-octoml commented on a diff in pull request #12105: [Collage] PruneCandidates and demo_collage_partition.py

[GitBox <gi...@apache.org>]

mbs-octoml commented on code in PR #12105:
URL: https://github.com/apache/tvm/pull/12105#discussion_r922400184


##########
tests/python/relay/collage/demo_collage_partitioner.py:
##########
@@ -0,0 +1,401 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Compares Collage with various other baselines."""
+
+# CAUTION: Requires some changes in python/tvm/autotvm/task/dispatcher.py
+# so that AutoTVM tuning records can be cached between runs and between
+# models. See https://github.com/mbs-octoml/mbs-tvm/tree/mbs-collage-hacks.

Review Comment:
   It's so that the autotvm tuning helpers in demo_collage_partition.py can use the existing tuning records as a cache which can be shared overall all models. Ie a poor man's TRS.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] jwfromm commented on a diff in pull request #12105: [Collage] PruneCandidates and demo_collage_partition.py

[GitBox <gi...@apache.org>]

jwfromm commented on code in PR #12105:
URL: https://github.com/apache/tvm/pull/12105#discussion_r922395117


##########
tests/python/relay/collage/demo_collage_partitioner.py:
##########
@@ -0,0 +1,401 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Compares Collage with various other baselines."""
+
+# CAUTION: Requires some changes in python/tvm/autotvm/task/dispatcher.py
+# so that AutoTVM tuning records can be cached between runs and between
+# models. See https://github.com/mbs-octoml/mbs-tvm/tree/mbs-collage-hacks.

Review Comment:
   Just noting for posterity, these hacks are needed because autotvm isnt properly caching results? Does that lead to much longer tuning times than necessary or some other breakage?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org