You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/02/24 07:08:33 UTC
[GitHub] [tvm] zxybazh opened a new pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
zxybazh opened a new pull request #10366:
URL: https://github.com/apache/tvm/pull/10366
This PR is further improvement of the meta schedule project (https://github.com/apache/tvm/issues/8473).
The architecture is re-designed by Junru and Xiyou. In this PR we introduced a gradient based task scheduler which favors tuning task with more potential and calculates the gradient towards maximizing the final efficiency given computing budget. The gradients are computed in the same way as in auto scheduler. Unit test is also included.
Thanks to all co-authors for contributing!
Co-authored-by: Junru Shao <[junrushao1994@gmail.com](mailto:junrushao1994@gmail.com)>
Co-authored-by: Bohan Hou <[32121147+spectrometerHBH@users.noreply.github.com](mailto:32121147+spectrometerHBH@users.noreply.github.com)>
Co-authored-by: Ruihang Lai <[lairuihangdongdong@qq.com](mailto:lairuihangdongdong@qq.com)>
Co-authored-by: Hongyi Jin <[3231950289@qq.com](mailto:3231950289@qq.com)>
Co-authored-by: Wuwei Lin <[wuwei@apache.org](mailto:wuwei@apache.org)>
Co-authored-by: Siyuan Feng <[Hzfengsy@sjtu.edu.cn](mailto:Hzfengsy@sjtu.edu.cn)>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r840998729
##########
File path: tests/python/unittest/test_meta_schedule_tune_tir.py
##########
@@ -98,14 +97,14 @@ def test_tune_matmul_cuda_tensor_core():
target = Target("nvidia/geforce-rtx-3070")
config = ReplayTraceConfig(
num_trials_per_iter=32,
- num_trials_total=320,
+ max_trials_per_task=320,
Review comment:
I think something weird is going on.
First, `max_trials_global` is not used at https://github.com/zxybazh/tvm/blob/e3fbb797a88308e4ce3d671939a83084ae1826b8/python/tvm/meta_schedule/search_strategy/replay_trace.py#L60
And if I use
```
config=ReplayTraceConfig(
num_trials_per_iter=32,
max_trials_per_task=32,
...
```
, `max_trials_per_task` acts like a global max trials, so only one task is tuned, as shown below. This contradicts the name `max_trials_per_task ` and does something very different from the previous `num_trials_total`.
```
ID | Name | FLOP | Weight | Speed (GFLOPS) | Latency (us) | Weighted Latency (us) | Trials | Terminated
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 | fused_nn_conv2d_add | 12870144 | 1 | 51.3887 | 250.4468 | 250.4468 | 32 |
1 | fused_nn_conv2d_add_1 | 12895232 | 1 | N/A | N/A | N/A | 0 |
2 | fused_nn_conv2d_add_2 | 12945408 | 1 | N/A | N/A | N/A | 0 |
3 | fused_layout_transform | 1 | 1 | N/A | N/A | N/A | 0 |
4 | fused_nn_conv2d_add_nn_relu | 237633536 | 1 | N/A | N/A | N/A | 0 |
5 | fused_nn_max_pool2d | 1806336 | 1 | N/A | N/A | N/A | 0 |
6 | fused_nn_conv2d_add_nn_relu_1 | 231612416 | 2 | N/A | N/A | N/A | 0 |
7 | fused_nn_conv2d_add_add_nn_relu | 231813120 | 2 | N/A | N/A | N/A | 0 |
8 | fused_nn_conv2d_add_nn_relu_2 | 115806208 | 1 | N/A | N/A | N/A | 0 |
9 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_nn_relu | 93227008 | 1 | N/A | N/A | N/A | 0 |
10 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_add_nn_relu | 93327360 | 2 | N/A | N/A | N/A | 0 |
11 | fused_nn_conv2d_add_nn_relu_3 | 115705856 | 1 | N/A | N/A | N/A | 0 |
12 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_nn_relu_1 | 98600960 | 1 | N/A | N/A | N/A | 0 |
13 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_add_nn_relu_1 | 98651136 | 2 | N/A | N/A | N/A | 0 |
14 | fused_nn_conv2d_add_nn_relu_4 | 115655680 | 1 | N/A | N/A | N/A | 0 |
15 | fused_nn_conv2d_add_nn_relu_5 | 231261184 | 1 | N/A | N/A | N/A | 0 |
16 | fused_nn_conv2d_add_add_nn_relu_1 | 231286272 | 2 | N/A | N/A | N/A | 0 |
17 | fused_nn_adaptive_avg_pool2d | 25600 | 1 | N/A | N/A | N/A | 0 |
18 | fused_layout_transform_reshape_squeeze | 1 | 1 | N/A | N/A | N/A | 0 |
19 | fused_nn_dense_add | 1025000 | 1 | N/A | N/A | N/A | 0 |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r840995932
##########
File path: tests/python/unittest/test_meta_schedule_tune_tir.py
##########
@@ -98,14 +97,14 @@ def test_tune_matmul_cuda_tensor_core():
target = Target("nvidia/geforce-rtx-3070")
config = ReplayTraceConfig(
num_trials_per_iter=32,
- num_trials_total=320,
+ max_trials_per_task=320,
Review comment:
@zxybazh @junrushao1994 All uses of `ReplayTraceConfig` in tests are broken since now it has three attributes https://github.com/zxybazh/tvm/blob/e3fbb797a88308e4ce3d671939a83084ae1826b8/python/tvm/meta_schedule/search_strategy/replay_trace.py#L52-L57
But they are not detected in CI since the tests are skipped.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r840998729
##########
File path: tests/python/unittest/test_meta_schedule_tune_tir.py
##########
@@ -98,14 +97,14 @@ def test_tune_matmul_cuda_tensor_core():
target = Target("nvidia/geforce-rtx-3070")
config = ReplayTraceConfig(
num_trials_per_iter=32,
- num_trials_total=320,
+ max_trials_per_task=320,
Review comment:
I think something weird is going on.
First, `max_trials_global` is not used at https://github.com/zxybazh/tvm/blob/e3fbb797a88308e4ce3d671939a83084ae1826b8/python/tvm/meta_schedule/search_strategy/replay_trace.py#L60
And if I use
```
config=ReplayTraceConfig(
num_trials_per_iter=32,
max_trials_per_task=32,
...
```
, `max_trials_per_task` acts like a global max trials, so only one task is tuned, as shown below. So to correctly tune all tasks, I need to set `max_trials_per_task = 32 * num_tasks`, which contradicts the name `max_trials_per_task ` and does something very different from the previous `num_trials_total`.
```
ID | Name | FLOP | Weight | Speed (GFLOPS) | Latency (us) | Weighted Latency (us) | Trials | Terminated
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 | fused_nn_conv2d_add | 12870144 | 1 | 51.3887 | 250.4468 | 250.4468 | 32 |
1 | fused_nn_conv2d_add_1 | 12895232 | 1 | N/A | N/A | N/A | 0 |
2 | fused_nn_conv2d_add_2 | 12945408 | 1 | N/A | N/A | N/A | 0 |
3 | fused_layout_transform | 1 | 1 | N/A | N/A | N/A | 0 |
4 | fused_nn_conv2d_add_nn_relu | 237633536 | 1 | N/A | N/A | N/A | 0 |
5 | fused_nn_max_pool2d | 1806336 | 1 | N/A | N/A | N/A | 0 |
6 | fused_nn_conv2d_add_nn_relu_1 | 231612416 | 2 | N/A | N/A | N/A | 0 |
7 | fused_nn_conv2d_add_add_nn_relu | 231813120 | 2 | N/A | N/A | N/A | 0 |
8 | fused_nn_conv2d_add_nn_relu_2 | 115806208 | 1 | N/A | N/A | N/A | 0 |
9 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_nn_relu | 93227008 | 1 | N/A | N/A | N/A | 0 |
10 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_add_nn_relu | 93327360 | 2 | N/A | N/A | N/A | 0 |
11 | fused_nn_conv2d_add_nn_relu_3 | 115705856 | 1 | N/A | N/A | N/A | 0 |
12 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_nn_relu_1 | 98600960 | 1 | N/A | N/A | N/A | 0 |
13 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_add_nn_relu_1 | 98651136 | 2 | N/A | N/A | N/A | 0 |
14 | fused_nn_conv2d_add_nn_relu_4 | 115655680 | 1 | N/A | N/A | N/A | 0 |
15 | fused_nn_conv2d_add_nn_relu_5 | 231261184 | 1 | N/A | N/A | N/A | 0 |
16 | fused_nn_conv2d_add_add_nn_relu_1 | 231286272 | 2 | N/A | N/A | N/A | 0 |
17 | fused_nn_adaptive_avg_pool2d | 25600 | 1 | N/A | N/A | N/A | 0 |
18 | fused_layout_transform_reshape_squeeze | 1 | 1 | N/A | N/A | N/A | 0 |
19 | fused_nn_dense_add | 1025000 | 1 | N/A | N/A | N/A | 0 |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r833854183
##########
File path: src/meta_schedule/utils.h
##########
@@ -50,6 +51,201 @@
#include "../tir/schedule/utils.h"
namespace tvm {
+namespace tir {
+
+inline double CountFlop(const IRModule& mod) {
+ struct TResult {
+ using TTable = std::unordered_map<int32_t, double>;
+
+ TResult() = default;
+
+ explicit TResult(const tvm::DataType& dtype) { Add(dtype); }
+
+ void Add(const tvm::DataType& dtype) { data_[DataType2Int(dtype)] += 1; }
+
+ TResult operator+=(const TResult& rhs) {
+ for (const auto& kv : rhs.data_) {
+ data_[kv.first] += kv.second;
+ }
+ return *this;
+ }
+
+ TResult operator*=(int64_t rhs) {
+ for (auto& kv : data_) {
+ kv.second *= rhs;
+ }
+ return *this;
+ }
+
+ TResult MaxWith(const TResult& rhs) {
+ for (const auto& kv : rhs.data_) {
+ double& v = data_[kv.first];
+ if (v < kv.second) {
+ v = kv.second;
+ }
+ }
+ return *this;
+ }
+
+ struct DType {
+ uint8_t code : 8;
+ uint8_t bits : 8;
+ uint16_t lanes : 16;
+ };
+ static_assert(sizeof(DType) == 4, "Incorrect size of DType");
+
+ static String Int2Str(int32_t dtype) {
+ union {
+ DType dst;
+ int32_t src;
+ } converter;
+ converter.src = dtype;
+ static std::string type_code_tab[] = {"int", "uint", "float", "handle", "bfloat"};
+ std::ostringstream os;
+ os << type_code_tab[converter.dst.code];
+ os << static_cast<int>(converter.dst.bits);
+ if (converter.dst.lanes != 1) {
+ os << "x" << static_cast<int>(converter.dst.lanes);
+ }
+ return os.str();
+ }
+
+ static int32_t DataType2Int(const tvm::DataType& dtype) {
+ union {
+ DType src;
+ int32_t dst;
+ } converter;
+ converter.src.code = dtype.code();
+ converter.src.bits = dtype.bits();
+ converter.src.lanes = dtype.lanes();
+ return converter.dst;
+ }
+
+ TTable data_;
+ };
+
+ class FlopCounter : public ExprFunctor<TResult(const PrimExpr& n)>,
+ public StmtFunctor<TResult(const Stmt& n)> {
+ public:
+ ~FlopCounter() {}
+
+ TResult VisitExpr(const PrimExpr& expr) override { return ExprFunctor::VisitExpr(expr); }
+ TResult VisitStmt(const Stmt& stmt) override { return StmtFunctor::VisitStmt(stmt); }
+
+ TResult VisitStmt_(const IfThenElseNode* branch) override {
+ TResult cond = VisitExpr(branch->condition);
+ cond += VisitStmt(branch->then_case).MaxWith(VisitStmt(branch->else_case));
+ return cond;
+ }
+
+ TResult VisitStmt_(const BufferStoreNode* store) override {
+ TResult result = VisitExpr(store->value);
+ for (const PrimExpr& e : store->indices) {
+ result += VisitExpr(e);
+ }
+ return result;
+ }
+
+ TResult VisitStmt_(const SeqStmtNode* seq) override {
+ TResult result;
+ for (const Stmt& stmt : seq->seq) {
+ result += VisitStmt(stmt);
+ }
+ return result;
+ }
+
+ TResult VisitStmt_(const BlockRealizeNode* block) override {
+ return VisitStmt(block->block->body);
+ }
+
+ TResult VisitStmt_(const BlockNode* block) override {
+ TResult result;
+ if (block->init.defined()) {
+ result += VisitStmt(block->init.value());
+ }
+ result += VisitStmt(block->body);
+ return result;
+ }
+
+ TResult VisitStmt_(const ForNode* loop) override {
+ TResult result = VisitStmt(loop->body);
+ const auto* int_imm = loop->extent.as<IntImmNode>();
+ ICHECK(int_imm) << "TypeError: Expect the extent of a loop to be IntImm, but gets: "
+ << loop->extent->GetTypeKey();
+ result *= int_imm->value;
+ return result;
+ }
+
+#define TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(Node) \
+ TResult VisitExpr_(const Node* op) final { \
+ TResult result(op->dtype); \
+ result += VisitExpr(op->a); \
+ result += VisitExpr(op->b); \
+ return result; \
+ }
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(AddNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(SubNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(MulNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(DivNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(ModNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(FloorDivNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(FloorModNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(MinNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(MaxNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(EQNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(NENode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(LTNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(LENode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(GTNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(GENode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(AndNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(OrNode);
+#undef TVM_META_SCHEDULE_FLOP_COUNTER_BINARY
+ TResult VisitExpr_(const CastNode* op) override { return VisitExpr(op->value); }
+ TResult VisitExpr_(const VarNode* op) override { return TResult(); }
+ TResult VisitExpr_(const SizeVarNode* op) override { return TResult(); }
+ TResult VisitExpr_(const BufferLoadNode* op) override { return TResult(); }
+ TResult VisitExpr_(const IntImmNode* op) override { return TResult(); }
+ TResult VisitExpr_(const FloatImmNode* op) override { return TResult(); }
+ TResult VisitExpr_(const NotNode* op) override {
+ TResult result(op->dtype);
+ result += VisitExpr(op->a);
+ return result;
+ }
+ TResult VisitExpr_(const SelectNode* op) override {
+ TResult cond = VisitExpr(op->condition);
+ cond += VisitExpr(op->true_value).MaxWith(VisitExpr(op->false_value));
+ return cond;
+ }
+ TResult VisitExpr_(const CallNode* op) override {
+ TResult ret;
+ for (const auto& x : op->args) {
+ ret += VisitExpr(x);
+ }
+ return ret;
+ }
+ };
+ FlopCounter counter;
+ TResult result;
+ for (const auto& kv : mod->functions) {
+ const BaseFunc& base_func = kv.second;
+ if (const auto* prim_func = base_func.as<PrimFuncNode>()) {
+ result += counter.VisitStmt(prim_func->body);
+ }
+ }
+ double cnt = 0.0;
+ int i32 = TResult::DataType2Int(tvm::DataType::Int(32));
+ int i64 = TResult::DataType2Int(tvm::DataType::Int(64));
+ int u1 = TResult::DataType2Int(tvm::DataType::UInt(1));
+ for (const auto& kv : result.data_) {
+ if (kv.first != i32 && kv.first != i64 && kv.first != u1) {
Review comment:
We will lift the method as a standalone analysis function and expose it for more general usage. Will work on that after @zxybazh’s vacation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
zxybazh commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r816411413
##########
File path: src/meta_schedule/task_scheduler/gradient_based.cc
##########
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include "../utils.h"
+
+namespace tvm {
+namespace meta_schedule {
+
+/*! \brief The gradient based task scheduler. */
+class GradientBasedNode final : public TaskSchedulerNode {
+ public:
+ int backward_window_size;
+ double alpha, beta;
+
+ bool done_round_robin; // whether the warm up round robin has been done
+ int task_id = -1; // The current task id processed.
+ support::LinearCongruentialEngine::TRandState rand_state; // the random state
+ std::vector<int> task_cnts; // task tuning counts
+ std::vector<double> task_weights; // task weights
+ std::vector<double> task_best_latencies; // best latency achived by the task
+ std::vector<double> task_flop_counts; // total flop count of the task
+ std::vector<std::vector<double>> task_latency_history; // all task latency history
+
+ std::vector<std::string> task_tag; // tag of the task for grouping
+ std::map<std::string, int> tag_to_group; // map to find the group id given task tag
+ std::vector<std::set<int>> task_groups; // the task ids in a given group
+
+ TaskSchedulerNode::FObjectiveFunc objective_func; // the objective function
+
+ void VisitAttrs(tvm::AttrVisitor* v) {
+ TaskSchedulerNode::VisitAttrs(v);
+ v->Visit("task_id", &task_id);
+ }
+
+ static constexpr const char* _type_key = "meta_schedule.GradientBased";
+ TVM_DECLARE_FINAL_OBJECT_INFO(GradientBasedNode, TaskSchedulerNode);
+
+ protected:
+ double _compute_score(const std::vector<double>& latencies) {
+ Array<FloatImm> input_latencies;
+ for (double latency : latencies)
+ input_latencies.push_back(FloatImm(DataType::Float(32), latency));
+ return objective_func(input_latencies)->value;
+ }
+
+ void _adjust_similarity_group(int task_id) {
+ int group_id = tag_to_group[task_tag[task_id]];
+ if (task_groups[group_id].size() <= 1 ||
+ task_groups[group_id].find(task_id) == task_groups[group_id].end())
+ return;
+
+ double best_flops = -1.0;
+ int max_ct[3] = {-1, -1, -1}; // to find the 2nd largest
+ for (int i : task_groups[group_id]) {
+ best_flops = std::max(best_flops, task_flop_counts[i] / task_best_latencies[i]);
+ max_ct[0] = task_cnts[i];
+ std::sort(max_ct, max_ct + 3);
+ }
+ double cur_flops = task_flop_counts[task_id] / task_best_latencies[task_id];
+ // if we tune a task for many times but it still cannot achieve
+ // a similar speed to the fastest one in its group, this means this task
+ // is actually not similar to other tasks in its group.
+ // So we will remove it from its original group.
+ if (cur_flops < best_flops / beta && task_cnts[task_id] > 5 + max_ct[1]) {
+ task_groups[group_id].erase(task_id);
+ }
Review comment:
Thanks for point it out, originally I checked if a couple lines above that the groups won't be changed if the group set won't find the task. I have rewritten the logic to create a single element group afterwards.
##########
File path: src/meta_schedule/task_scheduler/gradient_based.cc
##########
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include "../utils.h"
+
+namespace tvm {
+namespace meta_schedule {
+
+/*! \brief The gradient based task scheduler. */
+class GradientBasedNode final : public TaskSchedulerNode {
+ public:
+ int backward_window_size;
+ double alpha, beta;
+
+ bool done_round_robin; // whether the warm up round robin has been done
+ int task_id = -1; // The current task id processed.
+ support::LinearCongruentialEngine::TRandState rand_state; // the random state
+ std::vector<int> task_cnts; // task tuning counts
+ std::vector<double> task_weights; // task weights
+ std::vector<double> task_best_latencies; // best latency achived by the task
+ std::vector<double> task_flop_counts; // total flop count of the task
+ std::vector<std::vector<double>> task_latency_history; // all task latency history
+
+ std::vector<std::string> task_tag; // tag of the task for grouping
+ std::map<std::string, int> tag_to_group; // map to find the group id given task tag
+ std::vector<std::set<int>> task_groups; // the task ids in a given group
+
+ TaskSchedulerNode::FObjectiveFunc objective_func; // the objective function
+
+ void VisitAttrs(tvm::AttrVisitor* v) {
+ TaskSchedulerNode::VisitAttrs(v);
+ v->Visit("task_id", &task_id);
+ }
+
+ static constexpr const char* _type_key = "meta_schedule.GradientBased";
+ TVM_DECLARE_FINAL_OBJECT_INFO(GradientBasedNode, TaskSchedulerNode);
+
+ protected:
+ double _compute_score(const std::vector<double>& latencies) {
+ Array<FloatImm> input_latencies;
+ for (double latency : latencies)
+ input_latencies.push_back(FloatImm(DataType::Float(32), latency));
+ return objective_func(input_latencies)->value;
+ }
+
+ void _adjust_similarity_group(int task_id) {
+ int group_id = tag_to_group[task_tag[task_id]];
+ if (task_groups[group_id].size() <= 1 ||
+ task_groups[group_id].find(task_id) == task_groups[group_id].end())
+ return;
+
+ double best_flops = -1.0;
+ int max_ct[3] = {-1, -1, -1}; // to find the 2nd largest
+ for (int i : task_groups[group_id]) {
+ best_flops = std::max(best_flops, task_flop_counts[i] / task_best_latencies[i]);
+ max_ct[0] = task_cnts[i];
+ std::sort(max_ct, max_ct + 3);
+ }
+ double cur_flops = task_flop_counts[task_id] / task_best_latencies[task_id];
+ // if we tune a task for many times but it still cannot achieve
+ // a similar speed to the fastest one in its group, this means this task
+ // is actually not similar to other tasks in its group.
+ // So we will remove it from its original group.
+ if (cur_flops < best_flops / beta && task_cnts[task_id] > 5 + max_ct[1]) {
+ task_groups[group_id].erase(task_id);
+ }
Review comment:
Thanks for pointing it out, originally I checked if a couple lines above that the groups won't be changed if the group set won't find the task. I have rewritten the logic to create a single element group afterwards.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
zxybazh commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r816411413
##########
File path: src/meta_schedule/task_scheduler/gradient_based.cc
##########
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include "../utils.h"
+
+namespace tvm {
+namespace meta_schedule {
+
+/*! \brief The gradient based task scheduler. */
+class GradientBasedNode final : public TaskSchedulerNode {
+ public:
+ int backward_window_size;
+ double alpha, beta;
+
+ bool done_round_robin; // whether the warm up round robin has been done
+ int task_id = -1; // The current task id processed.
+ support::LinearCongruentialEngine::TRandState rand_state; // the random state
+ std::vector<int> task_cnts; // task tuning counts
+ std::vector<double> task_weights; // task weights
+ std::vector<double> task_best_latencies; // best latency achived by the task
+ std::vector<double> task_flop_counts; // total flop count of the task
+ std::vector<std::vector<double>> task_latency_history; // all task latency history
+
+ std::vector<std::string> task_tag; // tag of the task for grouping
+ std::map<std::string, int> tag_to_group; // map to find the group id given task tag
+ std::vector<std::set<int>> task_groups; // the task ids in a given group
+
+ TaskSchedulerNode::FObjectiveFunc objective_func; // the objective function
+
+ void VisitAttrs(tvm::AttrVisitor* v) {
+ TaskSchedulerNode::VisitAttrs(v);
+ v->Visit("task_id", &task_id);
+ }
+
+ static constexpr const char* _type_key = "meta_schedule.GradientBased";
+ TVM_DECLARE_FINAL_OBJECT_INFO(GradientBasedNode, TaskSchedulerNode);
+
+ protected:
+ double _compute_score(const std::vector<double>& latencies) {
+ Array<FloatImm> input_latencies;
+ for (double latency : latencies)
+ input_latencies.push_back(FloatImm(DataType::Float(32), latency));
+ return objective_func(input_latencies)->value;
+ }
+
+ void _adjust_similarity_group(int task_id) {
+ int group_id = tag_to_group[task_tag[task_id]];
+ if (task_groups[group_id].size() <= 1 ||
+ task_groups[group_id].find(task_id) == task_groups[group_id].end())
+ return;
+
+ double best_flops = -1.0;
+ int max_ct[3] = {-1, -1, -1}; // to find the 2nd largest
+ for (int i : task_groups[group_id]) {
+ best_flops = std::max(best_flops, task_flop_counts[i] / task_best_latencies[i]);
+ max_ct[0] = task_cnts[i];
+ std::sort(max_ct, max_ct + 3);
+ }
+ double cur_flops = task_flop_counts[task_id] / task_best_latencies[task_id];
+ // if we tune a task for many times but it still cannot achieve
+ // a similar speed to the fastest one in its group, this means this task
+ // is actually not similar to other tasks in its group.
+ // So we will remove it from its original group.
+ if (cur_flops < best_flops / beta && task_cnts[task_id] > 5 + max_ct[1]) {
+ task_groups[group_id].erase(task_id);
+ }
Review comment:
Thanks for pointing it out, originally I checked a couple lines above that the groups won't be changed if the group set won't find the task. I have rewritten the logic to create a single element group afterwards.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Hzfengsy merged pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
Hzfengsy merged pull request #10366:
URL: https://github.com/apache/tvm/pull/10366
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
zxybazh commented on pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#issuecomment-1083687484
Hey Junru, the changes of API looks reasonable to me. Would you please retrigger the CI?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r841030902
##########
File path: tests/python/unittest/test_meta_schedule_tune_tir.py
##########
@@ -98,14 +97,14 @@ def test_tune_matmul_cuda_tensor_core():
target = Target("nvidia/geforce-rtx-3070")
config = ReplayTraceConfig(
num_trials_per_iter=32,
- num_trials_total=320,
+ max_trials_per_task=320,
Review comment:
> We updated the usage of this API in the following files:
python/tvm/meta_schedule/testing/tune_te_meta_schedule.py
python/tvm/meta_schedule/testing/tune_relay_meta_schedule.py
tests/python/unittest/test_meta_schedule_measure_callback.py
tests/python/unittest/test_meta_schedule_search_strategy.py
tests/python/unittest/test_meta_schedule_task_scheduler.py
tests/python/unittest/test_meta_schedule_tune_relay.py
tests/python/unittest/test_meta_schedule_tune_te.py
tests/python/unittest/test_meta_schedule_tune_tir.py
I don't see any updated usage... can you double check? Actually `ReplayTraceConfig` seems to be used only in three integration test files.
But yeah, I didn't notice that `max_trials_global` is used outside of `ReplayTraceConfig` class as `config.max_trials_global`, so I thought its value doesn't matter. I'll revisit this issue after seeing your fix to the integration tests.
I understand the risk of being an early user, but I'd like to see at least reasonable documentation of the API at all times, regardless of being stable / unstable. Especially when you introduce a breaking change. In this particular case, the new attribute `max_trials_global` is not documented and there is no example of how to use this value, as far as I can see.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r840998729
##########
File path: tests/python/unittest/test_meta_schedule_tune_tir.py
##########
@@ -98,14 +97,14 @@ def test_tune_matmul_cuda_tensor_core():
target = Target("nvidia/geforce-rtx-3070")
config = ReplayTraceConfig(
num_trials_per_iter=32,
- num_trials_total=320,
+ max_trials_per_task=320,
Review comment:
I think something weird is going on.
First, `max_trials_global` is not used at https://github.com/zxybazh/tvm/blob/e3fbb797a88308e4ce3d671939a83084ae1826b8/python/tvm/meta_schedule/search_strategy/replay_trace.py#L60
And if I use
```
config=ReplayTraceConfig(
num_trials_per_iter=32,
max_trials_per_task=32,
...
```
, `max_trials_per_task` acts like a global max trials, so only one task is tuned, as shown below. So to correctly tune all tasks, I need to set `max_trials_per_task = 32 * num_tasks`, which contradicts the name `max_trials_per_task ` and does something very different from the previous `num_trials_total`.
In other words, `num_trials_per_iter` is acting like `max_trials_per_task`, and `max_trials_per_task` is acting like `max_trials_global`.
```
ID | Name | FLOP | Weight | Speed (GFLOPS) | Latency (us) | Weighted Latency (us) | Trials | Terminated
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 | fused_nn_conv2d_add | 12870144 | 1 | 51.3887 | 250.4468 | 250.4468 | 32 |
1 | fused_nn_conv2d_add_1 | 12895232 | 1 | N/A | N/A | N/A | 0 |
2 | fused_nn_conv2d_add_2 | 12945408 | 1 | N/A | N/A | N/A | 0 |
3 | fused_layout_transform | 1 | 1 | N/A | N/A | N/A | 0 |
4 | fused_nn_conv2d_add_nn_relu | 237633536 | 1 | N/A | N/A | N/A | 0 |
5 | fused_nn_max_pool2d | 1806336 | 1 | N/A | N/A | N/A | 0 |
6 | fused_nn_conv2d_add_nn_relu_1 | 231612416 | 2 | N/A | N/A | N/A | 0 |
7 | fused_nn_conv2d_add_add_nn_relu | 231813120 | 2 | N/A | N/A | N/A | 0 |
8 | fused_nn_conv2d_add_nn_relu_2 | 115806208 | 1 | N/A | N/A | N/A | 0 |
9 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_nn_relu | 93227008 | 1 | N/A | N/A | N/A | 0 |
10 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_add_nn_relu | 93327360 | 2 | N/A | N/A | N/A | 0 |
11 | fused_nn_conv2d_add_nn_relu_3 | 115705856 | 1 | N/A | N/A | N/A | 0 |
12 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_nn_relu_1 | 98600960 | 1 | N/A | N/A | N/A | 0 |
13 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_add_nn_relu_1 | 98651136 | 2 | N/A | N/A | N/A | 0 |
14 | fused_nn_conv2d_add_nn_relu_4 | 115655680 | 1 | N/A | N/A | N/A | 0 |
15 | fused_nn_conv2d_add_nn_relu_5 | 231261184 | 1 | N/A | N/A | N/A | 0 |
16 | fused_nn_conv2d_add_add_nn_relu_1 | 231286272 | 2 | N/A | N/A | N/A | 0 |
17 | fused_nn_adaptive_avg_pool2d | 25600 | 1 | N/A | N/A | N/A | 0 |
18 | fused_layout_transform_reshape_squeeze | 1 | 1 | N/A | N/A | N/A | 0 |
19 | fused_nn_dense_add | 1025000 | 1 | N/A | N/A | N/A | 0 |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r815189229
##########
File path: python/tvm/meta_schedule/task_scheduler/gradient_based.py
##########
@@ -0,0 +1,146 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Gradient Based Task Scheduler"""
+import math
+
+from typing import TYPE_CHECKING, List, Optional
+from tvm._ffi import register_object
+from tvm._ffi.registry import register_func
+
+from tvm.ir import IRModule
+from ..measure_callback import MeasureCallback
+from ..builder import Builder
+from ..runner import Runner
+from ..database import Database
+from ..cost_model import CostModel
+from .task_scheduler import TaskScheduler
+
+from .. import _ffi_api
+
+if TYPE_CHECKING:
+ from ..tune_context import TuneContext
+
+
+@register_func("meta_schedule.task_scheduler.derive_similarity_tag")
+def derive_similarity_tag(mod: IRModule, log_base: float = 1.618) -> str:
+ """Get the tags for smilarity group creation
+
+ Parameters
+ ---------
+ mod : IRModule
+ The input workload.
+ log_base : float
+ The log base to normalize the flop count.
Review comment:
Out of curiosity, why log base matters? It seems to me that you only use it for FLOP (to avoid overflow)?
```suggestion
The log base to normalize the flop count. Default natural (1.618).
```
##########
File path: python/tvm/meta_schedule/task_scheduler/gradient_based.py
##########
@@ -0,0 +1,146 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Gradient Based Task Scheduler"""
+import math
+
+from typing import TYPE_CHECKING, List, Optional
+from tvm._ffi import register_object
+from tvm._ffi.registry import register_func
+
+from tvm.ir import IRModule
+from ..measure_callback import MeasureCallback
+from ..builder import Builder
+from ..runner import Runner
+from ..database import Database
+from ..cost_model import CostModel
+from .task_scheduler import TaskScheduler
+
+from .. import _ffi_api
+
+if TYPE_CHECKING:
+ from ..tune_context import TuneContext
+
+
+@register_func("meta_schedule.task_scheduler.derive_similarity_tag")
+def derive_similarity_tag(mod: IRModule, log_base: float = 1.618) -> str:
+ """Get the tags for smilarity group creation
+
+ Parameters
+ ---------
Review comment:
```suggestion
----------
```
##########
File path: python/tvm/meta_schedule/task_scheduler/gradient_based.py
##########
@@ -0,0 +1,146 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Gradient Based Task Scheduler"""
+import math
+
+from typing import TYPE_CHECKING, List, Optional
+from tvm._ffi import register_object
+from tvm._ffi.registry import register_func
+
+from tvm.ir import IRModule
+from ..measure_callback import MeasureCallback
+from ..builder import Builder
+from ..runner import Runner
+from ..database import Database
+from ..cost_model import CostModel
+from .task_scheduler import TaskScheduler
+
+from .. import _ffi_api
+
+if TYPE_CHECKING:
+ from ..tune_context import TuneContext
+
+
+@register_func("meta_schedule.task_scheduler.derive_similarity_tag")
+def derive_similarity_tag(mod: IRModule, log_base: float = 1.618) -> str:
+ """Get the tags for smilarity group creation
+
+ Parameters
+ ---------
+ mod : IRModule
+ The input workload.
+ log_base : float
+ The log base to normalize the flop count.
+
+ Return
+ ------
+ tag : str
+ The generated similarity tag.
+ """
+ ret = ""
+ for var in mod.get_global_vars():
+
Review comment:
remove this empty line.
##########
File path: src/meta_schedule/task_scheduler/gradient_based.cc
##########
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include "../utils.h"
+
+namespace tvm {
+namespace meta_schedule {
+
+/*! \brief The gradient based task scheduler. */
+class GradientBasedNode final : public TaskSchedulerNode {
+ public:
+ int backward_window_size;
+ double alpha, beta;
+
+ bool done_round_robin; // whether the warm up round robin has been done
+ int task_id = -1; // The current task id processed.
+ support::LinearCongruentialEngine::TRandState rand_state; // the random state
+ std::vector<int> task_cnts; // task tuning counts
+ std::vector<double> task_weights; // task weights
+ std::vector<double> task_best_latencies; // best latency achived by the task
+ std::vector<double> task_flop_counts; // total flop count of the task
+ std::vector<std::vector<double>> task_latency_history; // all task latency history
+
+ std::vector<std::string> task_tag; // tag of the task for grouping
+ std::map<std::string, int> tag_to_group; // map to find the group id given task tag
+ std::vector<std::set<int>> task_groups; // the task ids in a given group
+
+ TaskSchedulerNode::FObjectiveFunc objective_func; // the objective function
+
+ void VisitAttrs(tvm::AttrVisitor* v) {
+ TaskSchedulerNode::VisitAttrs(v);
+ v->Visit("task_id", &task_id);
+ }
+
+ static constexpr const char* _type_key = "meta_schedule.GradientBased";
+ TVM_DECLARE_FINAL_OBJECT_INFO(GradientBasedNode, TaskSchedulerNode);
+
+ protected:
+ double _compute_score(const std::vector<double>& latencies) {
+ Array<FloatImm> input_latencies;
+ for (double latency : latencies)
+ input_latencies.push_back(FloatImm(DataType::Float(32), latency));
+ return objective_func(input_latencies)->value;
+ }
+
+ void _adjust_similarity_group(int task_id) {
+ int group_id = tag_to_group[task_tag[task_id]];
+ if (task_groups[group_id].size() <= 1 ||
+ task_groups[group_id].find(task_id) == task_groups[group_id].end())
+ return;
+
+ double best_flops = -1.0;
+ int max_ct[3] = {-1, -1, -1}; // to find the 2nd largest
+ for (int i : task_groups[group_id]) {
+ best_flops = std::max(best_flops, task_flop_counts[i] / task_best_latencies[i]);
+ max_ct[0] = task_cnts[i];
+ std::sort(max_ct, max_ct + 3);
+ }
+ double cur_flops = task_flop_counts[task_id] / task_best_latencies[task_id];
+ // if we tune a task for many times but it still cannot achieve
+ // a similar speed to the fastest one in its group, this means this task
+ // is actually not similar to other tasks in its group.
+ // So we will remove it from its original group.
+ if (cur_flops < best_flops / beta && task_cnts[task_id] > 5 + max_ct[1]) {
+ task_groups[group_id].erase(task_id);
+ }
Review comment:
What happens afterward? Will this task join another group?
Also should `tag_to_group` be updated as well?
##########
File path: python/tvm/meta_schedule/task_scheduler/gradient_based.py
##########
@@ -0,0 +1,146 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Gradient Based Task Scheduler"""
+import math
+
+from typing import TYPE_CHECKING, List, Optional
+from tvm._ffi import register_object
+from tvm._ffi.registry import register_func
+
+from tvm.ir import IRModule
+from ..measure_callback import MeasureCallback
+from ..builder import Builder
+from ..runner import Runner
+from ..database import Database
+from ..cost_model import CostModel
+from .task_scheduler import TaskScheduler
+
+from .. import _ffi_api
+
+if TYPE_CHECKING:
+ from ..tune_context import TuneContext
+
+
+@register_func("meta_schedule.task_scheduler.derive_similarity_tag")
+def derive_similarity_tag(mod: IRModule, log_base: float = 1.618) -> str:
+ """Get the tags for smilarity group creation
+
+ Parameters
+ ---------
+ mod : IRModule
+ The input workload.
+ log_base : float
+ The log base to normalize the flop count.
+
+ Return
+ ------
+ tag : str
+ The generated similarity tag.
+ """
+ ret = ""
+ for var in mod.get_global_vars():
+
+ if "meta_scheduler_task_scheduler_tag" in mod[var].attrs:
+ ret += mod[var].attrs.meta_scheduler_task_scheduler_tag + "_"
+ if ret:
+ flop_count = _ffi_api.TaskSchedulerFlopCount(mod) # type: ignore # pylint: disable=no-member
+ ret += "%d" % int(math.log(flop_count + 1, log_base))
+ return ret
+
+
+@register_object("meta_schedule.GradientBased")
+class GradientBased(TaskScheduler):
+ """Gradient Based Task Scheduler"""
+
+ def __init__(
+ self,
+ tasks: List["TuneContext"],
+ builder: Builder,
+ runner: Runner,
+ database: Database,
+ *,
+ alpha: float = 0.2,
+ beta: float = 2.0,
+ backward_window_size: int = 3,
+ seed: int = -1,
+ task_weights: List[float] = None,
+ objective_func_name: str = "meta_schedule.task_scheduler.objective_func",
+ tag_generation_func_name: str = "meta_schedule.task_scheduler.derive_similarity_tag",
+ cost_model: Optional[CostModel] = None,
+ measure_callbacks: Optional[List[MeasureCallback]] = None,
+ ) -> None:
+ """Constructor.
+
+ Parameters
+ ----------
+ tasks : List[TuneContext]
+ List of tasks to schedule.
+ builder : Builder
+ The builder.
+ runner : Runner
+ The runner.
+ database : Database
+ The database.
+ alpha: float
+ The parameter alpha to control gradient computation.
+ beta: float
+ The parameter beta to control gradient computation.
+ backward_window_size: int
+ The parameter to control backward window size.
+ seed: int
+ The random seed.
Review comment:
Please specify the default values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
zxybazh commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r814344180
##########
File path: src/meta_schedule/task_scheduler/gradient_based.cc
##########
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include "../utils.h"
+
+namespace tvm {
+namespace meta_schedule {
+
+/*! \brief The gradient based task scheduler. */
+class GradientBasedNode final : public TaskSchedulerNode {
+ public:
+ int backward_window_size;
+ double alpha, beta;
+
+ bool done_round_robin; // whether the warm up round robin has been done
+ int task_id = -1; // The current task id processed.
+ support::LinearCongruentialEngine::TRandState rand_state; // the random state
+ std::vector<int> task_cnts; // task tuning counts
+ std::vector<double> task_weights; // task weights
+ std::vector<double> task_best_latencies; // best latency achived by the task
+ std::vector<double> task_flop_counts; // total flop count of the task
+ std::vector<std::vector<double>> task_latency_history; // all task latency history
+
+ std::vector<std::string> task_tag; // tag of the task for grouping
+ std::map<std::string, int> tag_to_group; // map to find the group id given task tag
+ std::vector<std::set<int>> task_groups; // the task ids in a given group
+
+ TaskSchedulerNode::FObjectiveFunc objective_func; // the objective function
+
+ void VisitAttrs(tvm::AttrVisitor* v) {
+ TaskSchedulerNode::VisitAttrs(v);
+ v->Visit("task_id", &task_id);
+ }
+
+ static constexpr const char* _type_key = "meta_schedule.GradientBased";
+ TVM_DECLARE_FINAL_OBJECT_INFO(GradientBasedNode, TaskSchedulerNode);
+
+ protected:
+ double _compute_score(const std::vector<double>& latencies) {
+ Array<FloatImm> input_latencies;
+ for (double latency : latencies)
+ input_latencies.push_back(FloatImm(DataType::Float(32), latency));
+ return objective_func(input_latencies)->value;
+ }
+
+ void _adjust_similarity_group(int task_id) {
+ int group_id = tag_to_group[task_tag[task_id]];
+ if (task_groups[group_id].size() <= 1 ||
+ task_groups[group_id].find(task_id) == task_groups[group_id].end())
+ return;
+
+ double best_flops = -1.0;
+ int max_ct[3] = {-1, -1, -1}; // to find the 2nd largest
+ for (int i : task_groups[group_id]) {
+ best_flops = std::max(best_flops, task_flop_counts[i] / task_best_latencies[i]);
+ max_ct[0] = task_cnts[i];
+ std::sort(max_ct, max_ct + 3);
+ }
+ double cur_flops = task_flop_counts[task_id] / task_best_latencies[task_id];
+ // if we tune a task for many times but it still cannot achieve
+ // a similar speed to the fastest one in its group, this means this task
+ // is actually not similar to other tasks in its group.
+ // So we will remove it from its original group.
+ if (cur_flops < best_flops / beta && task_cnts[task_id] > 5 + max_ct[1]) {
+ task_groups[group_id].erase(task_id);
+ }
+ }
+
+ int NextTaskId() final {
+ int n_tasks = this->tasks.size();
+ // Check if warmed up with round robin
+ if (!done_round_robin) {
+ task_id++;
+ if (task_id == n_tasks) {
+ done_round_robin = true;
+ } else {
+ return task_id;
+ }
+ }
Review comment:
Simply checking `task_cnts` would suffice, no need to use `task_id` acutally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r825528340
##########
File path: src/meta_schedule/utils.h
##########
@@ -50,6 +51,201 @@
#include "../tir/schedule/utils.h"
namespace tvm {
+namespace tir {
+
+inline double CountFlop(const IRModule& mod) {
+ struct TResult {
+ using TTable = std::unordered_map<int32_t, double>;
+
+ TResult() = default;
+
+ explicit TResult(const tvm::DataType& dtype) { Add(dtype); }
+
+ void Add(const tvm::DataType& dtype) { data_[DataType2Int(dtype)] += 1; }
+
+ TResult operator+=(const TResult& rhs) {
+ for (const auto& kv : rhs.data_) {
+ data_[kv.first] += kv.second;
+ }
+ return *this;
+ }
+
+ TResult operator*=(int64_t rhs) {
+ for (auto& kv : data_) {
+ kv.second *= rhs;
+ }
+ return *this;
+ }
+
+ TResult MaxWith(const TResult& rhs) {
+ for (const auto& kv : rhs.data_) {
+ double& v = data_[kv.first];
+ if (v < kv.second) {
+ v = kv.second;
+ }
+ }
+ return *this;
+ }
+
+ struct DType {
+ uint8_t code : 8;
+ uint8_t bits : 8;
+ uint16_t lanes : 16;
+ };
+ static_assert(sizeof(DType) == 4, "Incorrect size of DType");
+
+ static String Int2Str(int32_t dtype) {
+ union {
+ DType dst;
+ int32_t src;
+ } converter;
+ converter.src = dtype;
+ static std::string type_code_tab[] = {"int", "uint", "float", "handle", "bfloat"};
+ std::ostringstream os;
+ os << type_code_tab[converter.dst.code];
+ os << static_cast<int>(converter.dst.bits);
+ if (converter.dst.lanes != 1) {
+ os << "x" << static_cast<int>(converter.dst.lanes);
+ }
+ return os.str();
+ }
+
+ static int32_t DataType2Int(const tvm::DataType& dtype) {
+ union {
+ DType src;
+ int32_t dst;
+ } converter;
+ converter.src.code = dtype.code();
+ converter.src.bits = dtype.bits();
+ converter.src.lanes = dtype.lanes();
+ return converter.dst;
+ }
+
+ TTable data_;
+ };
+
+ class FlopCounter : public ExprFunctor<TResult(const PrimExpr& n)>,
+ public StmtFunctor<TResult(const Stmt& n)> {
+ public:
+ ~FlopCounter() {}
+
+ TResult VisitExpr(const PrimExpr& expr) override { return ExprFunctor::VisitExpr(expr); }
+ TResult VisitStmt(const Stmt& stmt) override { return StmtFunctor::VisitStmt(stmt); }
+
+ TResult VisitStmt_(const IfThenElseNode* branch) override {
+ TResult cond = VisitExpr(branch->condition);
+ cond += VisitStmt(branch->then_case).MaxWith(VisitStmt(branch->else_case));
+ return cond;
+ }
+
+ TResult VisitStmt_(const BufferStoreNode* store) override {
+ TResult result = VisitExpr(store->value);
+ for (const PrimExpr& e : store->indices) {
+ result += VisitExpr(e);
+ }
+ return result;
+ }
+
+ TResult VisitStmt_(const SeqStmtNode* seq) override {
+ TResult result;
+ for (const Stmt& stmt : seq->seq) {
+ result += VisitStmt(stmt);
+ }
+ return result;
+ }
+
+ TResult VisitStmt_(const BlockRealizeNode* block) override {
+ return VisitStmt(block->block->body);
+ }
+
+ TResult VisitStmt_(const BlockNode* block) override {
+ TResult result;
+ if (block->init.defined()) {
+ result += VisitStmt(block->init.value());
+ }
+ result += VisitStmt(block->body);
+ return result;
+ }
+
+ TResult VisitStmt_(const ForNode* loop) override {
+ TResult result = VisitStmt(loop->body);
+ const auto* int_imm = loop->extent.as<IntImmNode>();
+ ICHECK(int_imm) << "TypeError: Expect the extent of a loop to be IntImm, but gets: "
+ << loop->extent->GetTypeKey();
+ result *= int_imm->value;
+ return result;
+ }
+
+#define TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(Node) \
+ TResult VisitExpr_(const Node* op) final { \
+ TResult result(op->dtype); \
+ result += VisitExpr(op->a); \
+ result += VisitExpr(op->b); \
+ return result; \
+ }
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(AddNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(SubNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(MulNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(DivNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(ModNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(FloorDivNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(FloorModNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(MinNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(MaxNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(EQNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(NENode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(LTNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(LENode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(GTNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(GENode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(AndNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(OrNode);
+#undef TVM_META_SCHEDULE_FLOP_COUNTER_BINARY
+ TResult VisitExpr_(const CastNode* op) override { return VisitExpr(op->value); }
+ TResult VisitExpr_(const VarNode* op) override { return TResult(); }
+ TResult VisitExpr_(const SizeVarNode* op) override { return TResult(); }
+ TResult VisitExpr_(const BufferLoadNode* op) override { return TResult(); }
+ TResult VisitExpr_(const IntImmNode* op) override { return TResult(); }
+ TResult VisitExpr_(const FloatImmNode* op) override { return TResult(); }
+ TResult VisitExpr_(const NotNode* op) override {
+ TResult result(op->dtype);
+ result += VisitExpr(op->a);
+ return result;
+ }
+ TResult VisitExpr_(const SelectNode* op) override {
+ TResult cond = VisitExpr(op->condition);
+ cond += VisitExpr(op->true_value).MaxWith(VisitExpr(op->false_value));
+ return cond;
+ }
+ TResult VisitExpr_(const CallNode* op) override {
+ TResult ret;
+ for (const auto& x : op->args) {
+ ret += VisitExpr(x);
+ }
+ return ret;
+ }
+ };
+ FlopCounter counter;
+ TResult result;
+ for (const auto& kv : mod->functions) {
+ const BaseFunc& base_func = kv.second;
+ if (const auto* prim_func = base_func.as<PrimFuncNode>()) {
+ result += counter.VisitStmt(prim_func->body);
+ }
+ }
+ double cnt = 0.0;
+ int i32 = TResult::DataType2Int(tvm::DataType::Int(32));
+ int i64 = TResult::DataType2Int(tvm::DataType::Int(64));
+ int u1 = TResult::DataType2Int(tvm::DataType::UInt(1));
+ for (const auto& kv : result.data_) {
+ if (kv.first != i32 && kv.first != i64 && kv.first != u1) {
Review comment:
Can we remove `kv.first != i32`? Right now when I tune int8 workloads, whose output is always int32, I get 0 GFLOPS like
```
Trial #0: GFLOPs: 0.0000. Time: 0.0576 ms. Best GFLOPs: 0.0000
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
masahi commented on pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#issuecomment-1076989028
Can we merge this? @zxybazh @junrushao1994
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r840998729
##########
File path: tests/python/unittest/test_meta_schedule_tune_tir.py
##########
@@ -98,14 +97,14 @@ def test_tune_matmul_cuda_tensor_core():
target = Target("nvidia/geforce-rtx-3070")
config = ReplayTraceConfig(
num_trials_per_iter=32,
- num_trials_total=320,
+ max_trials_per_task=320,
Review comment:
I think something weird is going on.
First, `max_trials_global` is not used at https://github.com/zxybazh/tvm/blob/e3fbb797a88308e4ce3d671939a83084ae1826b8/python/tvm/meta_schedule/search_strategy/replay_trace.py#L60
And if I use
```
config=ReplayTraceConfig(
num_trials_per_iter=32,
max_trials_per_task=32,
...
```
, `max_trials_per_task` acts like a global max trials, so only one task is tuned, as shown below. So to correctly tune all tasks, I need to set `max_trials_per_task = 32 * num_tasks`, which contradicts the name `max_trials_per_task ` and does something very different from the previous `num_trials_total`.
In other words, `num_trials_per_iter` is acting like `max_trials_per_task, and `max_trials_per_task` is acting like `max_trials_global`.
```
ID | Name | FLOP | Weight | Speed (GFLOPS) | Latency (us) | Weighted Latency (us) | Trials | Terminated
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 | fused_nn_conv2d_add | 12870144 | 1 | 51.3887 | 250.4468 | 250.4468 | 32 |
1 | fused_nn_conv2d_add_1 | 12895232 | 1 | N/A | N/A | N/A | 0 |
2 | fused_nn_conv2d_add_2 | 12945408 | 1 | N/A | N/A | N/A | 0 |
3 | fused_layout_transform | 1 | 1 | N/A | N/A | N/A | 0 |
4 | fused_nn_conv2d_add_nn_relu | 237633536 | 1 | N/A | N/A | N/A | 0 |
5 | fused_nn_max_pool2d | 1806336 | 1 | N/A | N/A | N/A | 0 |
6 | fused_nn_conv2d_add_nn_relu_1 | 231612416 | 2 | N/A | N/A | N/A | 0 |
7 | fused_nn_conv2d_add_add_nn_relu | 231813120 | 2 | N/A | N/A | N/A | 0 |
8 | fused_nn_conv2d_add_nn_relu_2 | 115806208 | 1 | N/A | N/A | N/A | 0 |
9 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_nn_relu | 93227008 | 1 | N/A | N/A | N/A | 0 |
10 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_add_nn_relu | 93327360 | 2 | N/A | N/A | N/A | 0 |
11 | fused_nn_conv2d_add_nn_relu_3 | 115705856 | 1 | N/A | N/A | N/A | 0 |
12 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_nn_relu_1 | 98600960 | 1 | N/A | N/A | N/A | 0 |
13 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_add_nn_relu_1 | 98651136 | 2 | N/A | N/A | N/A | 0 |
14 | fused_nn_conv2d_add_nn_relu_4 | 115655680 | 1 | N/A | N/A | N/A | 0 |
15 | fused_nn_conv2d_add_nn_relu_5 | 231261184 | 1 | N/A | N/A | N/A | 0 |
16 | fused_nn_conv2d_add_add_nn_relu_1 | 231286272 | 2 | N/A | N/A | N/A | 0 |
17 | fused_nn_adaptive_avg_pool2d | 25600 | 1 | N/A | N/A | N/A | 0 |
18 | fused_layout_transform_reshape_squeeze | 1 | 1 | N/A | N/A | N/A | 0 |
19 | fused_nn_dense_add | 1025000 | 1 | N/A | N/A | N/A | 0 |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
zxybazh commented on pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#issuecomment-1058375815
@junrushao1994 Would you please take another look and get it merged some time soon?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#issuecomment-1059846512
Will review next week!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
zxybazh commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r817282426
##########
File path: python/tvm/meta_schedule/task_scheduler/gradient_based.py
##########
@@ -0,0 +1,146 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Gradient Based Task Scheduler"""
+import math
+
+from typing import TYPE_CHECKING, List, Optional
+from tvm._ffi import register_object
+from tvm._ffi.registry import register_func
+
+from tvm.ir import IRModule
+from ..measure_callback import MeasureCallback
+from ..builder import Builder
+from ..runner import Runner
+from ..database import Database
+from ..cost_model import CostModel
+from .task_scheduler import TaskScheduler
+
+from .. import _ffi_api
+
+if TYPE_CHECKING:
+ from ..tune_context import TuneContext
+
+
+@register_func("meta_schedule.task_scheduler.derive_similarity_tag")
+def derive_similarity_tag(mod: IRModule, log_base: float = 1.618) -> str:
+ """Get the tags for smilarity group creation
+
+ Parameters
+ ---------
+ mod : IRModule
+ The input workload.
+ log_base : float
+ The log base to normalize the flop count.
Review comment:
Make sense. We allow the user to define the function and simply provide the function name so I think won't make a difference that we remove the log base.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r816456183
##########
File path: src/meta_schedule/task_scheduler/gradient_based.cc
##########
@@ -24,66 +24,80 @@ namespace meta_schedule {
/*! \brief The gradient based task scheduler. */
class GradientBasedNode final : public TaskSchedulerNode {
public:
- int backward_window_size;
- double alpha, beta;
+ int backward_window_size; // the backward windows size for backward gradient computation
+ double alpha, beta; // alpha, beta as in gradient computation
+ bool done_round_robin; // whether the round-robin warm up has been done
+
+ int task_id = -1; // the current task id processing
- bool done_round_robin; // whether the warm up round robin has been done
- int task_id = -1; // The current task id processed.
support::LinearCongruentialEngine::TRandState rand_state; // the random state
std::vector<int> task_cnts; // task tuning counts
std::vector<double> task_weights; // task weights
std::vector<double> task_best_latencies; // best latency achived by the task
std::vector<double> task_flop_counts; // total flop count of the task
std::vector<std::vector<double>> task_latency_history; // all task latency history
- std::vector<std::string> task_tag; // tag of the task for grouping
- std::map<std::string, int> tag_to_group; // map to find the group id given task tag
- std::vector<std::set<int>> task_groups; // the task ids in a given group
+ std::vector<int> task_group_id; // group id of the task
+ std::vector<std::set<int>> task_groups; // the task ids in a given group
TaskSchedulerNode::FObjectiveFunc objective_func; // the objective function
void VisitAttrs(tvm::AttrVisitor* v) {
TaskSchedulerNode::VisitAttrs(v);
+ v->Visit("backward_window_size", &backward_window_size);
+ v->Visit("alpha", &alpha);
+ v->Visit("beta", &beta);
v->Visit("task_id", &task_id);
}
static constexpr const char* _type_key = "meta_schedule.GradientBased";
TVM_DECLARE_FINAL_OBJECT_INFO(GradientBasedNode, TaskSchedulerNode);
protected:
+ /*!
+ * \brief Compute the objective function score for given workload latencies.
+ * \param latencies The current best latencies of each workload.
+ * \return The computed objective function score.
+ */
double _compute_score(const std::vector<double>& latencies) {
Array<FloatImm> input_latencies;
for (double latency : latencies)
input_latencies.push_back(FloatImm(DataType::Float(32), latency));
return objective_func(input_latencies)->value;
}
+ /*!
+ * \brief Adjuest the similarity group information for given task.
+ * \param task_id The id of the task to adjust similarity group.
+ */
void _adjust_similarity_group(int task_id) {
- int group_id = tag_to_group[task_tag[task_id]];
- if (task_groups[group_id].size() <= 1 ||
- task_groups[group_id].find(task_id) == task_groups[group_id].end())
- return;
+ int group_id = task_group_id[task_id];
+ if (task_groups[group_id].size() <= 1) return;
double best_flops = -1.0;
- int max_ct[3] = {-1, -1, -1}; // to find the 2nd largest
+ int max_cnt[3] = {-1, -1, -1}; // to find the 2nd largest
for (int i : task_groups[group_id]) {
best_flops = std::max(best_flops, task_flop_counts[i] / task_best_latencies[i]);
- max_ct[0] = task_cnts[i];
- std::sort(max_ct, max_ct + 3);
+ max_cnt[0] = task_cnts[i];
+ std::sort(max_cnt, max_cnt + 3); // place the 2nd largest to #1
}
double cur_flops = task_flop_counts[task_id] / task_best_latencies[task_id];
// if we tune a task for many times but it still cannot achieve
// a similar speed to the fastest one in its group, this means this task
// is actually not similar to other tasks in its group.
// So we will remove it from its original group.
Review comment:
Suggestion: So we will move it from the current group to a standalone group.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
comaniac commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r816455446
##########
File path: python/tvm/meta_schedule/task_scheduler/gradient_based.py
##########
@@ -0,0 +1,146 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Gradient Based Task Scheduler"""
+import math
+
+from typing import TYPE_CHECKING, List, Optional
+from tvm._ffi import register_object
+from tvm._ffi.registry import register_func
+
+from tvm.ir import IRModule
+from ..measure_callback import MeasureCallback
+from ..builder import Builder
+from ..runner import Runner
+from ..database import Database
+from ..cost_model import CostModel
+from .task_scheduler import TaskScheduler
+
+from .. import _ffi_api
+
+if TYPE_CHECKING:
+ from ..tune_context import TuneContext
+
+
+@register_func("meta_schedule.task_scheduler.derive_similarity_tag")
+def derive_similarity_tag(mod: IRModule, log_base: float = 1.618) -> str:
+ """Get the tags for smilarity group creation
+
+ Parameters
+ ---------
+ mod : IRModule
+ The input workload.
+ log_base : float
+ The log base to normalize the flop count.
Review comment:
I see. Then it might be worthwhile to consider not exposing this option to users if we don't expect any benefit of other bases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
zxybazh commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r816409784
##########
File path: src/meta_schedule/task_scheduler/gradient_based.cc
##########
@@ -0,0 +1,295 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include "../utils.h"
+
+namespace tvm {
+namespace meta_schedule {
+
+/*! \brief The gradient based task scheduler. */
+class GradientBasedNode final : public TaskSchedulerNode {
+ public:
+ int backward_window_size;
+ double alpha, beta;
+
+ bool done_round_robin; // whether the warm up round robin has been done
+ int task_id = -1; // The current task id processed.
+ support::LinearCongruentialEngine::TRandState rand_state; // the random state
+ std::vector<int> task_cnts; // task tuning counts
+ std::vector<double> task_weights; // task weights
+ std::vector<double> task_best_latencies; // best latency achived by the task
+ std::vector<double> task_flop_counts; // total flop count of the task
+ std::vector<std::vector<double>> task_latency_history; // all task latency history
+
+ std::vector<std::string> task_tag; // tag of the task for grouping
+ std::map<std::string, int> tag_to_group; // map to find the group id given task tag
+ std::vector<std::set<int>> task_groups; // the task ids in a given group
+
+ TaskSchedulerNode::FObjectiveFunc objective_func; // the objective function
+
+ void VisitAttrs(tvm::AttrVisitor* v) {
+ TaskSchedulerNode::VisitAttrs(v);
+ v->Visit("task_id", &task_id);
+ }
+
+ static constexpr const char* _type_key = "meta_schedule.GradientBased";
+ TVM_DECLARE_FINAL_OBJECT_INFO(GradientBasedNode, TaskSchedulerNode);
+
+ protected:
+ double _compute_score(const std::vector<double>& latencies) {
+ Array<FloatImm> input_latencies;
+ for (double latency : latencies)
+ input_latencies.push_back(FloatImm(DataType::Float(32), latency));
+ return objective_func(input_latencies)->value;
+ }
+
+ void _adjust_similarity_group(int task_id) {
+ int group_id = tag_to_group[task_tag[task_id]];
+ if (task_groups[group_id].size() <= 1 ||
+ task_groups[group_id].find(task_id) == task_groups[group_id].end())
+ return;
+
+ double best_flops = -1.0;
+ int max_ct[3] = {-1, -1, -1}; // to find the 2nd largest
+ for (int i : task_groups[group_id]) {
+ best_flops = std::max(best_flops, task_flop_counts[i] / task_best_latencies[i]);
+ max_ct[0] = task_cnts[i];
+ std::sort(max_ct, max_ct + 3);
+ }
+ double cur_flops = task_flop_counts[task_id] / task_best_latencies[task_id];
+ // if we tune a task for many times but it still cannot achieve
+ // a similar speed to the fastest one in its group, this means this task
+ // is actually not similar to other tasks in its group.
+ // So we will remove it from its original group.
+ if (cur_flops < best_flops / beta && task_cnts[task_id] > 5 + max_ct[1]) {
+ task_groups[group_id].erase(task_id);
+ }
+ }
+
+ int NextTaskId() final {
+ int n_tasks = this->tasks.size();
+ // Check if warmed up with round robin
+ if (!done_round_robin) {
+ task_id++;
+ if (task_id == n_tasks) {
+ done_round_robin = true;
+ } else {
+ return task_id;
+ }
+ }
Review comment:
We have to do it differently because we are asyncly sending measure candidates to runner and fetching the results. There may not be count of tuning results when we are looking for the next task id to tune.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r841025653
##########
File path: tests/python/unittest/test_meta_schedule_tune_tir.py
##########
@@ -98,14 +97,14 @@ def test_tune_matmul_cuda_tensor_core():
target = Target("nvidia/geforce-rtx-3070")
config = ReplayTraceConfig(
num_trials_per_iter=32,
- num_trials_total=320,
+ max_trials_per_task=320,
Review comment:
Hey @masahi, thanks for detecting this issue swiftly! I apologize for the inconvenience as the end API is still in flux.
Yes, this PR breaks the existing unstable MetaSchedule API, and we want to do this from very early on, so that the APIs won't be broken again after formally announcing MetaSchedule is ready for production use.
The intention of having the parameter `config` is to provide an all-in-one place for end users to better configure the search - even if it's less realistic given builder/runner/database/cost_model/task_scheduler actually all need to be configured separately, it's intended to be the most frequent parameter users need to tweak.
Then let's discuss about `max_trials_per_task` and `max_trials_global`. Imagine a service that allows users to set how many a global uplimit of total number of trials for an entire model, as well as a per-task limit for each individual task extracted, then these are the two parameters to tweak. This PR adds this feature essentially for completeness of such potential SaaS feature request.
I have to admit that this breaking change to unannounced APIs could be damaging particularly to early users like you for sudden surprises; On the other hand, we do want to work together, do the correct thing to make those APIs look right to the end users. In the near future, we will likely introduce breaking changes including:
- Based on your work, refactoring `integration` into `extract_task` and `apply_history_best`
- More structured and readable logging
- Less error-prone interface in `tune.py` (needs to collect more feedbacks)
We will do our best to ping early users like you to make sure people are aware of our breaking change before officially announcing MetaSchedule is product-ready. After being product ready, we will be less aggressive in terms of refactoring. Thank you so much for your understanding!
> First, max_trials_global is not used at https://github.com/zxybazh/tvm/blob/e3fbb797a88308e4ce3d671939a83084ae1826b8/python/tvm/meta_schedule/search_strategy/replay_trace.py#L60
Yes, and this parameter is only used for configuring the task scheduler.
> And if I use
> ```
> config=ReplayTraceConfig(
> num_trials_per_iter=32,
> max_trials_per_task=32,
> ...
> ```
> , `max_trials_per_task` acts like a global max trials, so only one task is tuned, as shown below. This contradicts the name `max_trials_per_task ` and does something very different from the previous `num_trials_total`.
Indeed this behavior doesn't make any sense. May I ask what your `max_trials_global` is in this case? It could lead to early stopping if it's less than `32 * num_tasks`.
> All uses of ReplayTraceConfig in tests are broken since now it has three attributes.
We updated the usage of this API in the following files:
- python/tvm/meta_schedule/testing/tune_te_meta_schedule.py
- python/tvm/meta_schedule/testing/tune_relay_meta_schedule.py
- tests/python/unittest/test_meta_schedule_measure_callback.py
- tests/python/unittest/test_meta_schedule_search_strategy.py
- tests/python/unittest/test_meta_schedule_task_scheduler.py
- tests/python/unittest/test_meta_schedule_tune_relay.py
- tests/python/unittest/test_meta_schedule_tune_te.py
- tests/python/unittest/test_meta_schedule_tune_tir.py
And indeed we missed the following two files (essentially skipped ones):
- tests/python/unittest/test_meta_schedule_tune_tir.py
- tests/python/unittest/test_meta_schedule_tune_te.py
- tests/python/unittest/test_meta_schedule_tune_relay.py
Clearly we forgot to do so for the skipped files, and I will shoot a PR to get them fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r841031450
##########
File path: tests/python/unittest/test_meta_schedule_tune_tir.py
##########
@@ -98,14 +97,14 @@ def test_tune_matmul_cuda_tensor_core():
target = Target("nvidia/geforce-rtx-3070")
config = ReplayTraceConfig(
num_trials_per_iter=32,
- num_trials_total=320,
+ max_trials_per_task=320,
Review comment:
> I understand the risk of being an early user, but I'd like to see at least reasonable documentation of the API at all times, regardless of being stable / unstable. Especially when you introduce a breaking change. In this particular case, the new attribute `max_trials_global` is not documented and there is no example of how to use this value, as far as I can see.
That's a reasonable ask. I'm drafting a bugfix PR which will in the mean time make sure the document is clear. I'm sorry that it could potentially miss some corner cases during our death march last week on performance alignment in BERT.
A further step, I would love to have a user-facing API audit with you and the rest of the community, so that we could build broader consensus of what things should look like.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
zxybazh commented on pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#issuecomment-1050148432
CC @junrushao1994 @comaniac
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
zxybazh commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r816410567
##########
File path: python/tvm/meta_schedule/task_scheduler/gradient_based.py
##########
@@ -0,0 +1,146 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Gradient Based Task Scheduler"""
+import math
+
+from typing import TYPE_CHECKING, List, Optional
+from tvm._ffi import register_object
+from tvm._ffi.registry import register_func
+
+from tvm.ir import IRModule
+from ..measure_callback import MeasureCallback
+from ..builder import Builder
+from ..runner import Runner
+from ..database import Database
+from ..cost_model import CostModel
+from .task_scheduler import TaskScheduler
+
+from .. import _ffi_api
+
+if TYPE_CHECKING:
+ from ..tune_context import TuneContext
+
+
+@register_func("meta_schedule.task_scheduler.derive_similarity_tag")
+def derive_similarity_tag(mod: IRModule, log_base: float = 1.618) -> str:
+ """Get the tags for smilarity group creation
+
+ Parameters
+ ---------
+ mod : IRModule
+ The input workload.
+ log_base : float
+ The log base to normalize the flop count.
Review comment:
Basically it won't matter much, the usage of base is only for workload grouping and then optimal flop estimation. This is same as the practice in autoscheduler.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
zxybazh commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r826203835
##########
File path: src/meta_schedule/utils.h
##########
@@ -50,6 +51,201 @@
#include "../tir/schedule/utils.h"
namespace tvm {
+namespace tir {
+
+inline double CountFlop(const IRModule& mod) {
+ struct TResult {
+ using TTable = std::unordered_map<int32_t, double>;
+
+ TResult() = default;
+
+ explicit TResult(const tvm::DataType& dtype) { Add(dtype); }
+
+ void Add(const tvm::DataType& dtype) { data_[DataType2Int(dtype)] += 1; }
+
+ TResult operator+=(const TResult& rhs) {
+ for (const auto& kv : rhs.data_) {
+ data_[kv.first] += kv.second;
+ }
+ return *this;
+ }
+
+ TResult operator*=(int64_t rhs) {
+ for (auto& kv : data_) {
+ kv.second *= rhs;
+ }
+ return *this;
+ }
+
+ TResult MaxWith(const TResult& rhs) {
+ for (const auto& kv : rhs.data_) {
+ double& v = data_[kv.first];
+ if (v < kv.second) {
+ v = kv.second;
+ }
+ }
+ return *this;
+ }
+
+ struct DType {
+ uint8_t code : 8;
+ uint8_t bits : 8;
+ uint16_t lanes : 16;
+ };
+ static_assert(sizeof(DType) == 4, "Incorrect size of DType");
+
+ static String Int2Str(int32_t dtype) {
+ union {
+ DType dst;
+ int32_t src;
+ } converter;
+ converter.src = dtype;
+ static std::string type_code_tab[] = {"int", "uint", "float", "handle", "bfloat"};
+ std::ostringstream os;
+ os << type_code_tab[converter.dst.code];
+ os << static_cast<int>(converter.dst.bits);
+ if (converter.dst.lanes != 1) {
+ os << "x" << static_cast<int>(converter.dst.lanes);
+ }
+ return os.str();
+ }
+
+ static int32_t DataType2Int(const tvm::DataType& dtype) {
+ union {
+ DType src;
+ int32_t dst;
+ } converter;
+ converter.src.code = dtype.code();
+ converter.src.bits = dtype.bits();
+ converter.src.lanes = dtype.lanes();
+ return converter.dst;
+ }
+
+ TTable data_;
+ };
+
+ class FlopCounter : public ExprFunctor<TResult(const PrimExpr& n)>,
+ public StmtFunctor<TResult(const Stmt& n)> {
+ public:
+ ~FlopCounter() {}
+
+ TResult VisitExpr(const PrimExpr& expr) override { return ExprFunctor::VisitExpr(expr); }
+ TResult VisitStmt(const Stmt& stmt) override { return StmtFunctor::VisitStmt(stmt); }
+
+ TResult VisitStmt_(const IfThenElseNode* branch) override {
+ TResult cond = VisitExpr(branch->condition);
+ cond += VisitStmt(branch->then_case).MaxWith(VisitStmt(branch->else_case));
+ return cond;
+ }
+
+ TResult VisitStmt_(const BufferStoreNode* store) override {
+ TResult result = VisitExpr(store->value);
+ for (const PrimExpr& e : store->indices) {
+ result += VisitExpr(e);
+ }
+ return result;
+ }
+
+ TResult VisitStmt_(const SeqStmtNode* seq) override {
+ TResult result;
+ for (const Stmt& stmt : seq->seq) {
+ result += VisitStmt(stmt);
+ }
+ return result;
+ }
+
+ TResult VisitStmt_(const BlockRealizeNode* block) override {
+ return VisitStmt(block->block->body);
+ }
+
+ TResult VisitStmt_(const BlockNode* block) override {
+ TResult result;
+ if (block->init.defined()) {
+ result += VisitStmt(block->init.value());
+ }
+ result += VisitStmt(block->body);
+ return result;
+ }
+
+ TResult VisitStmt_(const ForNode* loop) override {
+ TResult result = VisitStmt(loop->body);
+ const auto* int_imm = loop->extent.as<IntImmNode>();
+ ICHECK(int_imm) << "TypeError: Expect the extent of a loop to be IntImm, but gets: "
+ << loop->extent->GetTypeKey();
+ result *= int_imm->value;
+ return result;
+ }
+
+#define TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(Node) \
+ TResult VisitExpr_(const Node* op) final { \
+ TResult result(op->dtype); \
+ result += VisitExpr(op->a); \
+ result += VisitExpr(op->b); \
+ return result; \
+ }
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(AddNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(SubNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(MulNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(DivNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(ModNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(FloorDivNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(FloorModNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(MinNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(MaxNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(EQNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(NENode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(LTNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(LENode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(GTNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(GENode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(AndNode);
+ TVM_META_SCHEDULE_FLOP_COUNTER_BINARY(OrNode);
+#undef TVM_META_SCHEDULE_FLOP_COUNTER_BINARY
+ TResult VisitExpr_(const CastNode* op) override { return VisitExpr(op->value); }
+ TResult VisitExpr_(const VarNode* op) override { return TResult(); }
+ TResult VisitExpr_(const SizeVarNode* op) override { return TResult(); }
+ TResult VisitExpr_(const BufferLoadNode* op) override { return TResult(); }
+ TResult VisitExpr_(const IntImmNode* op) override { return TResult(); }
+ TResult VisitExpr_(const FloatImmNode* op) override { return TResult(); }
+ TResult VisitExpr_(const NotNode* op) override {
+ TResult result(op->dtype);
+ result += VisitExpr(op->a);
+ return result;
+ }
+ TResult VisitExpr_(const SelectNode* op) override {
+ TResult cond = VisitExpr(op->condition);
+ cond += VisitExpr(op->true_value).MaxWith(VisitExpr(op->false_value));
+ return cond;
+ }
+ TResult VisitExpr_(const CallNode* op) override {
+ TResult ret;
+ for (const auto& x : op->args) {
+ ret += VisitExpr(x);
+ }
+ return ret;
+ }
+ };
+ FlopCounter counter;
+ TResult result;
+ for (const auto& kv : mod->functions) {
+ const BaseFunc& base_func = kv.second;
+ if (const auto* prim_func = base_func.as<PrimFuncNode>()) {
+ result += counter.VisitStmt(prim_func->body);
+ }
+ }
+ double cnt = 0.0;
+ int i32 = TResult::DataType2Int(tvm::DataType::Int(32));
+ int i64 = TResult::DataType2Int(tvm::DataType::Int(64));
+ int u1 = TResult::DataType2Int(tvm::DataType::UInt(1));
+ for (const auto& kv : result.data_) {
+ if (kv.first != i32 && kv.first != i64 && kv.first != u1) {
Review comment:
CC @junrushao1994
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#issuecomment-1076989948
@masahi thanks for asking! We realized that there is an important problem not addressed yet in this PR (task weight prioritization). With other tasks of higher priority, we would love to delay the PR to next month.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #10366: [MetaSchedule] Add Gradient Based Task Scheduler
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #10366:
URL: https://github.com/apache/tvm/pull/10366#discussion_r840998729
##########
File path: tests/python/unittest/test_meta_schedule_tune_tir.py
##########
@@ -98,14 +97,14 @@ def test_tune_matmul_cuda_tensor_core():
target = Target("nvidia/geforce-rtx-3070")
config = ReplayTraceConfig(
num_trials_per_iter=32,
- num_trials_total=320,
+ max_trials_per_task=320,
Review comment:
I think something weird is going on.
First, `max_trials_global` is not used at https://github.com/zxybazh/tvm/blob/e3fbb797a88308e4ce3d671939a83084ae1826b8/python/tvm/meta_schedule/search_strategy/replay_trace.py#L60
And if I use
```
config=ReplayTraceConfig(
num_trials_per_iter=32,
max_trials_per_task=32,
...
```
, `max_trials_per_task` acts like a global max trials, so only one task is tuned, as shown below. So to correctly tune all tasks, I need to set `max_trials_per_task = 32 * num_tasks`, which contradicts the name `max_trials_per_task ` and does something very different from the previous `num_trials_total`.
```
ID | Name | FLOP | Weight | Speed (GFLOPS) | Latency (us) | Weighted Latency (us) | Trials | Terminated
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 | fused_nn_conv2d_add | 12870144 | 1 | 51.3887 | 250.4468 | 250.4468 | 32 |
1 | fused_nn_conv2d_add_1 | 12895232 | 1 | N/A | N/A | N/A | 0 |
2 | fused_nn_conv2d_add_2 | 12945408 | 1 | N/A | N/A | N/A | 0 |
3 | fused_layout_transform | 1 | 1 | N/A | N/A | N/A | 0 |
4 | fused_nn_conv2d_add_nn_relu | 237633536 | 1 | N/A | N/A | N/A | 0 |
5 | fused_nn_max_pool2d | 1806336 | 1 | N/A | N/A | N/A | 0 |
6 | fused_nn_conv2d_add_nn_relu_1 | 231612416 | 2 | N/A | N/A | N/A | 0 |
7 | fused_nn_conv2d_add_add_nn_relu | 231813120 | 2 | N/A | N/A | N/A | 0 |
8 | fused_nn_conv2d_add_nn_relu_2 | 115806208 | 1 | N/A | N/A | N/A | 0 |
9 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_nn_relu | 93227008 | 1 | N/A | N/A | N/A | 0 |
10 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_add_nn_relu | 93327360 | 2 | N/A | N/A | N/A | 0 |
11 | fused_nn_conv2d_add_nn_relu_3 | 115705856 | 1 | N/A | N/A | N/A | 0 |
12 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_nn_relu_1 | 98600960 | 1 | N/A | N/A | N/A | 0 |
13 | fused_nn_contrib_conv2d_winograd_without_weight_transform_add_add_nn_relu_1 | 98651136 | 2 | N/A | N/A | N/A | 0 |
14 | fused_nn_conv2d_add_nn_relu_4 | 115655680 | 1 | N/A | N/A | N/A | 0 |
15 | fused_nn_conv2d_add_nn_relu_5 | 231261184 | 1 | N/A | N/A | N/A | 0 |
16 | fused_nn_conv2d_add_add_nn_relu_1 | 231286272 | 2 | N/A | N/A | N/A | 0 |
17 | fused_nn_adaptive_avg_pool2d | 25600 | 1 | N/A | N/A | N/A | 0 |
18 | fused_layout_transform_reshape_squeeze | 1 | 1 | N/A | N/A | N/A | 0 |
19 | fused_nn_dense_add | 1025000 | 1 | N/A | N/A | N/A | 0 |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org