[GitHub] [incubator-doris] weizuo93 commented on a change in pull request #4670: [Optimize] Optimize the execution model of compaction to limit memory consumption

[GitBox <gi...@apache.org>]

weizuo93 commented on a change in pull request #4670:
URL: https://github.com/apache/incubator-doris/pull/4670#discussion_r495658322



##########
File path: be/src/common/config.h
##########
@@ -310,13 +306,25 @@ namespace config {
     // if compaction of a tablet failed, this tablet should not be chosen to
     // compaction until this interval passes.
     CONF_mInt64(min_compaction_failure_interval_sec, "600"); // 10 min
-    // Too many compaction tasks may run out of memory.
-    // This config is to limit the max concurrency of running compaction tasks.
-    // -1 means no limit, and the max concurrency will be:
-    //      C = (cumulative_compaction_num_threads_per_disk + base_compaction_num_threads_per_disk) * dir_num
-    // set it to larger than C will be set to equal to C.
-    // This config can be set to 0, which means to forbid any compaction, for some special cases.
-    CONF_Int32(max_compaction_concurrency, "-1");
+
+    // This config can be set to limit thread number in compaction thread pool.
+    CONF_mInt32(min_compaction_threads, "10");
+    CONF_mInt32(max_compaction_threads, "10");
+
+    // The upper limit of "permits" held by all compaction tasks. This config can be set to limit memory consumption for compaction.
+    CONF_mInt64(total_permits_for_compaction_score, "10000")
+
+    // Whether compaction task is allowed to start when compaction score of current tablet is out of upper limit.
+    CONF_mBool(enable_over_sold, "true");

Review comment:
       OK !




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org