You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2022/08/08 09:07:39 UTC

[GitHub] [iotdb] THUMarkLau opened a new pull request, #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

THUMarkLau opened a new pull request, #6914:
URL: https://github.com/apache/iotdb/pull/6914

   See [IOTDB-3164](https://issues.apache.org/jira/browse/IOTDB-3164).
   
   This PR is one part of the compaction memory management. It does the following jobs:
   - Allocate memory before cross space compaction, and release the memory after the execution.
   - Remove the `RewriteCompactionFileSelector`, which is deduplicated with `RewriteCrossSpaceCompactionSelector`.
   
   To complete the compaction memory management, we need to complete the following jobs:
   - Manage the memory usage of unsequence inner space compaction, which uses read point performer.
   - Design and implement the memory estimator for read chunk performer.
   - Manage the memory usage of sequence inner space comapction, which uses read chunk performer.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] qiaojialin commented on a diff in pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
qiaojialin commented on code in PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#discussion_r954909864


##########
server/src/main/java/org/apache/iotdb/db/engine/compaction/cross/rewrite/RewriteCrossSpaceCompactionSelector.java:
##########
@@ -56,6 +75,223 @@ public RewriteCrossSpaceCompactionSelector(
     this.dataRegionId = dataRegionId;
     this.timePartition = timePartition;
     this.tsFileManager = tsFileManager;
+    this.memoryBudget = config.getCrossCompactionMemoryBudget();

Review Comment:
   apply this memory from the storage engine, not from the whole system



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] sonarcloud[bot] commented on pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#issuecomment-1229180324

   SonarCloud Quality Gate failed.&nbsp; &nbsp; [![Quality Gate failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/failed-16px.png 'Quality Gate failed')](https://sonarcloud.io/dashboard?id=apache_incubator-iotdb&pullRequest=6914)
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=BUG) [![D](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/D-16px.png 'D')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=BUG) [2 Bugs](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=CODE_SMELL) [11 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=CODE_SMELL)
   
   [![89.5%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/60-16px.png '89.5%')](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=6914&metric=new_coverage&view=list) [89.5% Coverage](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=6914&metric=new_coverage&view=list)  
   [![0.0%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.0%')](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=6914&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=6914&metric=new_duplicated_lines_density&view=list)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] coveralls commented on pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
coveralls commented on PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#issuecomment-1208037956

   
   [![Coverage Status](https://coveralls.io/builds/51488729/badge)](https://coveralls.io/builds/51488729)
   
   Coverage increased (+0.002%) to 42.623% when pulling **b822582432414a9d8f35c87ecc9b63d1a6fba96d on IOTDB-3164-O** into **39db994138c878b9e4f055d4c62e56f9ceb86693 on master**.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] qiaojialin commented on a diff in pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
qiaojialin commented on code in PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#discussion_r955756137


##########
server/src/main/java/org/apache/iotdb/db/conf/IoTDBConfig.java:
##########
@@ -623,7 +699,7 @@ public class IoTDBConfig {
   private TSEncoding defaultTextEncoding = TSEncoding.PLAIN;
 
   /** How much memory (in byte) can be used by a single merge task. */
-  private long crossCompactionMemoryBudget = (long) (Runtime.getRuntime().maxMemory() * 0.1);
+  private long crossCompactionMemoryBudget = (long) (allocateMemoryForWrite * 0.1D);

Review Comment:
   The default variable is 
   
   private double compactionProportion = 0.2;
   
   ```suggestion
     private long crossCompactionMemoryBudget = (long) (allocateMemoryForWrite * 0.2D);
   ```



##########
server/src/main/java/org/apache/iotdb/db/rescon/SystemInfo.java:
##########
@@ -43,10 +44,15 @@ public class SystemInfo {
   private long totalStorageGroupMemCost = 0L;
   private volatile boolean rejected = false;
 
-  private static long memorySizeForWrite = config.getAllocateMemoryForWrite();
+  private static long memorySizeForWrite =
+      (long) (config.getAllocateMemoryForWrite() * config.getMemtableProportion());

Review Comment:
   ```suggestion
         (long) (config.getAllocateMemoryForStorageEngine() * config.getWriteProportion());
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] THUMarkLau commented on pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
THUMarkLau commented on PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#issuecomment-1210194987

   I use IoTDB benchmark tests changed after the performance, and the master (commit: a8f757d6e4fecc2251d2042a025960bf29c6e0f3) are compared. The result of the comparison is shown in the figure below. In terms of average performance, the write performance after the change decreases by about 2.8%.
   
   **master branch**
   ![e173a42189cf91925f9f73475afac2b](https://user-images.githubusercontent.com/37140360/183825856-ee4e5850-a190-4dfd-848f-8c693f94af15.png)
   
   **with memory management for cross space compaction**
   ![dd9c26571d2659edfb92e40e6d87df9](https://user-images.githubusercontent.com/37140360/183825884-dda2ae80-4cd4-4eb0-b0df-3bfe5abe0e07.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] THUMarkLau commented on a diff in pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
THUMarkLau commented on code in PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#discussion_r956863233


##########
server/src/main/java/org/apache/iotdb/db/rescon/SystemInfo.java:
##########
@@ -170,6 +176,27 @@ public synchronized void resetFlushingMemTableCost(long flushingMemTableCost) {
     this.flushingMemTablesCost -= flushingMemTableCost;
   }
 
+  public void addCompactionMemoryCost(long memoryCost) throws InterruptedException {
+    long originSize = this.compactionMemoryCost.get();
+    while (originSize + memoryCost > memorySizeForCompaction

Review Comment:
   The originSize is updated in the while block.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] qiaojialin commented on a diff in pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
qiaojialin commented on code in PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#discussion_r956731057


##########
server/src/main/java/org/apache/iotdb/db/rescon/SystemInfo.java:
##########
@@ -170,6 +176,27 @@ public synchronized void resetFlushingMemTableCost(long flushingMemTableCost) {
     this.flushingMemTablesCost -= flushingMemTableCost;
   }
 
+  public void addCompactionMemoryCost(long memoryCost) throws InterruptedException {
+    long originSize = this.compactionMemoryCost.get();
+    while (originSize + memoryCost > memorySizeForCompaction

Review Comment:
   If there are two tasks call this method concurrently, 
   Task1 originSize = 10;
   Task2 originSize = 10;
   Task1.compactionMemoryCost.compacreAndSet(10, 10+30) = 40;
   Task2 compareAndSet is always false, and Task2 sleeps forever.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] THUMarkLau commented on a diff in pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
THUMarkLau commented on code in PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#discussion_r956716045


##########
server/src/main/java/org/apache/iotdb/db/conf/IoTDBConfig.java:
##########
@@ -623,7 +629,7 @@ public class IoTDBConfig {
   private TSEncoding defaultTextEncoding = TSEncoding.PLAIN;
 
   /** How much memory (in byte) can be used by a single merge task. */
-  private long crossCompactionMemoryBudget = (long) (Runtime.getRuntime().maxMemory() * 0.1);
+  private long crossCompactionMemoryBudget = (long) (allocateMemoryForWrite * 0.1D);

Review Comment:
   This param is removed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] qiaojialin commented on a diff in pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
qiaojialin commented on code in PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#discussion_r955919719


##########
server/src/main/java/org/apache/iotdb/db/conf/IoTDBConfig.java:
##########
@@ -623,7 +629,7 @@ public class IoTDBConfig {
   private TSEncoding defaultTextEncoding = TSEncoding.PLAIN;
 
   /** How much memory (in byte) can be used by a single merge task. */
-  private long crossCompactionMemoryBudget = (long) (Runtime.getRuntime().maxMemory() * 0.1);
+  private long crossCompactionMemoryBudget = (long) (allocateMemoryForWrite * 0.1D);

Review Comment:
   ```suggestion
     private long compactionMemoryBudget = (long) (allocateMemoryForStorageEngine * compactionProportion);
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] THUMarkLau commented on a diff in pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
THUMarkLau commented on code in PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#discussion_r956716231


##########
server/src/main/java/org/apache/iotdb/db/engine/compaction/cross/CrossSpaceCompactionTask.java:
##########
@@ -79,11 +83,14 @@ public CrossSpaceCompactionTask(
     this.unseqTsFileResourceList = tsFileManager.getUnsequenceListByTimePartition(timePartition);
     this.performer = performer;
     this.hashCode = this.toString().hashCode();
+    this.memoryCost = memoryCost;
   }
 
   @Override
   protected void doCompaction() {
     try {
+      SystemInfo.getInstance().addCompactionMemoryCost(memoryCost);

Review Comment:
   I move allocateMemory in a seperate try-catch block.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] sonarcloud[bot] commented on pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#issuecomment-1229180324

   SonarCloud Quality Gate failed.&nbsp; &nbsp; [![Quality Gate failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/failed-16px.png 'Quality Gate failed')](https://sonarcloud.io/dashboard?id=apache_incubator-iotdb&pullRequest=6914)
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=BUG) [![D](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/D-16px.png 'D')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=BUG) [2 Bugs](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=CODE_SMELL) [11 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=CODE_SMELL)
   
   [![89.5%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/60-16px.png '89.5%')](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=6914&metric=new_coverage&view=list) [89.5% Coverage](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=6914&metric=new_coverage&view=list)  
   [![0.0%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.0%')](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=6914&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=6914&metric=new_duplicated_lines_density&view=list)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] sonarcloud[bot] commented on pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#issuecomment-1208042547

   SonarCloud Quality Gate failed.&nbsp; &nbsp; [![Quality Gate failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/failed-16px.png 'Quality Gate failed')](https://sonarcloud.io/dashboard?id=apache_incubator-iotdb&pullRequest=6914)
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=BUG) [![D](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/D-16px.png 'D')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=BUG) [1 Bug](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=CODE_SMELL) [10 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=6914&resolved=false&types=CODE_SMELL)
   
   [![91.7%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/90-16px.png '91.7%')](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=6914&metric=new_coverage&view=list) [91.7% Coverage](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=6914&metric=new_coverage&view=list)  
   [![0.0%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.0%')](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=6914&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=6914&metric=new_duplicated_lines_density&view=list)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] qiaojialin merged pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
qiaojialin merged PR #6914:
URL: https://github.com/apache/iotdb/pull/6914


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] qiaojialin commented on a diff in pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
qiaojialin commented on code in PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#discussion_r956552179


##########
server/src/main/java/org/apache/iotdb/db/engine/compaction/cross/CrossSpaceCompactionTask.java:
##########
@@ -79,11 +83,14 @@ public CrossSpaceCompactionTask(
     this.unseqTsFileResourceList = tsFileManager.getUnsequenceListByTimePartition(timePartition);
     this.performer = performer;
     this.hashCode = this.toString().hashCode();
+    this.memoryCost = memoryCost;
   }
 
   @Override
   protected void doCompaction() {
     try {
+      SystemInfo.getInstance().addCompactionMemoryCost(memoryCost);

Review Comment:
   what if the allocateMemory throw interrupt exception without succeeding in adding?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] qiaojialin commented on pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
qiaojialin commented on PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#issuecomment-1228621844

   https://apache-iotdb.feishu.cn/docx/doxcnuh5hy6ee9Qb7uLCYXkFDdc


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [iotdb] THUMarkLau commented on a diff in pull request #6914: [IOTDB-3164] Manage the memory of cross space compaction in write memory controller

Posted by GitBox <gi...@apache.org>.
THUMarkLau commented on code in PR #6914:
URL: https://github.com/apache/iotdb/pull/6914#discussion_r955728005


##########
server/src/main/java/org/apache/iotdb/db/engine/compaction/cross/rewrite/RewriteCrossSpaceCompactionSelector.java:
##########
@@ -56,6 +75,223 @@ public RewriteCrossSpaceCompactionSelector(
     this.dataRegionId = dataRegionId;
     this.timePartition = timePartition;
     this.tsFileManager = tsFileManager;
+    this.memoryBudget = config.getCrossCompactionMemoryBudget();

Review Comment:
   reslove



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org