You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2021/03/23 12:32:58 UTC
[GitHub] [iotdb] neuyilan commented on a change in pull request #2879: IOTDB-854 Limit the memory foorprint of the committed log cache
neuyilan commented on a change in pull request #2879:
URL: https://github.com/apache/iotdb/pull/2879#discussion_r599520609
##########
File path: cluster/src/main/java/org/apache/iotdb/cluster/log/manage/RaftLogManager.java
##########
@@ -582,14 +585,30 @@ public void commitTo(long newCommitIndex) throws LogExecutionException {
.clear();
}
try {
- int currentSize = committedEntryManager.getTotalSize();
- int deltaSize = entries.size();
- if (currentSize + deltaSize > maxNumOfLogsInMem) {
- int sizeToReserveForNew = maxNumOfLogsInMem - deltaSize;
+ boolean needToCompactLog = false;
+ int numToReserveForNew = minNumOfLogsInMem;
+ if (committedEntryManager.getTotalSize() + entries.size() > maxNumOfLogsInMem) {
+ needToCompactLog = true;
+ numToReserveForNew = maxNumOfLogsInMem - entries.size();
+ }
+
Review comment:
if `entries.size()=2,minNumOfLogsInMem=100,maxNumOfLogsInMem=200`, `committedEntryManager.getTotalSize()=199`, then the calculated `numToReserveForNew =200-2=198?` I think it should be `min(198, 100) = 100`
##########
File path: cluster/src/main/java/org/apache/iotdb/cluster/log/manage/RaftLogManager.java
##########
@@ -582,14 +585,30 @@ public void commitTo(long newCommitIndex) throws LogExecutionException {
.clear();
}
try {
- int currentSize = committedEntryManager.getTotalSize();
- int deltaSize = entries.size();
- if (currentSize + deltaSize > maxNumOfLogsInMem) {
- int sizeToReserveForNew = maxNumOfLogsInMem - deltaSize;
+ boolean needToCompactLog = false;
+ int numToReserveForNew = minNumOfLogsInMem;
+ if (committedEntryManager.getTotalSize() + entries.size() > maxNumOfLogsInMem) {
+ needToCompactLog = true;
+ numToReserveForNew = maxNumOfLogsInMem - entries.size();
+ }
+
+ long newEntryMemSize = 0;
+ for (Log entry : entries) {
+ newEntryMemSize += entry.getByteSize();
+ }
+ int sizeToReserveForNew = minNumOfLogsInMem;
+ if (newEntryMemSize + committedEntryManager.getEntryTotalMemSize() > maxLogMemSize) {
+ needToCompactLog = true;
+ sizeToReserveForNew =
+ committedEntryManager.maxLogNumShouldReserve(maxLogMemSize - newEntryMemSize);
+ }
+
Review comment:
What about if `maxLogMemSize < newEntryMemSize ` ?
In this case, this implementation will remove all commit logs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org