You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2021/07/22 09:01:15 UTC
[GitHub] [iotdb] HeimingZ opened a new pull request #3610: [IOTDB-1496] Timed flush memtable
HeimingZ opened a new pull request #3610:
URL: https://github.com/apache/iotdb/pull/3610
Because some memtables are accumulated slowly, so add memtable_flush_interval to flush inactive memtables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on a change in pull request #3610: [IOTDB-1496] Timed flush memtable
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #3610:
URL: https://github.com/apache/iotdb/pull/3610#discussion_r677234611
##########
File path: server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
##########
@@ -1552,6 +1552,29 @@ private void checkFileTTL(TsFileResource resource, long ttlLowerBound, boolean i
}
}
+ public void timedFlushMemTable() {
+ List<TsFileProcessor> tsFileProcessors = new ArrayList<>();
+ // only check unsequence tsfiles' memtables
+ tsFileProcessors.addAll(workUnsequenceTsFileProcessors.values());
+ long timestampBaseline = System.currentTimeMillis() - config.getUnseqMemtableFlushInterval();
Review comment:
put into writeLock
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3610: [IOTDB-1496] Timed flush memtable
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3610:
URL: https://github.com/apache/iotdb/pull/3610#issuecomment-885375690
[](https://coveralls.io/builds/41673736)
Coverage increased (+0.03%) to 68.094% when pulling **15077723f29b8c2ab40a4883752f1778f59529a6 on HeimingZ:IOTDB-1496(timed-flush-memtalbe)** into **47fa594403d5bb0c79d8414886b5a9fe6a24b778 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3610: [IOTDB-1496] Timed flush memtable
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3610:
URL: https://github.com/apache/iotdb/pull/3610#issuecomment-885375690
[](https://coveralls.io/builds/41676170)
Coverage increased (+0.007%) to 68.076% when pulling **782f85a53b8ee3c95f7192f176e678acdbfc22cd on HeimingZ:IOTDB-1496(timed-flush-memtalbe)** into **47fa594403d5bb0c79d8414886b5a9fe6a24b778 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HeimingZ commented on pull request #3610: [IOTDB-1496] Timed flush memtable
Posted by GitBox <gi...@apache.org>.
HeimingZ commented on pull request #3610:
URL: https://github.com/apache/iotdb/pull/3610#issuecomment-887139871
> Good Job.
> There are some problems. If some caches are in use and we just get cleaned up, it's not going to affect its performance.
1. Users can use unseq_memtable_flush_interval param to specify the alive time of memtables according to their use cases.
2. Change the timed flush level from all memtables to unsequence memtables because zombie memtables often oocurs in unsequence tsfiles.
3. Add enable_timed_flush_unseq_memtable param, so users can decide whether to enable this function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HeimingZ commented on a change in pull request #3610: [IOTDB-1496] Timed flush memtable
Posted by GitBox <gi...@apache.org>.
HeimingZ commented on a change in pull request #3610:
URL: https://github.com/apache/iotdb/pull/3610#discussion_r677241442
##########
File path: server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
##########
@@ -1552,6 +1552,29 @@ private void checkFileTTL(TsFileResource resource, long ttlLowerBound, boolean i
}
}
+ public void timedFlushMemTable() {
+ List<TsFileProcessor> tsFileProcessors = new ArrayList<>();
+ // only check unsequence tsfiles' memtables
+ tsFileProcessors.addAll(workUnsequenceTsFileProcessors.values());
+ long timestampBaseline = System.currentTimeMillis() - config.getUnseqMemtableFlushInterval();
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3610: [IOTDB-1496] Timed flush memtable
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3610:
URL: https://github.com/apache/iotdb/pull/3610#issuecomment-885375690
[](https://coveralls.io/builds/41676738)
Coverage increased (+0.01%) to 68.082% when pulling **782f85a53b8ee3c95f7192f176e678acdbfc22cd on HeimingZ:IOTDB-1496(timed-flush-memtalbe)** into **47fa594403d5bb0c79d8414886b5a9fe6a24b778 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3610: [IOTDB-1496] Timed flush memtable
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3610:
URL: https://github.com/apache/iotdb/pull/3610#issuecomment-885375690
[](https://coveralls.io/builds/41677431)
Coverage increased (+0.001%) to 68.07% when pulling **782f85a53b8ee3c95f7192f176e678acdbfc22cd on HeimingZ:IOTDB-1496(timed-flush-memtalbe)** into **47fa594403d5bb0c79d8414886b5a9fe6a24b778 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3610: [IOTDB-1496] Timed flush memtable
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3610:
URL: https://github.com/apache/iotdb/pull/3610#issuecomment-885375690
[](https://coveralls.io/builds/41671462)
Coverage decreased (-0.003%) to 68.066% when pulling **970dbb83a7cc2f96627963abab42e96d47e16256 on HeimingZ:IOTDB-1496(timed-flush-memtalbe)** into **47fa594403d5bb0c79d8414886b5a9fe6a24b778 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin merged pull request #3610: [IOTDB-1496] Timed flush memtable
Posted by GitBox <gi...@apache.org>.
qiaojialin merged pull request #3610:
URL: https://github.com/apache/iotdb/pull/3610
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls commented on pull request #3610: [IOTDB-1496] Timed flush memtable
Posted by GitBox <gi...@apache.org>.
coveralls commented on pull request #3610:
URL: https://github.com/apache/iotdb/pull/3610#issuecomment-885375690
[](https://coveralls.io/builds/41593135)
Coverage increased (+0.001%) to 68.167% when pulling **2c680d99621538426aa805106cd4dde5049a0912 on HeimingZ:IOTDB-1496(timed-flush-memtalbe)** into **5a348a49275db78d9a17e5ff166570533f850d37 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3610: [IOTDB-1496] Timed flush memtable
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3610:
URL: https://github.com/apache/iotdb/pull/3610#issuecomment-885375690
[](https://coveralls.io/builds/41677665)
Coverage remained the same at 68.069% when pulling **782f85a53b8ee3c95f7192f176e678acdbfc22cd on HeimingZ:IOTDB-1496(timed-flush-memtalbe)** into **47fa594403d5bb0c79d8414886b5a9fe6a24b778 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3610: [IOTDB-1496] Timed flush memtable
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3610:
URL: https://github.com/apache/iotdb/pull/3610#issuecomment-885375690
[](https://coveralls.io/builds/41670253)
Coverage increased (+0.02%) to 68.089% when pulling **c0c64ff1da10543312d4354c831a95c7b62a0e76 on HeimingZ:IOTDB-1496(timed-flush-memtalbe)** into **47fa594403d5bb0c79d8414886b5a9fe6a24b778 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org