You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2020/07/10 10:45:06 UTC

[GitHub] [incubator-iotdb] zhanglingzhe0820 opened a new pull request #1480: update log and add some reader close

zhanglingzhe0820 opened a new pull request #1480:
URL: https://github.com/apache/incubator-iotdb/pull/1480


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #1480: update log and add some reader close

Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #1480:
URL: https://github.com/apache/incubator-iotdb/pull/1480#discussion_r454336549



##########
File path: server/src/main/java/org/apache/iotdb/db/engine/storagegroup/TsFileProcessor.java
##########
@@ -600,11 +628,13 @@ private void deleteVmFiles(List<TsFileResource> vmMergeTsFiles,
         tsFileResource.getTsFile().getName());
     for (int i = 0; i < vmMergeTsFiles.size(); i++) {
       vmMergeWriters.get(i).close();
-      logger.info("{} vm file open a writer", vmMergeWriters.get(i).getFile().getName());
+      logger.info("{} vm file close a writer", vmMergeWriters.get(i).getFile().getName());

Review comment:
       change to debug level




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #1480: update log and add some reader close

Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #1480:
URL: https://github.com/apache/incubator-iotdb/pull/1480#discussion_r454338415



##########
File path: server/src/main/java/org/apache/iotdb/db/engine/storagegroup/TsFileProcessor.java
##########
@@ -802,9 +840,17 @@ public void flushOneMemTable() {
         mergeWorking = true;
         logger.info("{}: {} submit a vm merge task", storageGroupName,
             tsFileResource.getTsFile().getName());
+        List<List<TsFileResource>> currVmTsFileResources = new ArrayList<>();

Review comment:
       add javadoc, or rename this to copiedVmTsFileResources




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #1480: update log and add some reader close

Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #1480:
URL: https://github.com/apache/incubator-iotdb/pull/1480#discussion_r454337303



##########
File path: server/src/main/java/org/apache/iotdb/db/engine/storagegroup/TsFileProcessor.java
##########
@@ -683,10 +715,16 @@ public void flushOneMemTable() {
         RestorableTsFileIOWriter curWriter;
         if (config.isEnableVm()) {
           logger.info("[Flush] flush a vm");
-          File newVmFile = createNewVMFile(tsFileResource);
-          vmTsFileResources.add(new TsFileResource(newVmFile));
+          File newVmFile = createNewVMFile(tsFileResource, 0);
+          if (vmWriters.size() <= 0) {
+            vmWriters.add(new ArrayList<>());
+          }
+          if (vmTsFileResources.size() <= 0) {
+            vmTsFileResources.add(new ArrayList<>());
+          }

Review comment:
       are they consistent and can be put together?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [incubator-iotdb] qiaojialin merged pull request #1480: update log and add some reader close

Posted by GitBox <gi...@apache.org>.
qiaojialin merged pull request #1480:
URL: https://github.com/apache/incubator-iotdb/pull/1480


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #1480: update log and add some reader close

Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #1480:
URL: https://github.com/apache/incubator-iotdb/pull/1480#discussion_r454336001



##########
File path: server/src/main/java/org/apache/iotdb/db/engine/storagegroup/TsFileProcessor.java
##########
@@ -586,12 +603,23 @@ private void releaseFlushedMemTable(IMemTable memTable) {
     }
   }
 
-  public static File createNewVMFile(TsFileResource tsFileResource) {
-    File parent = tsFileResource.getTsFile().getParentFile();
-    return FSFactoryProducer.getFSFactory().getFile(parent,
-        tsFileResource.getTsFile().getName() + IoTDBConstant.FILE_NAME_SEPARATOR + System
-            .currentTimeMillis()
-            + VM_SUFFIX);
+  public File createNewVMFile(TsFileResource tsFileResource, int level) {
+    vmFileCreateLock.writeLock().lock();
+    try {
+      TimeUnit.MILLISECONDS.sleep(1);
+      File parent = tsFileResource.getTsFile().getParentFile();
+      File newVmFile = FSFactoryProducer.getFSFactory().getFile(parent,
+          tsFileResource.getTsFile().getName() + IoTDBConstant.FILE_NAME_SEPARATOR + System
+              .currentTimeMillis() + IoTDBConstant.FILE_NAME_SEPARATOR + level

Review comment:
       I prefer to put the level ahead of system time...




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [incubator-iotdb] qiaojialin commented on a change in pull request #1480: update log and add some reader close

Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #1480:
URL: https://github.com/apache/incubator-iotdb/pull/1480#discussion_r454325658



##########
File path: server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
##########
@@ -574,16 +575,25 @@ else if (upgradeFolder.exists()) {
       }
     }
 
-    Map<String, List<TsFileResource>> vmTsFileResourceMap = new HashMap<>();
+    Map<String, List<List<TsFileResource>>> vmTsFileResourceMap = new HashMap<>();
     for (File f : vmFiles) {
       TsFileResource fileResource = new TsFileResource(f);
       fileResource.setClosed(false);
       String tsfilePrefix = f.getPath()
-          .substring(0, f.getPath().lastIndexOf(FILE_NAME_SEPARATOR));
-      vmTsFileResourceMap.computeIfAbsent(tsfilePrefix, k -> new ArrayList<>()).add(fileResource);
+          .substring(0, f.getPath().lastIndexOf(TSFILE_SUFFIX)) + TSFILE_SUFFIX;
+      String vmVersionStr = f.getPath()

Review comment:
       ```suggestion
         String vmLevel = f.getPath()
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org