You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iotdb.apache.org by qi...@apache.org on 2020/04/10 05:27:15 UTC
[incubator-iotdb] 01/01: update delete doc
This is an automated email from the ASF dual-hosted git repository.
qiaojialin pushed a commit to branch add_delete_doc
in repository https://gitbox.apache.org/repos/asf/incubator-iotdb.git
commit 2b3209d0dbcc91fbc79130cabf6bdc06715fce78
Author: qiaojialin <64...@qq.com>
AuthorDate: Fri Apr 10 13:26:54 2020 +0800
update delete doc
---
docs/SystemDesign/4-StorageEngine/6-DataManipulation.md | 17 ++++++++++-------
.../SystemDesign/4-StorageEngine/6-DataManipulation.md | 16 ++++++++++------
.../db/engine/storagegroup/StorageGroupProcessor.java | 2 +-
3 files changed, 21 insertions(+), 14 deletions(-)
diff --git a/docs/SystemDesign/4-StorageEngine/6-DataManipulation.md b/docs/SystemDesign/4-StorageEngine/6-DataManipulation.md
index 60d6253..78dd676 100644
--- a/docs/SystemDesign/4-StorageEngine/6-DataManipulation.md
+++ b/docs/SystemDesign/4-StorageEngine/6-DataManipulation.md
@@ -70,14 +70,17 @@ Old data is automatically deleted by merging, see:
* Corresponding interface
* JDBC's execute interface, using delete SQL statements
-* Main entrance: public void delete(String deviceId, String measurementId, long timestamp) StorageEngine.java
- * Find the corresponding StorageGroupProcessor
- * Find all TsfileProcessor affected
- * Pre-write log
- * Find all TsfileResources affected
- * Record the point in time of deletion in the mod file
- * If the file is not closed (the corresponding TsfileProcessor exists), delete the data in memory
+Each StorageGroupProcessor maintains a ascending version for each partition, which is managed by SimpleFileVersionController.
+Each memtable will apply a version when submitted to flush. After flushing to TsFile, a current position-version will added to TsFileMetadata.
+This information will be used to set version to ChunkMetadata when query.
+* main entrance: public void delete(String deviceId, String measurementId, long timestamp) StorageEngine.java
+ * Find the corresponding StorageGroupProcessor
+ * Find all impacted working TsFileProcessors to write WAL
+ * Find all impacted TsFileResources to record a Modification in its mods file, the Modification format is: path,deleteTime,version
+ * If the TsFile is not closed,get its TsFileProcessor
+ * If there exists the working memtable, delete data in it
+ * If there exists flushing memtable,record the deleted time in it for query.(Notice that the Modification is recorded in mods for these memtables)
## Data TTL setting
diff --git a/docs/zh/SystemDesign/4-StorageEngine/6-DataManipulation.md b/docs/zh/SystemDesign/4-StorageEngine/6-DataManipulation.md
index 8b60018..51f024b 100644
--- a/docs/zh/SystemDesign/4-StorageEngine/6-DataManipulation.md
+++ b/docs/zh/SystemDesign/4-StorageEngine/6-DataManipulation.md
@@ -71,13 +71,17 @@
* 对应的接口
* JDBC 的 execute 接口,使用delete SQL语句
+每个 StorageGroupProsessor 中针对每个分区会维护一个自增的版本号,由 SimpleFileVersionController 管理。
+每个内存缓冲区 memtable 在持久化的时候会申请一个版本号。持久化到 TsFile 后,会在 TsFileMetadata 中记录此 memtable 对应的 多个 ChunkGroup 的终止位置和版本号。
+查询时会根据此信息对 ChunkMetadata 赋 version。
+
* 总入口: public void delete(String deviceId, String measurementId, long timestamp) StorageEngine.java
- * 找到对应的 StorageGroupProcessor
- * 找到受影响的所有TsfileProcessor
- * 写写前日志
- * 找到受影响的所有TsfileResource
- * 在mod文件中记录删除的时间点
- * 如果文件没有关闭(存在对应的TsfileProcessor),则删除内存中的数据
+ * 找到对应的 StorageGroupProcessor
+ * 找到受影响的所有 working TsFileProcessor 记录写前日志
+ * 找到受影响的所有 TsFileResource,在其对应的 mods 文件中记录一条记录:path,deleteTime,version
+ * 如果文件没有关闭,拿到对应的 TsFileProcessor
+ * 如果存在 working memtable:则删除内存中的数据
+ * 如果存在 正在 flush 的 memtable,记录一条记录,查询时跳过删掉的数据(注意此时文件中已经为这些 memtable 记录了 mods)
## 数据TTL设置
diff --git a/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java b/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
index d3afe72..27d0acf 100755
--- a/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
+++ b/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
@@ -1149,7 +1149,7 @@ public class StorageGroupProcessor {
// time partition to divide storage group
long timePartitionId = StorageEngine.getTimePartition(timestamp);
- // write log
+ // write log to impacted working TsFileProcessors
logDeletion(timestamp, deviceId, measurementId, timePartitionId);
Path fullPath = new Path(deviceId, measurementId);