You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iotdb.apache.org by qi...@apache.org on 2020/04/10 05:27:14 UTC
[incubator-iotdb] branch add_delete_doc created (now 2b3209d)
This is an automated email from the ASF dual-hosted git repository.
qiaojialin pushed a change to branch add_delete_doc
in repository https://gitbox.apache.org/repos/asf/incubator-iotdb.git.
at 2b3209d update delete doc
This branch includes the following new commits:
new 2b3209d update delete doc
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[incubator-iotdb] 01/01: update delete doc
Posted by qi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
qiaojialin pushed a commit to branch add_delete_doc
in repository https://gitbox.apache.org/repos/asf/incubator-iotdb.git
commit 2b3209d0dbcc91fbc79130cabf6bdc06715fce78
Author: qiaojialin <64...@qq.com>
AuthorDate: Fri Apr 10 13:26:54 2020 +0800
update delete doc
---
docs/SystemDesign/4-StorageEngine/6-DataManipulation.md | 17 ++++++++++-------
.../SystemDesign/4-StorageEngine/6-DataManipulation.md | 16 ++++++++++------
.../db/engine/storagegroup/StorageGroupProcessor.java | 2 +-
3 files changed, 21 insertions(+), 14 deletions(-)
diff --git a/docs/SystemDesign/4-StorageEngine/6-DataManipulation.md b/docs/SystemDesign/4-StorageEngine/6-DataManipulation.md
index 60d6253..78dd676 100644
--- a/docs/SystemDesign/4-StorageEngine/6-DataManipulation.md
+++ b/docs/SystemDesign/4-StorageEngine/6-DataManipulation.md
@@ -70,14 +70,17 @@ Old data is automatically deleted by merging, see:
* Corresponding interface
* JDBC's execute interface, using delete SQL statements
-* Main entrance: public void delete(String deviceId, String measurementId, long timestamp) StorageEngine.java
- * Find the corresponding StorageGroupProcessor
- * Find all TsfileProcessor affected
- * Pre-write log
- * Find all TsfileResources affected
- * Record the point in time of deletion in the mod file
- * If the file is not closed (the corresponding TsfileProcessor exists), delete the data in memory
+Each StorageGroupProcessor maintains a ascending version for each partition, which is managed by SimpleFileVersionController.
+Each memtable will apply a version when submitted to flush. After flushing to TsFile, a current position-version will added to TsFileMetadata.
+This information will be used to set version to ChunkMetadata when query.
+* main entrance: public void delete(String deviceId, String measurementId, long timestamp) StorageEngine.java
+ * Find the corresponding StorageGroupProcessor
+ * Find all impacted working TsFileProcessors to write WAL
+ * Find all impacted TsFileResources to record a Modification in its mods file, the Modification format is: path，deleteTime，version
+ * If the TsFile is not closed，get its TsFileProcessor
+ * If there exists the working memtable, delete data in it
+ * If there exists flushing memtable，record the deleted time in it for query.（Notice that the Modification is recorded in mods for these memtables）
## Data TTL setting
diff --git a/docs/zh/SystemDesign/4-StorageEngine/6-DataManipulation.md b/docs/zh/SystemDesign/4-StorageEngine/6-DataManipulation.md
index 8b60018..51f024b 100644
--- a/docs/zh/SystemDesign/4-StorageEngine/6-DataManipulation.md
+++ b/docs/zh/SystemDesign/4-StorageEngine/6-DataManipulation.md
@@ -71,13 +71,17 @@
* 对应的接口
* JDBC 的 execute 接口，使用delete SQL语句
+每个 StorageGroupProsessor 中针对每个分区会维护一个自增的版本号，由 SimpleFileVersionController 管理。
+每个内存缓冲区 memtable 在持久化的时候会申请一个版本号。持久化到 TsFile 后，会在 TsFileMetadata 中记录此 memtable 对应的 多个 ChunkGroup 的终止位置和版本号。
+查询时会根据此信息对 ChunkMetadata 赋 version。
+
* 总入口: public void delete(String deviceId, String measurementId, long timestamp) StorageEngine.java
- * 找到对应的 StorageGroupProcessor
- * 找到受影响的所有TsfileProcessor
- * 写写前日志
- * 找到受影响的所有TsfileResource
- * 在mod文件中记录删除的时间点
- * 如果文件没有关闭（存在对应的TsfileProcessor），则删除内存中的数据
+ * 找到对应的 StorageGroupProcessor
+ * 找到受影响的所有 working TsFileProcessor 记录写前日志
+ * 找到受影响的所有 TsFileResource，在其对应的 mods 文件中记录一条记录：path，deleteTime，version
+ * 如果文件没有关闭，拿到对应的 TsFileProcessor
+ * 如果存在 working memtable：则删除内存中的数据
+ * 如果存在 正在 flush 的 memtable，记录一条记录，查询时跳过删掉的数据（注意此时文件中已经为这些 memtable 记录了 mods）
## 数据TTL设置
diff --git a/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java b/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
index d3afe72..27d0acf 100755
--- a/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
+++ b/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/StorageGroupProcessor.java
@@ -1149,7 +1149,7 @@ public class StorageGroupProcessor {
// time partition to divide storage group
long timePartitionId = StorageEngine.getTimePartition(timestamp);
- // write log
+ // write log to impacted working TsFileProcessors
logDeletion(timestamp, deviceId, measurementId, timePartitionId);
Path fullPath = new Path(deviceId, measurementId);