You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iotdb.apache.org by qi...@apache.org on 2022/11/16 11:17:59 UTC
[iotdb] branch rel/0.13 updated: [To rel/0.13][IOTDB-4946] Show Deadband Process Info (#8010)
This is an automated email from the ASF dual-hosted git repository.
qiaojialin pushed a commit to branch rel/0.13
in repository https://gitbox.apache.org/repos/asf/iotdb.git
The following commit(s) were added to refs/heads/rel/0.13 by this push:
new 88ba0d3d0e [To rel/0.13][IOTDB-4946] Show Deadband Process Info (#8010)
88ba0d3d0e is described below
commit 88ba0d3d0ea3e852d3952a110ccee7eccbc57e55
Author: Marcos_Zyk <38...@users.noreply.github.com>
AuthorDate: Wed Nov 16 19:17:53 2022 +0800
[To rel/0.13][IOTDB-4946] Show Deadband Process Info (#8010)
---
client-py/tests/test_dataframe.py | 4 +
docs/UserGuide/Data-Concept/Compression.md | 91 ----------------------
.../{Compression.md => Deadband-Process.md} | 29 +------
docs/zh/UserGuide/Data-Concept/Compression.md | 86 --------------------
.../{Compression.md => Deadband-Process.md} | 26 +------
.../iotdb/db/integration/IoTDBMetadataFetchIT.java | 78 +++++++++++++++----
.../versionadaption/IoTDBDDLVersionAdaptionIT.java | 30 +++----
.../org/apache/iotdb/db/conf/IoTDBConstant.java | 4 +
.../org/apache/iotdb/db/metadata/MManager.java | 10 ++-
.../org/apache/iotdb/db/metadata/mtree/MTree.java | 6 +-
.../apache/iotdb/db/metadata/utils/MetaUtils.java | 20 +++++
.../db/query/dataset/ShowTimeSeriesResult.java | 17 +++-
.../db/query/dataset/ShowTimeseriesDataSet.java | 10 ++-
.../iotdb/db/protocol/rest/IoTDBRestServiceIT.java | 4 +
.../db/query/dataset/ShowTimeSeriesResultTest.java | 4 +-
site/src/main/.vuepress/config.js | 2 +
.../zeppelin/iotdb/IoTDBInterpreterTest.java | 14 ++--
17 files changed, 162 insertions(+), 273 deletions(-)
diff --git a/client-py/tests/test_dataframe.py b/client-py/tests/test_dataframe.py
index 9a341e1e48..6ff2d30ca4 100644
--- a/client-py/tests/test_dataframe.py
+++ b/client-py/tests/test_dataframe.py
@@ -66,6 +66,8 @@ def test_non_time_query():
"compression",
"tags",
"attributes",
+ "deadband",
+ "deadband parameters"
]
assert_array_equal(
df.values,
@@ -78,6 +80,8 @@ def test_non_time_query():
"SNAPPY",
None,
None,
+ None,
+ None,
]
],
)
diff --git a/docs/UserGuide/Data-Concept/Compression.md b/docs/UserGuide/Data-Concept/Compression.md
index adc808f3e0..99dea55e7b 100644
--- a/docs/UserGuide/Data-Concept/Compression.md
+++ b/docs/UserGuide/Data-Concept/Compression.md
@@ -37,97 +37,6 @@ IoTDB allows you to specify the compression method of the column when creating a
The specified syntax for compression is detailed in [Create Timeseries Statement](../Reference/SQL-Reference.md).
-## SDT
-
-The Swinging Door Trending (SDT) algorithm is a lossy compression algorithm.
-SDT has low computational complexity and uses a linear trend to represent a quantity of data.
-
-In IoTDB SDT compresses and discards data when flushing into the disk.
-
-IoTDB allows you to specify the properties of SDT when creating a time series, and supports three properties:
-
-* CompDev (Compression Deviation)
-
-CompDev is the most important parameter in SDT that represents the maximum difference between the
-current sample and the current linear trend. CompDev needs to be greater than 0 to perform compression.
-
-* CompMinTime (Compression Minimum Time Interval)
-
-CompMinTime is a parameter measures the time distance between two stored data points, which is used for noisy reduction.
-If the time interval between the current point and the last stored point is less than or equal to its value,
-current point will NOT be stored regardless of compression deviation.
-The default value is 0 with time unit ms.
-
-* CompMaxTime (Compression Maximum Time Interval)
-
-CompMaxTime is a parameter measure the time distance between two stored data points.
-If the time interval between the current point and the last stored point is greater than or equal to its value,
-current point will be stored regardless of compression deviation.
-The default value is 9,223,372,036,854,775,807 with time unit ms.
-
-The specified syntax for SDT is detailed in [Create Timeseries Statement](../Reference/SQL-Reference.md).
-
-Supported datatypes:
-
-* INT32 (Integer)
-* INT64 (Long Integer)
-* FLOAT (Single Precision Floating Point)
-* DOUBLE (Double Precision Floating Point)
-
-The following is an example of using SDT compression.
-
-```
-IoTDB> CREATE TIMESERIES root.sg1.d0.s0 WITH DATATYPE=INT32,ENCODING=PLAIN,LOSS=SDT,COMPDEV=2
-```
-
-Prior to flushing and SDT compression, the results are shown below:
-
-```
-IoTDB> SELECT s0 FROM root.sg1.d0
-+-----------------------------+--------------+
-| Time|root.sg1.d0.s0|
-+-----------------------------+--------------+
-|2017-11-01T00:06:00.001+08:00| 1|
-|2017-11-01T00:06:00.002+08:00| 1|
-|2017-11-01T00:06:00.003+08:00| 1|
-|2017-11-01T00:06:00.004+08:00| 1|
-|2017-11-01T00:06:00.005+08:00| 1|
-|2017-11-01T00:06:00.006+08:00| 1|
-|2017-11-01T00:06:00.007+08:00| 1|
-|2017-11-01T00:06:00.015+08:00| 10|
-|2017-11-01T00:06:00.016+08:00| 20|
-|2017-11-01T00:06:00.017+08:00| 1|
-|2017-11-01T00:06:00.018+08:00| 30|
-+-----------------------------+--------------+
-Total line number = 11
-It costs 0.008s
-```
-
-After flushing and SDT compression, the results are shown below:
-
-```
-IoTDB> FLUSH
-IoTDB> SELECT s0 FROM root.sg1.d0
-+-----------------------------+--------------+
-| Time|root.sg1.d0.s0|
-+-----------------------------+--------------+
-|2017-11-01T00:06:00.001+08:00| 1|
-|2017-11-01T00:06:00.007+08:00| 1|
-|2017-11-01T00:06:00.015+08:00| 10|
-|2017-11-01T00:06:00.016+08:00| 20|
-|2017-11-01T00:06:00.017+08:00| 1|
-+-----------------------------+--------------+
-Total line number = 5
-It costs 0.044s
-```
-
-SDT takes effect when flushing to the disk. The SDT algorithm always stores the first point and does not store the last point.
-
-The data in [2017-11-01T00:06:00.001, 2017-11-01T00:06:00.007] is within the compression deviation thus discarded.
-The data point at time 2017-11-01T00:06:00.007 is stored because the next data point at time 2017-11-01T00:06:00.015
-exceeds compression deviation. When a data point exceeds the compression deviation, SDT stores the last read
-point and updates the upper and lower boundaries. The last point at time 2017-11-01T00:06:00.018 is not stored.
-
## Compression Ratio Statistics
Compression ratio statistics file: data/system/storage_groups/compression_ratio/Ratio-{ratio_sum}-{memtable_flush_time}
diff --git a/docs/UserGuide/Data-Concept/Compression.md b/docs/UserGuide/Data-Concept/Deadband-Process.md
similarity index 75%
copy from docs/UserGuide/Data-Concept/Compression.md
copy to docs/UserGuide/Data-Concept/Deadband-Process.md
index adc808f3e0..32fe030ac6 100644
--- a/docs/UserGuide/Data-Concept/Compression.md
+++ b/docs/UserGuide/Data-Concept/Deadband-Process.md
@@ -19,27 +19,11 @@
-->
-# Compression
-
-When the time series is written and encoded as binary data according to the specified type, IoTDB compresses the data using compression technology to further improve space storage efficiency. Although both encoding and compression are designed to improve storage efficiency, encoding techniques are usually available only for specific data types (e.g., second-order differential encoding is only suitable for INT32 or INT64 data type, and storing floating-point numbers requires multiplying t [...]
-
-## Basic Compression Methods
-
-IoTDB allows you to specify the compression method of the column when creating a time series, and supports the following compression methods:
-
-* UNCOMPRESSED
-
-* SNAPPY
-
-* LZ4
-
-* GZIP
-
-The specified syntax for compression is detailed in [Create Timeseries Statement](../Reference/SQL-Reference.md).
+# Deadband Process
## SDT
-The Swinging Door Trending (SDT) algorithm is a lossy compression algorithm.
+The Swinging Door Trending (SDT) algorithm is a deadband process algorithm.
SDT has low computational complexity and uses a linear trend to represent a quantity of data.
In IoTDB SDT compresses and discards data when flushing into the disk.
@@ -127,12 +111,3 @@ The data in [2017-11-01T00:06:00.001, 2017-11-01T00:06:00.007] is within the com
The data point at time 2017-11-01T00:06:00.007 is stored because the next data point at time 2017-11-01T00:06:00.015
exceeds compression deviation. When a data point exceeds the compression deviation, SDT stores the last read
point and updates the upper and lower boundaries. The last point at time 2017-11-01T00:06:00.018 is not stored.
-
-## Compression Ratio Statistics
-
-Compression ratio statistics file: data/system/storage_groups/compression_ratio/Ratio-{ratio_sum}-{memtable_flush_time}
-
-* ratio_sum: sum of memtable compression ratios
-* memtable_flush_time: memtable flush times
-
-The average compression ratio can be calculated by `ratio_sum / memtable_flush_time`
\ No newline at end of file
diff --git a/docs/zh/UserGuide/Data-Concept/Compression.md b/docs/zh/UserGuide/Data-Concept/Compression.md
index 0bdf473694..b00448a9d7 100644
--- a/docs/zh/UserGuide/Data-Concept/Compression.md
+++ b/docs/zh/UserGuide/Data-Concept/Compression.md
@@ -34,92 +34,6 @@ IoTDB 允许在创建一个时间序列的时候指定该列的压缩方式。
压缩方式的指定语法详见本文 [SQL 参考文档](../Reference/SQL-Reference.md)。
-### 旋转门压缩
-
-旋转门压缩（SDT）算法是一种有损压缩算法。SDT 的计算复杂度较低，并使用线性趋势来表示大量数据。
-
-在 IoTDB 中，SDT 在刷新到磁盘时会压缩并丢弃数据。
-
-IoTDB 允许您在创建时间序列时指定 SDT 的属性，并支持以下三个属性：
-
-* CompDev （Compression Deviation，压缩偏差）
-
-CompDev 是 SDT 中最重要的参数，它表示当前样本与当前线性趋势之间的最大差值。CompDev 设置的值需要大于 0。
-
-* CompMinTime （Compression Minimum Time Interval，最小压缩时间间隔）
-
-CompMinTime 是测量两个存储的数据点之间的时间距离的参数，用于减少噪声。
-如果当前点和最后存储的点之间的时间间隔小于或等于其值，则无论压缩偏差如何，都不会存储当前点。
-默认值为 0，单位为毫秒。
-
-* CompMaxTime （Compression Maximum Time Interval，最大压缩时间间隔）
-
-CompMaxTime 是测量两个存储的数据点之间的时间距离的参数。
-如果当前点和最后一个存储点之间的时间间隔大于或等于其值，
-无论压缩偏差如何，都将存储当前点。
-默认值为 9,223,372,036,854,775,807，单位为毫秒。
-
-支持的数据类型：
-
-* INT32（整型）
-* INT64（长整型）
-* FLOAT（单精度浮点数）
-* DOUBLE（双精度浮点数）
-
-SDT 的指定语法详见本文 [SQL 参考文档](../Reference/SQL-Reference.md)。
-
-以下是使用 SDT 压缩的示例。
-
-```
-IoTDB> CREATE TIMESERIES root.sg1.d0.s0 WITH DATATYPE=INT32,ENCODING=PLAIN,LOSS=SDT,COMPDEV=2
-```
-
-刷入磁盘和 SDT 压缩之前，结果如下所示：
-
-```
-IoTDB> SELECT s0 FROM root.sg1.d0
-+-----------------------------+--------------+
-| Time|root.sg1.d0.s0|
-+-----------------------------+--------------+
-|2017-11-01T00:06:00.001+08:00| 1|
-|2017-11-01T00:06:00.002+08:00| 1|
-|2017-11-01T00:06:00.003+08:00| 1|
-|2017-11-01T00:06:00.004+08:00| 1|
-|2017-11-01T00:06:00.005+08:00| 1|
-|2017-11-01T00:06:00.006+08:00| 1|
-|2017-11-01T00:06:00.007+08:00| 1|
-|2017-11-01T00:06:00.015+08:00| 10|
-|2017-11-01T00:06:00.016+08:00| 20|
-|2017-11-01T00:06:00.017+08:00| 1|
-|2017-11-01T00:06:00.018+08:00| 30|
-+-----------------------------+--------------+
-Total line number = 11
-It costs 0.008s
-```
-
-刷入磁盘和 SDT 压缩之后，结果如下所示：
-```
-IoTDB> FLUSH
-IoTDB> SELECT s0 FROM root.sg1.d0
-+-----------------------------+--------------+
-| Time|root.sg1.d0.s0|
-+-----------------------------+--------------+
-|2017-11-01T00:06:00.001+08:00| 1|
-|2017-11-01T00:06:00.007+08:00| 1|
-|2017-11-01T00:06:00.015+08:00| 10|
-|2017-11-01T00:06:00.016+08:00| 20|
-|2017-11-01T00:06:00.017+08:00| 1|
-+-----------------------------+--------------+
-Total line number = 5
-It costs 0.044s
-```
-
-SDT 在刷新到磁盘时进行压缩。 SDT 算法始终存储第一个点，并且不存储最后一个点。
-
-时间范围在 [2017-11-01T00:06:00.001, 2017-11-01T00:06:00.007] 的数据在压缩偏差内，因此被压缩和丢弃。
-之所以存储时间为 2017-11-01T00:06:00.007 的数据点，是因为下一个数据点 2017-11-01T00:06:00.015 的值超过压缩偏差。
-当一个数据点超过压缩偏差时，SDT 将存储上一个读取的数据点，并重新计算上下压缩边界。作为最后一个数据点，不存储时间 2017-11-01T00:06:00.018。
-
### 压缩比统计信息
压缩比统计信息文件：data/system/storage_groups/compression_ratio/Ratio-{ratio_sum}-{memtable_flush_time}
diff --git a/docs/zh/UserGuide/Data-Concept/Compression.md b/docs/zh/UserGuide/Data-Concept/Deadband-Process.md
similarity index 77%
copy from docs/zh/UserGuide/Data-Concept/Compression.md
copy to docs/zh/UserGuide/Data-Concept/Deadband-Process.md
index 0bdf473694..42733e695b 100644
--- a/docs/zh/UserGuide/Data-Concept/Compression.md
+++ b/docs/zh/UserGuide/Data-Concept/Deadband-Process.md
@@ -19,24 +19,11 @@
-->
-## 压缩方式
-
-当时间序列写入并按照指定的类型编码为二进制数据后，IoTDB 会使用压缩技术对该数据进行压缩，进一步提升空间存储效率。虽然编码和压缩都旨在提升存储效率，但编码技术通常只适合特定的数据类型（如二阶差分编码只适合与 INT32 或者 INT64 编码，存储浮点数需要先将他们乘以 10m 以转换为整数），然后将它们转换为二进制流。压缩方式（SNAPPY）针对二进制流进行压缩，因此压缩方式的使用不再受数据类型的限制。
-
-### 基本压缩方式
-
-IoTDB 允许在创建一个时间序列的时候指定该列的压缩方式。现阶段 IoTDB 支持以下几种压缩方式：
-
-* UNCOMPRESSED（不压缩）
-* SNAPPY 压缩
-* LZ4 压缩
-* GZIP 压缩
-
-压缩方式的指定语法详见本文 [SQL 参考文档](../Reference/SQL-Reference.md)。
+## 死区处理
### 旋转门压缩
-旋转门压缩（SDT）算法是一种有损压缩算法。SDT 的计算复杂度较低，并使用线性趋势来表示大量数据。
+旋转门压缩（SDT）算法是一种死区处理算法。SDT 的计算复杂度较低，并使用线性趋势来表示大量数据。
在 IoTDB 中，SDT 在刷新到磁盘时会压缩并丢弃数据。
@@ -119,12 +106,3 @@ SDT 在刷新到磁盘时进行压缩。 SDT 算法始终存储第一个点，
时间范围在 [2017-11-01T00:06:00.001, 2017-11-01T00:06:00.007] 的数据在压缩偏差内，因此被压缩和丢弃。
之所以存储时间为 2017-11-01T00:06:00.007 的数据点，是因为下一个数据点 2017-11-01T00:06:00.015 的值超过压缩偏差。
当一个数据点超过压缩偏差时，SDT 将存储上一个读取的数据点，并重新计算上下压缩边界。作为最后一个数据点，不存储时间 2017-11-01T00:06:00.018。
-
-### 压缩比统计信息
-
-压缩比统计信息文件：data/system/storage_groups/compression_ratio/Ratio-{ratio_sum}-{memtable_flush_time}
-
-* ratio_sum: memtable压缩比的总和
-* memtable_flush_time: memtable刷盘的总次数
-
-通过 `ratio_sum / memtable_flush_time` 可以计算出平均压缩比
\ No newline at end of file
diff --git a/integration/src/test/java/org/apache/iotdb/db/integration/IoTDBMetadataFetchIT.java b/integration/src/test/java/org/apache/iotdb/db/integration/IoTDBMetadataFetchIT.java
index 568778ca0a..319ee2de8f 100644
--- a/integration/src/test/java/org/apache/iotdb/db/integration/IoTDBMetadataFetchIT.java
+++ b/integration/src/test/java/org/apache/iotdb/db/integration/IoTDBMetadataFetchIT.java
@@ -119,27 +119,27 @@ public class IoTDBMetadataFetchIT {
new Set[] {
new HashSet<>(
Collections.singletonList(
- "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,")),
+ "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,null,null,")),
new HashSet<>(
Arrays.asList(
- "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,",
- "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,",
- "root.ln.wf01.wt02.s1,null,root.ln.wf01.wt02,INT32,RLE,SNAPPY,null,null,",
- "root.ln.wf01.wt02.s2,null,root.ln.wf01.wt02,DOUBLE,GORILLA,SNAPPY,null,null,")),
+ "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt02.s1,null,root.ln.wf01.wt02,INT32,RLE,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt02.s2,null,root.ln.wf01.wt02,DOUBLE,GORILLA,SNAPPY,null,null,null,null,")),
new HashSet<>(
Arrays.asList(
- "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,",
- "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,")),
+ "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,null,null,")),
new HashSet<>(
Arrays.asList(
- "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,",
- "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,",
- "root.ln.wf01.wt02.s1,null,root.ln.wf01.wt02,INT32,RLE,SNAPPY,null,null,",
- "root.ln.wf01.wt02.s2,null,root.ln.wf01.wt02,DOUBLE,GORILLA,SNAPPY,null,null,",
- "root.ln1.wf01.wt01.status,null,root.ln1.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,",
- "root.ln1.wf01.wt01.temperature,null,root.ln1.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,",
- "root.ln2.wf01.wt01.status,null,root.ln2.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,",
- "root.ln2.wf01.wt01.temperature,null,root.ln2.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,")),
+ "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt02.s1,null,root.ln.wf01.wt02,INT32,RLE,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt02.s2,null,root.ln.wf01.wt02,DOUBLE,GORILLA,SNAPPY,null,null,null,null,",
+ "root.ln1.wf01.wt01.status,null,root.ln1.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,null,null,",
+ "root.ln1.wf01.wt01.temperature,null,root.ln1.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,null,null,",
+ "root.ln2.wf01.wt01.status,null,root.ln2.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,null,null,",
+ "root.ln2.wf01.wt01.temperature,null,root.ln2.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,null,null,")),
new HashSet<>()
};
for (int n = 0; n < sqls.length; n++) {
@@ -636,8 +636,8 @@ public class IoTDBMetadataFetchIT {
Set<String> standard =
new HashSet<>(
Arrays.asList(
- "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,",
- "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,"));
+ "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,null,null,"));
try {
boolean hasResultSet = statement.execute(sql);
if (hasResultSet) {
@@ -710,4 +710,48 @@ public class IoTDBMetadataFetchIT {
}
}
}
+
+ @Test
+ @Category({LocalStandaloneTest.class})
+ public void showDeadbandInfo() throws SQLException {
+ try (Connection connection = EnvFactory.getEnv().getConnection();
+ Statement statement = connection.createStatement()) {
+
+ String[] sqls =
+ new String[] {
+ "CREATE TIMESERIES root.sg1.d0.s0 WITH DATATYPE=INT32",
+ "CREATE TIMESERIES root.sg1.d0.s1 WITH DATATYPE=INT32,ENCODING=PLAIN,LOSS=SDT,COMPDEV=2",
+ "CREATE TIMESERIES root.sg1.d0.s2 WITH DATATYPE=INT32,ENCODING=PLAIN,LOSS=SDT, COMPDEV=0.01, COMPMINTIME=2, COMPMAXTIME=15"
+ };
+ for (String sql : sqls) {
+ statement.execute(sql);
+ }
+ Set<String> standard =
+ new HashSet<>(
+ Arrays.asList(
+ "root.sg1.d0.s0,null,root.sg1,INT32,RLE,SNAPPY,null,null,null,null,\n",
+ "root.sg1.d0.s1,null,root.sg1,INT32,PLAIN,SNAPPY,null,null,sdt,{compdev=2},\n",
+ "root.sg1.d0.s2,null,root.sg1,INT32,PLAIN,SNAPPY,null,null,sdt,{compdev=0.01, compmintime=2, compmaxtime=15},\n"));
+ try {
+ boolean hasResultSet = statement.execute("SHOW TIMESERIES root.sg1.d0.*");
+ if (hasResultSet) {
+ try (ResultSet resultSet = statement.getResultSet()) {
+ ResultSetMetaData resultSetMetaData = resultSet.getMetaData();
+ while (resultSet.next()) {
+ StringBuilder builder = new StringBuilder();
+ for (int i = 1; i <= resultSetMetaData.getColumnCount(); i++) {
+ builder.append(resultSet.getString(i)).append(",");
+ }
+ builder.append("\n");
+ Assert.assertTrue(standard.contains(builder.toString()));
+ }
+ }
+ }
+ } catch (SQLException e) {
+ fail(e.getMessage());
+ } finally {
+ statement.execute("delete timeseries root.sg1.d0.*");
+ }
+ }
+ }
}
diff --git a/integration/src/test/java/org/apache/iotdb/db/integration/versionadaption/IoTDBDDLVersionAdaptionIT.java b/integration/src/test/java/org/apache/iotdb/db/integration/versionadaption/IoTDBDDLVersionAdaptionIT.java
index 4929ab7ace..b9b126d2b4 100644
--- a/integration/src/test/java/org/apache/iotdb/db/integration/versionadaption/IoTDBDDLVersionAdaptionIT.java
+++ b/integration/src/test/java/org/apache/iotdb/db/integration/versionadaption/IoTDBDDLVersionAdaptionIT.java
@@ -109,27 +109,27 @@ public class IoTDBDDLVersionAdaptionIT {
new Set[] {
new HashSet<>(
Collections.singletonList(
- "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,")),
+ "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,null,null,")),
new HashSet<>(
Arrays.asList(
- "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,",
- "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,",
- "root.ln.wf01.wt02.s1,null,root.ln.wf01.wt02,INT32,RLE,SNAPPY,null,null,",
- "root.ln.wf01.wt02.s2,null,root.ln.wf01.wt02,DOUBLE,GORILLA,SNAPPY,null,null,")),
+ "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt02.s1,null,root.ln.wf01.wt02,INT32,RLE,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt02.s2,null,root.ln.wf01.wt02,DOUBLE,GORILLA,SNAPPY,null,null,null,null,")),
new HashSet<>(
Arrays.asList(
- "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,",
- "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,")),
+ "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,null,null,")),
new HashSet<>(
Arrays.asList(
- "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,",
- "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,",
- "root.ln.wf01.wt02.s1,null,root.ln.wf01.wt02,INT32,RLE,SNAPPY,null,null,",
- "root.ln.wf01.wt02.s2,null,root.ln.wf01.wt02,DOUBLE,GORILLA,SNAPPY,null,null,",
- "root.ln1.wf01.wt01.status,null,root.ln1.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,",
- "root.ln1.wf01.wt01.temperature,null,root.ln1.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,",
- "root.ln2.wf01.wt01.status,null,root.ln2.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,",
- "root.ln2.wf01.wt01.temperature,null,root.ln2.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,")),
+ "root.ln.wf01.wt01.status,null,root.ln.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt01.temperature,null,root.ln.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt02.s1,null,root.ln.wf01.wt02,INT32,RLE,SNAPPY,null,null,null,null,",
+ "root.ln.wf01.wt02.s2,null,root.ln.wf01.wt02,DOUBLE,GORILLA,SNAPPY,null,null,null,null,",
+ "root.ln1.wf01.wt01.status,null,root.ln1.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,null,null,",
+ "root.ln1.wf01.wt01.temperature,null,root.ln1.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,null,null,",
+ "root.ln2.wf01.wt01.status,null,root.ln2.wf01.wt01,BOOLEAN,PLAIN,SNAPPY,null,null,null,null,",
+ "root.ln2.wf01.wt01.temperature,null,root.ln2.wf01.wt01,FLOAT,RLE,SNAPPY,null,null,null,null,")),
new HashSet<>()
};
executeAndCheckResult(sqls, standards);
diff --git a/server/src/main/java/org/apache/iotdb/db/conf/IoTDBConstant.java b/server/src/main/java/org/apache/iotdb/db/conf/IoTDBConstant.java
index 6d09cb4ac6..c5a23bed7a 100644
--- a/server/src/main/java/org/apache/iotdb/db/conf/IoTDBConstant.java
+++ b/server/src/main/java/org/apache/iotdb/db/conf/IoTDBConstant.java
@@ -84,6 +84,8 @@ public class IoTDBConstant {
public static final String COLUMN_COUNT = "count";
public static final String COLUMN_TAGS = "tags";
public static final String COLUMN_ATTRIBUTES = "attributes";
+ public static final String COLUMN_DEADBAND = "deadband";
+ public static final String COLUMN_DEADBAND_PARAMETERS = "deadband parameters";
public static final String COLUMN_IS_ALIGNED = "isAligned";
public static final String QUERY_ID = "queryId";
public static final String STATEMENT = "statement";
@@ -148,6 +150,8 @@ public class IoTDBConstant {
public static final String SDT_COMP_DEV = "compdev";
public static final String SDT_COMP_MIN_TIME = "compmintime";
public static final String SDT_COMP_MAX_TIME = "compmaxtime";
+ public static final String[] SDT_PARAMETERS =
+ new String[] {SDT_COMP_DEV, SDT_COMP_MIN_TIME, SDT_COMP_MAX_TIME};
// data folder name
public static final String SEQUENCE_FLODER_NAME = "sequence";
diff --git a/server/src/main/java/org/apache/iotdb/db/metadata/MManager.java b/server/src/main/java/org/apache/iotdb/db/metadata/MManager.java
index 57cbe75cfe..0c18e4fda9 100644
--- a/server/src/main/java/org/apache/iotdb/db/metadata/MManager.java
+++ b/server/src/main/java/org/apache/iotdb/db/metadata/MManager.java
@@ -1372,6 +1372,8 @@ public class MManager {
Pair<Map<String, String>, Map<String, String>> tagAndAttributePair =
tagManager.readTagFile(leaf.getOffset());
IMeasurementSchema measurementSchema = leaf.getSchema();
+ Pair<String, String> deadbandInfo =
+ MetaUtils.parseDeadbandInfo(measurementSchema.getProps());
res.add(
new ShowTimeSeriesResult(
leaf.getFullPath(),
@@ -1384,7 +1386,9 @@ public class MManager {
? leaf.getLastCacheContainer().getCachedLast().getTimestamp()
: 0,
tagAndAttributePair.left,
- tagAndAttributePair.right));
+ tagAndAttributePair.right,
+ deadbandInfo.left,
+ deadbandInfo.right));
if (limit != 0) {
count++;
}
@@ -1429,7 +1433,9 @@ public class MManager {
CompressionType.valueOf(ansString.right[4]),
ansString.right[6] != null ? Long.parseLong(ansString.right[6]) : 0,
tagAndAttributePair.left,
- tagAndAttributePair.right));
+ tagAndAttributePair.right,
+ ansString.right[7],
+ ansString.right[8]));
} catch (IOException e) {
throw new MetadataException(
"Something went wrong while deserialize tag info of " + ansString.left.getFullPath(),
diff --git a/server/src/main/java/org/apache/iotdb/db/metadata/mtree/MTree.java b/server/src/main/java/org/apache/iotdb/db/metadata/mtree/MTree.java
index 8ec99e583b..81cf106b6d 100644
--- a/server/src/main/java/org/apache/iotdb/db/metadata/mtree/MTree.java
+++ b/server/src/main/java/org/apache/iotdb/db/metadata/mtree/MTree.java
@@ -1081,7 +1081,9 @@ public class MTree implements Serializable {
@Override
protected void collectMeasurement(IMeasurementMNode node) throws MetadataException {
IMeasurementSchema measurementSchema = node.getSchema();
- String[] tsRow = new String[7];
+ Pair<String, String> deadbandInfo =
+ MetaUtils.parseDeadbandInfo(measurementSchema.getProps());
+ String[] tsRow = new String[9];
tsRow[0] = node.getAlias();
tsRow[1] = getStorageGroupNodeInTraversePath().getFullPath();
tsRow[2] = measurementSchema.getType().toString();
@@ -1089,6 +1091,8 @@ public class MTree implements Serializable {
tsRow[4] = measurementSchema.getCompressor().toString();
tsRow[5] = String.valueOf(node.getOffset());
tsRow[6] = needLast ? String.valueOf(getLastTimeStamp(node, queryContext)) : null;
+ tsRow[7] = deadbandInfo.left;
+ tsRow[8] = deadbandInfo.right;
Pair<PartialPath, String[]> temp = new Pair<>(getCurrentPartialPath(node), tsRow);
result.add(temp);
}
diff --git a/server/src/main/java/org/apache/iotdb/db/metadata/utils/MetaUtils.java b/server/src/main/java/org/apache/iotdb/db/metadata/utils/MetaUtils.java
index 9b2ae8b194..2e0b09fe44 100644
--- a/server/src/main/java/org/apache/iotdb/db/metadata/utils/MetaUtils.java
+++ b/server/src/main/java/org/apache/iotdb/db/metadata/utils/MetaUtils.java
@@ -27,6 +27,7 @@ import org.apache.iotdb.db.metadata.path.MeasurementPath;
import org.apache.iotdb.db.metadata.path.PartialPath;
import org.apache.iotdb.db.utils.TestOnly;
import org.apache.iotdb.tsfile.read.common.Path;
+import org.apache.iotdb.tsfile.utils.Pair;
import java.util.ArrayList;
import java.util.Collections;
@@ -36,6 +37,9 @@ import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
+import static org.apache.iotdb.db.conf.IoTDBConstant.LOSS;
+import static org.apache.iotdb.db.conf.IoTDBConstant.SDT_PARAMETERS;
+
public class MetaUtils {
private MetaUtils() {}
@@ -260,4 +264,20 @@ public class MetaUtils {
}
return alignedPathToAggrIndexesMap;
}
+
+ public static Pair<String, String> parseDeadbandInfo(Map<String, String> props) {
+ if (props == null) {
+ return new Pair<>(null, null);
+ }
+ String deadband = props.get(LOSS);
+ Map<String, String> deadbandParameters = new HashMap<>();
+ for (String k : SDT_PARAMETERS) {
+ if (props.containsKey(k)) {
+ deadbandParameters.put(k, props.get(k));
+ }
+ }
+
+ return new Pair<>(
+ deadband, deadbandParameters.isEmpty() ? null : String.format("%s", deadbandParameters));
+ }
}
diff --git a/server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeSeriesResult.java b/server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeSeriesResult.java
index 236824d119..92a06731d2 100644
--- a/server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeSeriesResult.java
+++ b/server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeSeriesResult.java
@@ -42,6 +42,9 @@ public class ShowTimeSeriesResult extends ShowResult {
private Map<String, String> attributes;
private long lastTime;
+ private String deadband;
+ private String deadbandParameters;
+
public ShowTimeSeriesResult(
String name,
String alias,
@@ -51,7 +54,9 @@ public class ShowTimeSeriesResult extends ShowResult {
CompressionType compressor,
long lastTime,
Map<String, String> tags,
- Map<String, String> attributes) {
+ Map<String, String> attributes,
+ String deadband,
+ String deadbandParameter) {
super(name, sgName);
this.alias = alias;
this.dataType = dataType;
@@ -60,6 +65,8 @@ public class ShowTimeSeriesResult extends ShowResult {
this.tags = tags;
this.attributes = attributes;
this.lastTime = lastTime;
+ this.deadband = deadband;
+ this.deadbandParameters = deadbandParameter;
}
public ShowTimeSeriesResult() {
@@ -90,6 +97,14 @@ public class ShowTimeSeriesResult extends ShowResult {
return attributes;
}
+ public String getDeadband() {
+ return deadband;
+ }
+
+ public String getDeadbandParameters() {
+ return deadbandParameters;
+ }
+
@TestOnly
public void setTags(Map<String, String> tags) {
this.tags = tags;
diff --git a/server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeseriesDataSet.java b/server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeseriesDataSet.java
index 5f7766237f..cdb2596688 100644
--- a/server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeseriesDataSet.java
+++ b/server/src/main/java/org/apache/iotdb/db/query/dataset/ShowTimeseriesDataSet.java
@@ -35,6 +35,8 @@ import java.util.Map;
import java.util.stream.Collectors;
import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_ATTRIBUTES;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_DEADBAND;
+import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_DEADBAND_PARAMETERS;
import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_STORAGE_GROUP;
import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TAGS;
import static org.apache.iotdb.db.conf.IoTDBConstant.COLUMN_TIMESERIES;
@@ -55,7 +57,9 @@ public class ShowTimeseriesDataSet extends ShowDataSet {
new PartialPath(COLUMN_TIMESERIES_ENCODING, false),
new PartialPath(COLUMN_TIMESERIES_COMPRESSION, false),
new PartialPath(COLUMN_TAGS, false),
- new PartialPath(COLUMN_ATTRIBUTES, false)
+ new PartialPath(COLUMN_ATTRIBUTES, false),
+ new PartialPath(COLUMN_DEADBAND, false),
+ new PartialPath(COLUMN_DEADBAND_PARAMETERS, false)
};
private static final TSDataType[] resourceTypes = {
TSDataType.TEXT,
@@ -65,6 +69,8 @@ public class ShowTimeseriesDataSet extends ShowDataSet {
TSDataType.TEXT,
TSDataType.TEXT,
TSDataType.TEXT,
+ TSDataType.TEXT,
+ TSDataType.TEXT,
TSDataType.TEXT
};
@@ -92,6 +98,8 @@ public class ShowTimeseriesDataSet extends ShowDataSet {
updateRecord(record, result.getCompressor().toString());
updateRecord(record, result.getTag());
updateRecord(record, result.getAttribute());
+ updateRecord(record, result.getDeadband());
+ updateRecord(record, result.getDeadbandParameters());
records.add(record);
putRecord(record);
}
diff --git a/server/src/test/java/org/apache/iotdb/db/protocol/rest/IoTDBRestServiceIT.java b/server/src/test/java/org/apache/iotdb/db/protocol/rest/IoTDBRestServiceIT.java
index a5d67b5eb0..80d71163e5 100644
--- a/server/src/test/java/org/apache/iotdb/db/protocol/rest/IoTDBRestServiceIT.java
+++ b/server/src/test/java/org/apache/iotdb/db/protocol/rest/IoTDBRestServiceIT.java
@@ -581,6 +581,8 @@ public class IoTDBRestServiceIT {
add("compression");
add("tags");
add("attributes");
+ add("deadband");
+ add("deadband parameters");
}
};
List<Object> values1 =
@@ -640,6 +642,8 @@ public class IoTDBRestServiceIT {
add("compression");
add("tags");
add("attributes");
+ add("deadband");
+ add("deadband parameters");
}
};
List<Object> values1 =
diff --git a/server/src/test/java/org/apache/iotdb/db/query/dataset/ShowTimeSeriesResultTest.java b/server/src/test/java/org/apache/iotdb/db/query/dataset/ShowTimeSeriesResultTest.java
index 17c0248eee..78802a70b4 100644
--- a/server/src/test/java/org/apache/iotdb/db/query/dataset/ShowTimeSeriesResultTest.java
+++ b/server/src/test/java/org/apache/iotdb/db/query/dataset/ShowTimeSeriesResultTest.java
@@ -48,7 +48,9 @@ public class ShowTimeSeriesResultTest {
CompressionType.SNAPPY,
100,
tag,
- attribute);
+ attribute,
+ null,
+ "{}");
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
showTimeSeriesResult.serialize(outputStream);
diff --git a/site/src/main/.vuepress/config.js b/site/src/main/.vuepress/config.js
index 861a794a0b..2a7adbed52 100644
--- a/site/src/main/.vuepress/config.js
+++ b/site/src/main/.vuepress/config.js
@@ -677,6 +677,7 @@ var config = {
['Data-Concept/Data-Model-and-Terminology','Data Model and Terminology'],
['Data-Concept/Schema-Template','Schema Template'],
['Data-Concept/Data-Type','Data Type'],
+ ['Data-Concept/Deadband-Process','Deadband Process'],
['Data-Concept/Encoding','Encoding'],
['Data-Concept/Compression','Compression'],
['Data-Concept/Time-Partition','Time Partition'],
@@ -1597,6 +1598,7 @@ var config = {
['Data-Concept/Data-Model-and-Terminology','数据模型'],
['Data-Concept/Schema-Template','元数据模板'],
['Data-Concept/Data-Type','数据类型'],
+ ['Data-Concept/Deadband-Process','死区处理'],
['Data-Concept/Encoding','编码方式'],
['Data-Concept/Compression','压缩方式'],
['Data-Concept/Time-Partition','时间分区'],
diff --git a/zeppelin-interpreter/src/test/java/org/apache/zeppelin/iotdb/IoTDBInterpreterTest.java b/zeppelin-interpreter/src/test/java/org/apache/zeppelin/iotdb/IoTDBInterpreterTest.java
index 6948a6ac10..6eaf070ce5 100644
--- a/zeppelin-interpreter/src/test/java/org/apache/zeppelin/iotdb/IoTDBInterpreterTest.java
+++ b/zeppelin-interpreter/src/test/java/org/apache/zeppelin/iotdb/IoTDBInterpreterTest.java
@@ -315,13 +315,13 @@ public class IoTDBInterpreterTest {
public void testShowTimeseries() {
InterpreterResult actual = interpreter.internalInterpret("show timeseries", null);
String gt =
- "timeseries\talias\tstorage group\tdataType\tencoding\tcompression\ttags\tattributes\n"
- + "root.test.wf02.wt02.temperature\tnull\troot.test.wf02\tFLOAT\tGORILLA\tSNAPPY\tnull\tnull\n"
- + "root.test.wf02.wt02.status\tnull\troot.test.wf02\tBOOLEAN\tRLE\tSNAPPY\tnull\tnull\n"
- + "root.test.wf02.wt02.hardware\tnull\troot.test.wf02\tFLOAT\tGORILLA\tSNAPPY\tnull\tnull\n"
- + "root.test.wf01.wt01.temperature\tnull\troot.test.wf01\tFLOAT\tGORILLA\tSNAPPY\tnull\tnull\n"
- + "root.test.wf01.wt01.status\tnull\troot.test.wf01\tBOOLEAN\tRLE\tSNAPPY\tnull\tnull\n"
- + "root.test.wf01.wt01.hardware\tnull\troot.test.wf01\tFLOAT\tGORILLA\tSNAPPY\tnull\tnull";
+ "timeseries\talias\tstorage group\tdataType\tencoding\tcompression\ttags\tattributes\tdeadband\tdeadband parameters\n"
+ + "root.test.wf02.wt02.temperature\tnull\troot.test.wf02\tFLOAT\tGORILLA\tSNAPPY\tnull\tnull\tnull\tnull\n"
+ + "root.test.wf02.wt02.status\tnull\troot.test.wf02\tBOOLEAN\tRLE\tSNAPPY\tnull\tnull\tnull\tnull\n"
+ + "root.test.wf02.wt02.hardware\tnull\troot.test.wf02\tFLOAT\tGORILLA\tSNAPPY\tnull\tnull\tnull\tnull\n"
+ + "root.test.wf01.wt01.temperature\tnull\troot.test.wf01\tFLOAT\tGORILLA\tSNAPPY\tnull\tnull\tnull\tnull\n"
+ + "root.test.wf01.wt01.status\tnull\troot.test.wf01\tBOOLEAN\tRLE\tSNAPPY\tnull\tnull\tnull\tnull\n"
+ + "root.test.wf01.wt01.hardware\tnull\troot.test.wf01\tFLOAT\tGORILLA\tSNAPPY\tnull\tnull\tnull\tnull";
Assert.assertNotNull(actual);
Assert.assertEquals(Code.SUCCESS, actual.code());
Assert.assertEquals(gt, actual.message().get(0).getData());