You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iotdb.apache.org by qi...@apache.org on 2022/11/26 01:15:02 UTC
[iotdb] branch master updated: Update config manual (query part) (#8167)
This is an automated email from the ASF dual-hosted git repository.
qiaojialin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/iotdb.git
The following commit(s) were added to refs/heads/master by this push:
new 40fe9e08db Update config manual (query part) (#8167)
40fe9e08db is described below
commit 40fe9e08db0db2af408120f183fe7d2501d05000
Author: liuminghui233 <36...@users.noreply.github.com>
AuthorDate: Sat Nov 26 09:14:56 2022 +0800
Update config manual (query part) (#8167)
---
docs/UserGuide/Reference/Common-Config-Manual.md | 136 +++++++++------
.../zh/UserGuide/Reference/Common-Config-Manual.md | 189 +++++----------------
.../resources/conf/iotdb-common.properties | 72 +++-----
3 files changed, 150 insertions(+), 247 deletions(-)
diff --git a/docs/UserGuide/Reference/Common-Config-Manual.md b/docs/UserGuide/Reference/Common-Config-Manual.md
index 201a249059..0eaa2a975e 100644
--- a/docs/UserGuide/Reference/Common-Config-Manual.md
+++ b/docs/UserGuide/Reference/Common-Config-Manual.md
@@ -402,10 +402,54 @@ Trigger way: The client sends the command(sql) `load configuration` to the IoTDB
### Query Configurations
+* read\_consistency\_level
+
+| Name | mpp\_data\_exchange\_core\_pool\_size |
+|:-----------:|:---------------------------------------------|
+| Description | The read consistency level, </br>1. strong(Default, read from the leader replica) </br>2. weak(Read from a random replica) |
+| Type | string |
+| Default | strong |
+| Effective | After restarting system |
+
+* meta\_data\_cache\_enable
+
+|Name| meta\_data\_cache\_enable |
+|:---:|:---|
+|Description| Whether to cache meta data(BloomFilter, ChunkMetadata and TimeSeriesMetadata) or not.|
+|Type|Boolean|
+|Default| true |
+|Effective| After restarting system|
+
+* chunk\_timeseriesmeta\_free\_memory\_proportion
+
+|Name| chunk\_timeseriesmeta\_free\_memory\_proportion |
+|:---:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+|Description| Read memory Allocation Ratio: BloomFilterCache : ChunkCache : TimeSeriesMetadataCache : Coordinator : Operators : DataExchange : timeIndex in TsFileResourceList : others. |
+|Default| 1 : 100 : 200 : 300 : 400 |
+|Effective| After restarting system |
+
+* enable\_last\_cache
+
+|Name| enable\_last\_cache |
+|:---:|:---|
+|Description| Whether to enable LAST cache. |
+|Type| Boolean |
+|Default| true |
+|Effective|After restarting system|
+
+* max\_deduplicated\_path\_num
+
+|Name| max\_deduplicated\_path\_num |
+|:---:|:---|
+|Description| allowed max numbers of deduplicated path in one query. |
+|Type| Int32 |
+|Default| 1000 |
+|Effective|After restarting system|
+
* mpp\_data\_exchange\_core\_pool\_size
| Name | mpp\_data\_exchange\_core\_pool\_size |
-| :---------: | :------------------------------------------- |
+|:-----------:|:---------------------------------------------|
| Description | Core size of ThreadPool of MPP data exchange |
| Type | int32 |
| Default | 10 |
@@ -422,12 +466,12 @@ Trigger way: The client sends the command(sql) `load configuration` to the IoTDB
* mpp\_data\_exchange\_keep\_alive\_time\_in\_ms
-| Name | mpp\_data\_exchange\_keep\_alive\_time\_in\_ms |
-| :---------: | :--------------------------------------------- |
-| Description | Max waiting time for MPP data exchange |
-| Type | long |
-| Default | 1000 |
-| Effective | After restarting system |
+|Name| mpp\_data\_exchange\_keep\_alive\_time\_in\_ms |
+|:---:|:---|
+|Description| Max waiting time for MPP data exchange |
+|Type| long |
+|Default| 1000 |
+|Effective|After restarting system|
* driver\_task\_execution\_time\_slice\_in\_ms
@@ -456,41 +500,50 @@ Trigger way: The client sends the command(sql) `load configuration` to the IoTDB
| Default | 1000 |
| Effective | After restarting system |
-* default\_fill\_interval
+* slow\_query\_threshold
-| Name | default\_fill\_interval |
-| :---------: | :---------------------------------------------- |
-| Description | Default interval of `group by fill` query in ms |
-| Type | int32 |
-| Default | -1 |
-| Effective | After restarting system |
+|Name| slow\_query\_threshold |
+|:---:|:---|
+|Description| Time cost(ms) threshold for slow query. |
+|Type| Int32 |
+|Default| 5000 |
+|Effective|Trigger|
-* group\_by\_fill\_cache\_size\_in\_mb
+* query\_timeout\_threshold
-| Name | group\_by\_fill\_cache\_size\_in\_mb |
-| :---------: | :---------------------------------- |
-| Description | Cache size of `group by fill` query |
-| Type | Float |
-| Default | 1.0 |
-| Effective | After restarting system |
+|Name| query\_timeout\_threshold |
+|:---:|:---|
+|Description| The max executing time of query. unit: ms |
+|Type| Int32 |
+|Default| 60000 |
+|Effective| After restarting system|
-* coordinator\_read\_executor\_size
+* max\_allowed\_concurrent\_queries
-|Name| coordinator\_read\_executor\_size |
+|Name| max\_allowed\_concurrent\_queries |
|:---:|:---|
-|Description| The num of thread used in coordinator for query operation |
+|Description| The maximum allowed concurrently executing queries. |
|Type| Int32 |
-|Default| 50 |
-| Effective | After restarting system |
+|Default| 1000 |
+|Effective|After restarting system|
+
+* query\_thread\_count
+
+|Name| query\_thread\_count |
+|:---:|:---------------------------------------------------------------------------------------------------------------------|
+|Description| How many threads can concurrently execute query statement. When <= 0, use CPU core number. |
+|Type| Int32 |
+|Default | CPU core number |
+|Effective| After restarting system |
-* coordinator\_write\_executor\_size
+* batch\_size
-|Name| coordinator\_write\_executor\_size |
+|Name| batch\_size |
|:---:|:---|
-|Description| The num of thread used in coordinator for write operation |
+|Description| The amount of data iterate each time in server (the number of data strips, that is, the number of different timestamps.) |
|Type| Int32 |
-|Default| 50 |
-| Effective | After restarting system |
+|Default| 100000 |
+|Effective|After restarting system|
### Storage Engine Configuration
@@ -512,7 +565,6 @@ Trigger way: The client sends the command(sql) `load configuration` to the IoTDB
| Default | 1073741824 |
| Effective | when enable\_mem\_control is false & After restarting system |
-
* write\_memory\_variation\_report\_proportion
| Name | write\_memory\_variation\_report\_proportion |
@@ -594,24 +646,6 @@ Trigger way: The client sends the command(sql) `load configuration` to the IoTDB
|Default| 0 |
|Effective|After restarting system|
-* query\_thread\_count
-
-|Name| query\_thread\_count |
-|:---:|:---------------------------------------------------------------------------------------------------------------------|
-|Description| The thread number which can concurrently execute query statement. When <= 0, use CPU core number. The default is 16. |
-|Type| int32 |
-|Default| 16 |
-|Effective| After restarting system |
-
-* sub\_rawQuery\_thread\_count
-
-|Name| sub\_rawQuery\_thread\_count |
-|:---:|:-------------------------------------------------------------------------------------------------------------------------|
-|Description| The thread number which can concurrently read data for raw data query. When <= 0, use CPU core number. The default is 8. |
-|Type| int32 |
-|Default| 8 |
-|Effective| After restarting system |
-
* enable\_partial\_insert
|Name| enable\_partial\_insert |
@@ -969,8 +1003,6 @@ Trigger way: The client sends the command(sql) `load configuration` to the IoTDB
|Default| 1024 |
|Effective|hot-load|
-### Watermark Configuration
-
### Authorization Configuration
* authorizer\_provider\_class
diff --git a/docs/zh/UserGuide/Reference/Common-Config-Manual.md b/docs/zh/UserGuide/Reference/Common-Config-Manual.md
index 5a9ed1a59f..4e114d2934 100644
--- a/docs/zh/UserGuide/Reference/Common-Config-Manual.md
+++ b/docs/zh/UserGuide/Reference/Common-Config-Manual.md
@@ -457,14 +457,23 @@ IoTDB ConfigNode 和 DataNode 的通用配置参数位于 `conf` 目录下。
### 查询配置
+* read\_consistency\_level
+
+|名字| read\_consistency\_level |
+|:---:|:---|
+|描述| 查询一致性等级，取值 “strong” 时从 Leader 副本查询，取值 “weak” 时随机查询一个副本。|
+|类型| String |
+|默认值| strong |
+|改后生效方式| 重启服务生效 |
+
* meta\_data\_cache\_enable
-| 名字 | meta\_data\_cache\_enable |
-| :----------: | :------------------------------------------------------ |
-| 描述 | 是否缓存元数据Chunk Metadata 和 TimeSeries Metadata）。 |
-| 类型 | Boolean |
-| 默认值 | true |
-| 改后生效方式 | 重启服务生效 |
+|名字| meta\_data\_cache\_enable |
+|:---:|:---|
+|描述| 是否缓存元数据（包括 BloomFilter、Chunk Metadata 和 TimeSeries Metadata。）|
+|类型|Boolean|
+|默认值| true |
+|改后生效方式| 重启服务生效|
* chunk\_timeseriesmeta\_free\_memory\_proportion
@@ -484,13 +493,13 @@ IoTDB ConfigNode 和 DataNode 的通用配置参数位于 `conf` 目录下。
| 默认值 | true |
| 改后生效方式 | 重启服务生效 |
-* metadata\_node\_cache\_size
+* max\_deduplicated\_path\_num
-|名字| metadata\_node\_cache\_size |
+|名字| max\_deduplicated\_path\_num |
|:---:|:---|
-|描述| SchemaRegion的缓存大小。所有路径检查和将具有相应路径的SchemaRegion中的TSDataType的缓存，都将被用作提高写入速度。|
-|类型|int32|
-|默认值| 300000 |
+|描述| 单次查询允许的最大路径数。 |
+|类型| Int32 |
+|默认值| 1000 |
|改后生效方式|重启服务生效|
* mpp\_data\_exchange\_core\_pool\_size
@@ -520,24 +529,6 @@ IoTDB ConfigNode 和 DataNode 的通用配置参数位于 `conf` 目录下。
| 默认值 | 1000 |
| 改后生效方式 | 重启服务生效 |
-* default\_fill\_interval
-
-|名字| default\_fill\_interval |
-|:---:|:---|
-|描述| 填充查询中使用的默认时间段，默认-1表示无限过去时间，以毫秒ms为单位 |
-|类型| int32 |
-|默认值| -1 |
-|改后生效方式|重启服务生效|
-
-* group\_by\_fill\_cache\_size\_in\_mb
-
-| 名字 | group\_by\_fill\_cache\_size\_in\_mb |
-| :----------: | :--------------------------------- |
-| 描述 | 填充查询中使用的缓存大小，单位是MB |
-| 类型 | Float |
-| 默认值 | 1.0 |
-| 改后生效方式 | 重启服务生效 |
-
* driver\_task\_execution\_time\_slice\_in\_ms
| 名字 | driver\_task\_execution\_time\_slice\_in\_ms |
@@ -569,45 +560,45 @@ IoTDB ConfigNode 和 DataNode 的通用配置参数位于 `conf` 目录下。
|名字| slow\_query\_threshold |
|:---:|:---|
-|描述| 慢查询的时间成本（毫秒ms）阈值。 |
-|类型| int32 |
+|描述| 慢查询的时间阈值。单位：毫秒。|
+|类型| Int32 |
|默认值| 5000 |
|改后生效方式|热加载|
-* enable\_external\_sort
+* query\_timeout\_threshold
-| 名字 | enable\_external\_sort |
-| :----------: | :--------------------- |
-| 描述 | 是否开启外部排序功能 |
-| 类型 | Boolean |
-| 默认值 | true |
-| 改后生效方式 | 重启服务生效 |
+|名字| query\_timeout\_threshold |
+|:---:|:---|
+|描述| 查询的最大执行时间。单位：毫秒。|
+|类型| Int32 |
+|默认值| 60000 |
+|改后生效方式| 重启服务生效|
-* external\_sort\_threshold
+* max\_allowed\_concurrent\_queries
-|名字| external\_sort\_threshold |
+|名字| max\_allowed\_concurrent\_queries |
|:---:|:---|
-|描述| 单个时间序列的最大同时块读取数。若同时chunk读取的数量大于external_sort_threshold，则使用外部排序。当external_sort_threshold增加时，内存中同时排序的chunk数量可能会增加，这会占用更多的内存；external_sort_threshold 减小时，触发外部排序会增加耗时。|
-|类型| int32 |
+|描述| 允许的最大并发查询数量。 |
+|类型| Int32 |
|默认值| 1000 |
|改后生效方式|重启服务生效|
-* coordinator\_read\_executor\_size
+* query\_thread\_count
-|名字| coordinator\_read\_executor\_size |
-|:---:|:---|
-|描述| coordinator中用于执行查询操作的线程数 |
-|类型| Int32 |
-|默认值| 50 |
-|改后生效方式|重启服务生效|
+|名字| query\_thread\_count |
+|:---:|:----------------------------------------------------------------------------|
+|描述| 当 IoTDB 对内存中的数据进行查询时，最多启动多少个线程来执行该操作。如果该值小于等于 0，那么采用机器所安装的 CPU 核的数量。 |
+|类型| Int32 |
+|默认值| CPU 核数 |
+|改后生效方式| 重启服务生效 |
-* coordinator\_write\_executor\_size
+* batch\_size
-|名字| coordinator\_write\_executor\_size |
+|名字| batch\_size |
|:---:|:---|
-|描述| coordinator中用于执行写入操作的线程数 |
+|描述| 服务器中每次迭代的数据量（数据条目，即不同时间戳的数量。） |
|类型| Int32 |
-|默认值| 50 |
+|默认值| 100000 |
|改后生效方式|重启服务生效|
### 存储引擎配置
@@ -711,51 +702,6 @@ IoTDB ConfigNode 和 DataNode 的通用配置参数位于 `conf` 目录下。
|默认值| 0 |
|改后生效方式|重启服务生效|
-* query\_thread\_count
-
-|名字| query\_thread\_count |
-|:---:|:----------------------------------------------------------------------------|
-|描述| 当 IoTDB 对内存中的数据进行查询时，最多启动多少个线程来执行该操作。如果该值小于等于 0，那么采用机器所安装的 CPU 核的数量。默认值为 16。 |
-|类型| int32 |
-|默认值| 16 |
-|改后生效方式| 重启服务生效 |
-
-* sub\_rawQuery\_thread\_count
-
-|名字| sub\_rawQuery\_thread\_count |
-|:---:|:------------------------------------------------|
-|描述| 原始数据查询时，最多启动多少个线程来执行该操作。如果设置小于等于 0，会采用机器 CPU 核数 |
-|类型| int32 |
-|默认值| 8 |
-|改后生效方式| 重启服务生效 |
-
-* raw\_query\_blocking\_queue\_capacity
-
-|名字| raw\_query\_blocking\_queue\_capacity |
-|:---:|:--------------------------------------|
-|描述| 原始数据查询中，读任务的阻塞队列长度。默认值为 5 |
-|类型| int32 |
-|默认值| 5 |
-|改后生效方式| 重启服务生效 |
-
-* chunk\_buffer\_pool\_enable
-
-| 名字 | chunk\_buffer\_pool\_enable |
-| :----------: | :----------------------------------------------------------------------------------------- |
-| 描述 | 在将 memtable 序列化为内存中的字节时，是否开启由 IoTDB 而不是 JVM 接管内存管理，默认关闭。 |
-| 类型 | Boolean |
-| 默认值 | false |
-| 改后生效方式 | 重启服务生效 |
-
-* batch\_size
-
-|名字| batch\_size |
-|:---:|:---|
-|描述| 服务器中每次迭代的数据量（数据条目，即不同时间戳的数量。） |
-|类型| int32 |
-|默认值| 100000 |
-|改后生效方式|重启服务生效|
-
* enable\_partial\_insert
| 名字 | enable\_partial\_insert |
@@ -992,15 +938,6 @@ IoTDB ConfigNode 和 DataNode 的通用配置参数位于 `conf` 目录下。
|默认值| 16 |
|改后生效方式| 重启服务生效|
-* query\_timeout\_threshold
-
-|名字| query\_timeout\_threshold |
-|:---:|:---|
-|描述| 查询的最大执行时间。单位：毫秒。|
-|类型| int32 |
-|默认值| 60000 |
-|改后生效方式| 重启服务生效|
-
* sub\_compaction\_thread\_count
|名字| sub\_compaction\_thread\_count |
@@ -1131,44 +1068,6 @@ IoTDB ConfigNode 和 DataNode 的通用配置参数位于 `conf` 目录下。
|默认值| 1024 |
|改后生效方式|热加载|
-### 水印模块配置
-
-* watermark\_module\_opened
-
-| 名字 | watermark\_module\_opened |
-| :----------: | :------------------------ |
-| 描述 | 是否开启水印水印嵌入功能 |
-| 取值 | true or false |
-| 默认值 | false |
-| 改后生效方式 | 重启服务生效 |
-
-* watermark\_secret\_key
-
-| 名字 | watermark\_secret\_key |
-| :----------: | :--------------------- |
-| 描述 | 水印嵌入功能秘钥 |
-| 类型 | String |
-| 默认值 | IoTDB * 2019@Beijing |
-| 改后生效方式 | 重启服务生效 |
-
-* watermark\_bit\_string
-
-|名字| watermark\_bit\_string |
-|:---:|:---|
-|描述| 水印比特字符串 |
-|类型| int32 |
-|默认值| 100101110100 |
-|改后生效方式|重启服务生效|
-
-* watermark\_method
-
-| 名字 | watermark\_method |
-| :----------: | :----------------------------------------------------- |
-| 描述 | 水印嵌入方法 |
-| 类型 | String |
-| 默认值 | GroupBasedLSBMethod(embed_row_cycle=2,embed_lsb_num=5) |
-| 改后生效方式 | 重启服务生效 |
-
### 授权配置
* authorizer\_provider\_class
diff --git a/node-commons/src/assembly/resources/conf/iotdb-common.properties b/node-commons/src/assembly/resources/conf/iotdb-common.properties
index bba5ad8ec3..78d66087c4 100644
--- a/node-commons/src/assembly/resources/conf/iotdb-common.properties
+++ b/node-commons/src/assembly/resources/conf/iotdb-common.properties
@@ -328,7 +328,7 @@
# Datatype: string
# read_consistency_level=strong
-# whether to cache meta data(BloomFilter, ChunkMetadata and TimeSeriesMetadata) or not.
+# Whether to cache meta data(BloomFilter, ChunkMetadata and TimeSeriesMetadata) or not.
# Datatype: boolean
# meta_data_cache_enable=true
@@ -340,53 +340,54 @@
# Datatype: boolean
# enable_last_cache=true
-# allowed max numbers of deduplicated path in one query
+# Allowed max numbers of deduplicated path in one query
# it's just an advised value, the real limitation will be the smaller one between this and the one we calculated
# Datatype: int
# max_deduplicated_path_num=1000
+# Core size of ThreadPool of MPP data exchange
# Datatype: int
# mpp_data_exchange_core_pool_size=10
+# Max size of ThreadPool of MPP data exchange
# Datatype: int
# mpp_data_exchange_max_pool_size=10
+# Max waiting time for MPP data exchange
# Datatype: int
# mpp_data_exchange_keep_alive_time_in_ms=1000
-# the default time period that used in fill query, -1 by default means infinite past time
-# Datatype: int, Unit: ms
-# default_fill_interval=-1
-
-# Datatype: float
-# group_by_fill_cache_size_in_mb=1.0
-
-# the max execution time of a DriverTask
+# The max execution time of a DriverTask
# Datatype: int, Unit: ms
# driver_task_execution_time_slice_in_ms=100
-# the max capacity of a TsBlock
+# The max capacity of a TsBlock
# Datatype: int, Unit: byte
# max_tsblock_size_in_bytes=1048576
-# the max number of lines in a single TsBlock
+# The max number of lines in a single TsBlock
# Datatype: int
# max_tsblock_line_numbers=1000
-# time cost(ms) threshold for slow query
+# Time cost(ms) threshold for slow query
# Datatype: long
# slow_query_threshold=5000
-# Is external sort enable
-# Datatype: boolean
-# enable_external_sort=true
+# The max executing time of query. unit: ms
+# Datatype: int
+# query_timeout_threshold=60000
+
+# The maximum allowed concurrently executing queries
+# Datatype: int
+# max_allowed_concurrent_queries=1000
+
+# How many threads can concurrently execute query statement. When <= 0, use CPU core number.
+# Datatype: int
+# query_thread_count=0
-# The maximum number of simultaneous chunk reading for a single time series.
-# If the num of simultaneous chunk reading is greater than external_sort_threshold, external sorting is used.
-# When external_sort_threshold increases, the number of chunks sorted at the same time in memory may increase and this will occupy more memory.
-# When external_sort_threshold decreases, triggering external sorting will increase the time-consuming.
+# The amount of data iterate each time in server (the number of data strips, that is, the number of different timestamps.)
# Datatype: int
-# external_sort_threshold=1000
+# batch_size=100000
####################
### Storage Engine Configuration
@@ -471,27 +472,6 @@
# Datatype: int
# flush_thread_count=0
-# How many threads can concurrently execute query statement. When <= 0, use CPU core number.
-# Datatype: int
-# query_thread_count=0
-
-# How many threads can concurrently read data for raw data query. When <= 0, use CPU core number.
-# Datatype: int
-# sub_rawQuery_thread_count=8
-
-# Blocking queue size for read task in raw data query. Must >= 1.
-# Datatype: int
-# raw_query_blocking_queue_capacity=5
-
-# whether take over the memory management by IoTDB rather than JVM when serializing memtable as bytes in memory
-# (i.e., whether use ChunkBufferPool), value true, false
-# Datatype: boolean
-# chunk_buffer_pool_enable=false
-
-# The amount of data iterate each time in server (the number of data strips, that is, the number of different timestamps.)
-# Datatype: int
-# batch_size=100000
-
# In one insert (one device, one timestamp, multiple measurements),
# if enable partial insert, one measurement failure will not impact other measurements
# Datatype: boolean
@@ -609,14 +589,6 @@
# Datatype: int
# compaction_write_throughput_mb_per_sec=16
-# The max executing time of query. unit: ms
-# Datatype: int
-# query_timeout_threshold=60000
-
-# The maximum allowed concurrently executing queries
-# Datatype: int
-# max_allowed_concurrent_queries=1000
-
# The number of sub compaction threads to be set up to perform compaction.
# Currently only works for nonAligned data in cross space compaction and unseq inner space compaction.
# Set to 1 when less than or equal to 0.