You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iotdb.apache.org by qi...@apache.org on 2022/07/30 06:20:25 UTC
[iotdb] branch rel/0.13 updated: [To rel/0.13]Add default schema query fetch size config to 10000000 (#6844)
This is an automated email from the ASF dual-hosted git repository.
qiaojialin pushed a commit to branch rel/0.13
in repository https://gitbox.apache.org/repos/asf/iotdb.git
The following commit(s) were added to refs/heads/rel/0.13 by this push:
new c14219d964 [To rel/0.13]Add default schema query fetch size config to 10000000 (#6844)
c14219d964 is described below
commit c14219d964a2715a09459bc893a8eb9842522574
Author: Marcos_Zyk <38...@users.noreply.github.com>
AuthorDate: Sat Jul 30 14:20:20 2022 +0800
[To rel/0.13]Add default schema query fetch size config to 10000000 (#6844)
---
docs/UserGuide/Operate-Metadata/Node.md | 6 +++---
docs/UserGuide/Operate-Metadata/Timeseries.md | 2 +-
docs/UserGuide/Reference/Config-Manual.md | 9 +++++++++
docs/zh/UserGuide/Operate-Metadata/Node.md | 6 +++---
docs/zh/UserGuide/Operate-Metadata/Timeseries.md | 2 +-
docs/zh/UserGuide/Reference/Config-Manual.md | 9 +++++++++
server/src/assembly/resources/conf/iotdb-engine.properties | 4 ++++
.../src/main/java/org/apache/iotdb/db/conf/IoTDBConfig.java | 11 +++++++++++
.../main/java/org/apache/iotdb/db/conf/IoTDBDescriptor.java | 5 +++++
.../iotdb/db/qp/logical/sys/ShowChildNodesOperator.java | 3 ++-
.../iotdb/db/qp/logical/sys/ShowChildPathsOperator.java | 3 ++-
.../apache/iotdb/db/qp/logical/sys/ShowDevicesOperator.java | 3 ++-
.../iotdb/db/qp/logical/sys/ShowTimeSeriesOperator.java | 3 ++-
13 files changed, 54 insertions(+), 12 deletions(-)
diff --git a/docs/UserGuide/Operate-Metadata/Node.md b/docs/UserGuide/Operate-Metadata/Node.md
index b5710b501f..baa0a6ed6d 100644
--- a/docs/UserGuide/Operate-Metadata/Node.md
+++ b/docs/UserGuide/Operate-Metadata/Node.md
@@ -28,7 +28,7 @@ SHOW CHILD PATHS pathPattern ? limitClause
Return all child paths of all the paths matching pathPattern.
-The default size of result set is 1000. To query more child paths, please use ```limit``` and ```offset```.
+The default size of result set is 10000000. To query more child paths, please use ```limit``` and ```offset```.
Example：
@@ -55,7 +55,7 @@ SHOW CHILD NODES pathPattern ? limitClause
Return all child nodes of the pathPattern.
-The default size of result set is 1000. To query more child paths, please use ```limit``` and ```offset```.
+The default size of result set is 10000000. To query more child paths, please use ```limit``` and ```offset```.
Example：
@@ -137,7 +137,7 @@ It costs 0.002s
* SHOW DEVICES pathPattern? (WITH STORAGE GROUP)? limitClause? #showDevices
-The default size of result set is 1000. To query more child paths, please use ```limit``` and ```offset```.
+The default size of result set is 10000000. To query more child paths, please use ```limit``` and ```offset```.
Similar to `Show Timeseries`, IoTDB also supports two ways of viewing devices:
diff --git a/docs/UserGuide/Operate-Metadata/Timeseries.md b/docs/UserGuide/Operate-Metadata/Timeseries.md
index 87c168a3ce..fcbb49b6e4 100644
--- a/docs/UserGuide/Operate-Metadata/Timeseries.md
+++ b/docs/UserGuide/Operate-Metadata/Timeseries.md
@@ -86,7 +86,7 @@ IoTDB> delete timeseries root.ln.wf02.*
Timeseries information includes: timeseries path, alias of measurement, storage group it belongs to, data type, encoding type, compression type, tags and attributes.
-The default size of result set is 1000. To query more child paths, please use ```limit``` and ```offset```.
+The default size of result set is 10000000. To query more child paths, please use ```limit``` and ```offset```.
Examples:
diff --git a/docs/UserGuide/Reference/Config-Manual.md b/docs/UserGuide/Reference/Config-Manual.md
index fece0eb11d..cdf3c846bc 100644
--- a/docs/UserGuide/Reference/Config-Manual.md
+++ b/docs/UserGuide/Reference/Config-Manual.md
@@ -526,6 +526,15 @@ The permission definitions are in ${IOTDB\_CONF}/conf/jmx.access.
|Default| 10000 |
|Effective|After restarting system|
+* schema\_query\_fetch\_size
+
+|Name| schema\_query\_fetch\_size |
+|:---:|:---|
+|Description| The default max size amount of schema record returned in one schema query(e.g. show timeseries). |
+|Type|Int32|
+|Default| 10000000 |
+|Effective|After restarting system|
+
* enable\_stat\_monitor
|Name| enable\_stat\_monitor |
diff --git a/docs/zh/UserGuide/Operate-Metadata/Node.md b/docs/zh/UserGuide/Operate-Metadata/Node.md
index 02e907bb26..aa9a1a494c 100644
--- a/docs/zh/UserGuide/Operate-Metadata/Node.md
+++ b/docs/zh/UserGuide/Operate-Metadata/Node.md
@@ -29,7 +29,7 @@ SHOW CHILD PATHS pathPattern ? limitClause
可以查看此路径模式所匹配的所有路径的下一层的所有路径，即pathPattern.*所匹配的路径。
-查询结果集的大小默认为1000，如需查询更多信息，请使用```limit```和```offset```。
+查询结果集的大小默认为 10000000，如需查询更多信息，请使用```limit```和```offset```。
示例：
@@ -63,7 +63,7 @@ SHOW CHILD NODES pathPattern ? limitClause
可以查看此路径模式所匹配的节点的下一层的所有节点。
-查询结果集的大小默认为1000，如需查询更多信息，请使用```limit```和```offset```。
+查询结果集的大小默认为 10000000，如需查询更多信息，请使用```limit```和```offset```。
示例：
@@ -142,7 +142,7 @@ It costs 0.002s
* SHOW DEVICES pathPattern? (WITH STORAGE GROUP)? limitClause? #showDevices
-查询结果集的大小默认为1000，如需查询更多信息，请使用```limit```和```offset```。
+查询结果集的大小默认为 10000000，如需查询更多信息，请使用```limit```和```offset```。
与 `Show Timeseries` 相似，IoTDB 目前也支持两种方式查看设备。
diff --git a/docs/zh/UserGuide/Operate-Metadata/Timeseries.md b/docs/zh/UserGuide/Operate-Metadata/Timeseries.md
index 014c98fddf..7a91170b3b 100644
--- a/docs/zh/UserGuide/Operate-Metadata/Timeseries.md
+++ b/docs/zh/UserGuide/Operate-Metadata/Timeseries.md
@@ -83,7 +83,7 @@ IoTDB> delete timeseries root.ln.wf02.*
时间序列信息具体包括：时间序列路径名，存储组，Measurement 别名，数据类型，编码方式，压缩方式，属性和标签。
-查询结果集的大小默认为1000，如需查询更多信息，请使用```limit```和```offset```。
+查询结果集的大小默认为 10000000，如需查询更多信息，请使用```limit```和```offset```。
示例：
diff --git a/docs/zh/UserGuide/Reference/Config-Manual.md b/docs/zh/UserGuide/Reference/Config-Manual.md
index 33171bc0e6..2c7907d897 100644
--- a/docs/zh/UserGuide/Reference/Config-Manual.md
+++ b/docs/zh/UserGuide/Reference/Config-Manual.md
@@ -748,6 +748,15 @@ Server，客户端的使用方式详见 [SQL 命令行终端（CLI）](https://i
|默认值| 10 |
|改后生效方式|重启服务生效|
+* schema\_query\_fetch\_size
+
+|名字| schema\_query\_fetch\_size |
+|:---:|:---|
+|描述| 默认的一次元数据查询返回的最大条目数，例如show timeseries。|
+|类型|Int32|
+|默认值| 10000000 |
+|改后生效方式|重启服务生效|
+
### 升级配置
* update\_thread\_num
diff --git a/server/src/assembly/resources/conf/iotdb-engine.properties b/server/src/assembly/resources/conf/iotdb-engine.properties
index 840a3b6ad0..290b685ca9 100644
--- a/server/src/assembly/resources/conf/iotdb-engine.properties
+++ b/server/src/assembly/resources/conf/iotdb-engine.properties
@@ -395,6 +395,10 @@ timestamp_precision=ms
# Datatype: int
# io_task_queue_size_for_flushing=10
+# Size of schema query data set. The record number returned by one schema query won't be larger than value of this param.
+# Datatype: int
+# schema_query_fetch_size=10000000
+
####################
### Upgrade Configurations
####################
diff --git a/server/src/main/java/org/apache/iotdb/db/conf/IoTDBConfig.java b/server/src/main/java/org/apache/iotdb/db/conf/IoTDBConfig.java
index 16c7151e95..4cea02fa93 100644
--- a/server/src/main/java/org/apache/iotdb/db/conf/IoTDBConfig.java
+++ b/server/src/main/java/org/apache/iotdb/db/conf/IoTDBConfig.java
@@ -848,6 +848,9 @@ public class IoTDBConfig {
// OperationSyncConsumer concurrency size
private int operationSyncConsumerConcurrencySize = 4;
+ // The max record num returned in one schema query.
+ private int schemaQueryFetchSize = 10000000;
+
public IoTDBConfig() {
// empty constructor
}
@@ -2714,4 +2717,12 @@ public class IoTDBConfig {
public void setOperationSyncConsumerConcurrencySize(int operationSyncConsumerConcurrencySize) {
this.operationSyncConsumerConcurrencySize = operationSyncConsumerConcurrencySize;
}
+
+ public int getSchemaQueryFetchSize() {
+ return schemaQueryFetchSize;
+ }
+
+ public void setSchemaQueryFetchSize(int schemaQueryFetchSize) {
+ this.schemaQueryFetchSize = schemaQueryFetchSize;
+ }
}
diff --git a/server/src/main/java/org/apache/iotdb/db/conf/IoTDBDescriptor.java b/server/src/main/java/org/apache/iotdb/db/conf/IoTDBDescriptor.java
index 0aff2ac6e1..c277e9bc6a 100644
--- a/server/src/main/java/org/apache/iotdb/db/conf/IoTDBDescriptor.java
+++ b/server/src/main/java/org/apache/iotdb/db/conf/IoTDBDescriptor.java
@@ -845,6 +845,11 @@ public class IoTDBDescriptor {
"operation_sync_consumer_concurrency_size",
String.valueOf(conf.getOperationSyncConsumerConcurrencySize()))));
+ conf.setSchemaQueryFetchSize(
+ Integer.parseInt(
+ properties.getProperty(
+ "schema_query_fetch_size", String.valueOf(conf.getSchemaQueryFetchSize()))));
+
// At the same time, set TSFileConfig
TSFileDescriptor.getInstance()
.getConfig()
diff --git a/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowChildNodesOperator.java b/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowChildNodesOperator.java
index 4ba24da3e7..e6d645fbff 100644
--- a/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowChildNodesOperator.java
+++ b/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowChildNodesOperator.java
@@ -19,6 +19,7 @@
*/
package org.apache.iotdb.db.qp.logical.sys;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
import org.apache.iotdb.db.exception.query.QueryProcessException;
import org.apache.iotdb.db.metadata.path.PartialPath;
import org.apache.iotdb.db.qp.physical.PhysicalPlan;
@@ -30,7 +31,7 @@ public class ShowChildNodesOperator extends ShowOperator {
private PartialPath path;
- private int limit = 1000;
+ private int limit = IoTDBDescriptor.getInstance().getConfig().getSchemaQueryFetchSize();
private int offset = 0;
public ShowChildNodesOperator(int tokenIntType, PartialPath path) {
diff --git a/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowChildPathsOperator.java b/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowChildPathsOperator.java
index da96dac480..3fd0214153 100644
--- a/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowChildPathsOperator.java
+++ b/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowChildPathsOperator.java
@@ -19,6 +19,7 @@
*/
package org.apache.iotdb.db.qp.logical.sys;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
import org.apache.iotdb.db.exception.query.QueryProcessException;
import org.apache.iotdb.db.metadata.path.PartialPath;
import org.apache.iotdb.db.qp.physical.PhysicalPlan;
@@ -30,7 +31,7 @@ public class ShowChildPathsOperator extends ShowOperator {
private PartialPath path;
- private int limit = 1000;
+ private int limit = IoTDBDescriptor.getInstance().getConfig().getSchemaQueryFetchSize();
private int offset = 0;
public ShowChildPathsOperator(int tokenIntType, PartialPath path) {
diff --git a/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowDevicesOperator.java b/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowDevicesOperator.java
index df1a24e87e..95a22399d6 100644
--- a/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowDevicesOperator.java
+++ b/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowDevicesOperator.java
@@ -19,6 +19,7 @@
*/
package org.apache.iotdb.db.qp.logical.sys;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
import org.apache.iotdb.db.exception.query.QueryProcessException;
import org.apache.iotdb.db.metadata.path.PartialPath;
import org.apache.iotdb.db.qp.physical.PhysicalPlan;
@@ -28,7 +29,7 @@ import org.apache.iotdb.db.qp.strategy.PhysicalGenerator;
public class ShowDevicesOperator extends ShowOperator {
private PartialPath path;
- private int limit = 1000;
+ private int limit = IoTDBDescriptor.getInstance().getConfig().getSchemaQueryFetchSize();
private int offset = 0;
private boolean hasSgCol;
diff --git a/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowTimeSeriesOperator.java b/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowTimeSeriesOperator.java
index 102fb078aa..dc97144f5f 100644
--- a/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowTimeSeriesOperator.java
+++ b/server/src/main/java/org/apache/iotdb/db/qp/logical/sys/ShowTimeSeriesOperator.java
@@ -19,6 +19,7 @@
*/
package org.apache.iotdb.db.qp.logical.sys;
+import org.apache.iotdb.db.conf.IoTDBDescriptor;
import org.apache.iotdb.db.exception.query.QueryProcessException;
import org.apache.iotdb.db.metadata.path.PartialPath;
import org.apache.iotdb.db.qp.physical.PhysicalPlan;
@@ -31,7 +32,7 @@ public class ShowTimeSeriesOperator extends ShowOperator {
private boolean isContains;
private String key;
private String value;
- private int limit = 1000;
+ private int limit = IoTDBDescriptor.getInstance().getConfig().getSchemaQueryFetchSize();
private int offset = 0;
// if is true, the result will be sorted according to the inserting frequency of the timeseries
private final boolean orderByHeat;