You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iotdb.apache.org by ha...@apache.org on 2022/10/25 03:52:02 UTC
[iotdb] branch master updated: Update IoTDB nifi process doc (#7637)
This is an automated email from the ASF dual-hosted git repository.
haonan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/iotdb.git
The following commit(s) were added to refs/heads/master by this push:
new b0263f0c57 Update IoTDB nifi process doc (#7637)
b0263f0c57 is described below
commit b0263f0c5704f6421312258510f93901e25ade24
Author: Zhizhou Li <lz...@meteroi.com>
AuthorDate: Tue Oct 25 11:51:57 2022 +0800
Update IoTDB nifi process doc (#7637)
---
docs/UserGuide/Ecosystem-Integration/NiFi-IoTDB.md | 56 ++++++++++++++++------
.../UserGuide/Ecosystem-Integration/NiFi-IoTDB.md | 53 ++++++++++++++------
2 files changed, 80 insertions(+), 29 deletions(-)
diff --git a/docs/UserGuide/Ecosystem-Integration/NiFi-IoTDB.md b/docs/UserGuide/Ecosystem-Integration/NiFi-IoTDB.md
index 5e0c92bdc6..4f973a4fd9 100644
--- a/docs/UserGuide/Ecosystem-Integration/NiFi-IoTDB.md
+++ b/docs/UserGuide/Ecosystem-Integration/NiFi-IoTDB.md
@@ -46,29 +46,30 @@ Apache NiFi includes the following capabilities:
* Multi-tenant authorization and policy management
* Standard protocols for encrypted communication including TLS and SSH
-## PutIoTDB
+## PutIoTDBRecord
This is a processor that reads the content of the incoming FlowFile as individual records using the configured 'Record Reader' and writes them to Apache IoTDB using native interface.
-### Properties of PutIoTDB
+### Properties of PutIoTDBRecord
-| property | description | default value | necessary |
-| ------------- | ------------------------------------------------------------ | ------------- | --------- |
-| Host | The host of IoTDB. | null | true |
-| Port | The port of IoTDB. | 6667 | true |
-| Username | Username to access the IoTDB. | null | true |
-| Password | Password to access the IoTDB. | null | true |
-| Record Reader | Specifies the type of Record Reader controller service to use <br />for parsing the incoming data and determining the schema. | null | true |
+| property | description | default value | necessary |
+|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| ------------- | --------- |
+| Host | The host of IoTDB. | null | true |
+| Port | The port of IoTDB. | 6667 | true |
+| Username | Username to access the IoTDB. | null | true |
+| Password | Password to access the IoTDB. | null | true |
+| Prefix | The Prefix begin with root. that will be add to the tsName in data. <br /> It can be updated by expression language. | null | true |
+| Record Reader | Specifies the type of Record Reader controller service to use <br />for parsing the incoming data and determining the schema. | null | true |
| Schema | The schema that IoTDB needs doesn't support good by NiFi.<br/>Therefore, you can define the schema here. <br />Besides, you can set encoding type and compression type by this method.<br />If you don't set this property, the inferred schema will be used.<br /> It can be updated by expression language. | null | false |
-| Aligned | Whether using aligned interface? It can be updated by expression language. | false | false |
-| MaxRowNumber | Specifies the max row number of each tablet. It can be updated by expression language. | 1024 | false |
+| Aligned | Whether using aligned interface? It can be updated by expression language. | false | false |
+| MaxRowNumber | Specifies the max row number of each tablet. It can be updated by expression language. | 1024 | false |
### Inferred Schema of Flowfile
There are a couple of rules about flowfile:
1. The flowfile can be read by `Record Reader`.
-2. The schema of flowfile must contains a field `Time`, and it must be the first.
+2. The schema of flowfile must contain a field `Time`, and it must be the first.
3. The data type of time must be `STRING` or `LONG`.
4. Fields excepted time must start with `root.`.
5. The supported data types are `INT`, `LONG`, `FLOAT`, `DOUBLE`, `BOOLEAN`, `TEXT`.
@@ -83,12 +84,12 @@ The structure of property `Schema`:
{
"timeType": "LONG",
"fields": [{
- "tsName": "root.sg.d1.s1",
+ "tsName": "s1",
"dataType": "INT32",
"encoding": "RLE",
"compressionType": "GZIP"
}, {
- "tsName": "root.sg.d1.s2",
+ "tsName": "s2",
"dataType": "INT64",
"encoding": "RLE",
"compressionType": "GZIP"
@@ -102,7 +103,7 @@ The structure of property `Schema`:
1. The JSON of schema must contain `timeType` and `fields`.
2. There are only two options `LONG` and `STRING` for `timeType`.
3. The columns `tsName` and `dataType` must be set.
-4. The tsName must start with `root.`.
+4. The property `Prefix` will be added to tsName as the field name when add data to IoTDB.
5. The supported `dataTypes` are `INT32`, `INT64`, `FLOAT`, `DOUBLE`, `BOOLEAN`, `TEXT`.
6. The supported `encoding` are `PLAIN`, `DICTIONARY`, `RLE`, `DIFF`, `TS_2DIFF`, `BITMAP`, `GORILLA_V1`, `REGULAR`, `GORILLA`.
7. The supported `compressionType` are `UNCOMPRESSED`, `SNAPPY`, `GZIP`, `LZO`, `SDT`, `PAA`, `PLA`, `LZ4`.
@@ -113,3 +114,28 @@ The structure of property `Schema`:
| ------------ | ---------------------------------------------------- |
| success | Data can be written correctly or flow file is empty. |
| failure | The shema or flow file is abnormal. |
+
+
+## QueryIoTDBRecord
+
+This is a processor that reads the sql query from the incoming FlowFile and using it to query the result from IoTDB using native interface. Then it use the configured 'Record Writer' to generate the flowfile
+
+### Properties of QueryIoTDBRecord
+
+| property | description | default value | necessary |
+|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------| --------- |
+| Host | The host of IoTDB. | null | true |
+| Port | The port of IoTDB. | 6667 | true |
+| Username | Username to access the IoTDB. | null | true |
+| Password | Password to access the IoTDB. | null | true |
+| Record Writer | Specifies the Controller Service to use for writing results to a FlowFile. The Record Writer may use Inherit Schema to emulate the inferred schema behavior, i.e. An explicit schema need not be defined in the writer, and will be supplied by the same logic used to infer the schema from the column types. | null | true |
+| iotdb-query | The IoTDB query to execute. <bbr> Note: If there are incoming connections, then the query is created from incoming FlowFile's content otherwise"it is created from this property. | null | false |
+| iotdb-query-chunk-size | Chunking can be used to return results in a stream of smaller batches (each has a partial results up to a chunk size) rather than as a single response. Chunking queries can return an unlimited number of rows. Note: Chunking is enable when result chunk size is greater than 0 | 0 | false |
+
+
+## Relationships
+
+| relationship | description |
+| ------------ | ---------------------------------------------------- |
+| success | Data can be written correctly or flow file is empty. |
+| failure | The shema or flow file is abnormal. |
\ No newline at end of file
diff --git a/docs/zh/UserGuide/Ecosystem-Integration/NiFi-IoTDB.md b/docs/zh/UserGuide/Ecosystem-Integration/NiFi-IoTDB.md
index 372642e65a..686aa96a8c 100644
--- a/docs/zh/UserGuide/Ecosystem-Integration/NiFi-IoTDB.md
+++ b/docs/zh/UserGuide/Ecosystem-Integration/NiFi-IoTDB.md
@@ -46,22 +46,23 @@ Apache NiFi 包含以下功能：
* 多租户授权和策略管理
* 包括TLS和SSH的加密通信的标准协议
-## PutIoTDB
+## PutIoTDBRecord
这是一个用于数据写入的处理器。它使用配置的 Record Reader 将传入 FlowFile 的内容读取为单独的记录，并使用本机接口将它们写入 Apache IoTDB。
-### PutIoTDB的配置项
+### PutIoTDBRecord的配置项
-| 配置项 | 描述 | 默认值 | 是否必填 |
-| ------------- | ------------------------------------------------------------ | ------ | -------- |
-| Host | IoTDB 的主机名 | null | true |
-| Port | IoTDB 的端口 | 6667 | true |
-| Username | IoTDB 的用户名 | null | true |
-| Password | IoTDB 的密码 | null | true |
-| Record Reader | 指定一个 Record Reader controller service 来解析数据，并且推断数据格式。 | null | true |
+| 配置项 | 描述 | 默认值 | 是否必填 |
+| ------------- |---------------------------------------------------------------------------------------------------------------------------------------------------------------| ------ | -------- |
+| Host | IoTDB 的主机名 | null | true |
+| Port | IoTDB 的端口 | 6667 | true |
+| Username | IoTDB 的用户名 | null | true |
+| Password | IoTDB 的密码 | null | true |
+| Prefix | 将被写入IoTDB的数据的tsName前缀 以root. 开头 <br /> 可以使用Nifi expression language做动态替换. | null | true |
+| Record Reader | 指定一个 Record Reader controller service 来解析数据，并且推断数据格式。 | null | true |
| Schema | IoTDB 需要的 schema 不能很好的被 NiFi 支持，因此你可以在这里自定义 schema。<br />除此之外，你可以通过这个方式设置编码和压缩类型。如果你没有设置这个配置，就会使用 Record Reader 推断的 schema。<br />这个配置可以通过 Attributes 的表达式来更新。 | null | false |
-| Aligned | 是否使用 aligned 接口？<br />这个配置可以通过 Attributes 的表达式来更新。 | false | false |
-| MaxRowNumber | 指定 tablet 的最大行数。<br />这个配置可以通过 Attributes 的表达式来更新。 | 1024 | false |
+| Aligned | 是否使用 aligned 接口？<br />这个配置可以通过 Attributes 的表达式来更新。 | false | false |
+| MaxRowNumber | 指定 tablet 的最大行数。<br />这个配置可以通过 Attributes 的表达式来更新。 | 1024 | false |
### Flowfile 的推断数据类型
@@ -83,12 +84,12 @@ Apache NiFi 包含以下功能：
{
"timeType": "LONG",
"fields": [{
- "tsName": "root.sg.d1.s1",
+ "tsName": "s1",
"dataType": "INT32",
"encoding": "RLE",
"compressionType": "GZIP"
}, {
- "tsName": "root.sg.d1.s2",
+ "tsName": "s2",
"dataType": "INT64",
"encoding": "RLE",
"compressionType": "GZIP"
@@ -102,11 +103,35 @@ Apache NiFi 包含以下功能：
1. 定义 shema 的 JSON 中必须包含 `timeType` and `fields` 这两项。
2. `timeType` 只支持 `LONG` 和 `STRING` 这两个选项。
3. `tsName` 和 `dataType` 这两项必须被设置。
-4. `tsName` 必须以 `root.` 开头。
+4. 当数据插入IoTDB时，Prefix属性会被添加到 tsName以作为插入的字段名。
5. 支持的 `dataTypes` 有：`INT32`， `INT64`， `FLOAT`， `DOUBLE`， `BOOLEAN`， `TEXT`。
6. 支持的 `encoding` 有： `PLAIN`， `DICTIONARY`， `RLE`， `DIFF`， `TS_2DIFF`， `BITMAP`， `GORILLA_V1`， `REGULAR`， `GORILLA`。
7. 支持的 `compressionType` 有： `UNCOMPRESSED`， `SNAPPY`， `GZIP`， `LZO`， `SDT`， `PAA`， `PLA`， `LZ4`。
+## Relationships
+
+| relationship | 描述 |
+| ------------ | ----------------------- |
+| success | 数据能被正确的写入。 |
+| failure | schema 或者数据有异常。 |
+
+## QueryIoTDBRecord
+
+这是一个用于数据读取的处理器。它通过读取 FlowFile 的内容中的SQL 查询来对IoTDB的原生接口进行访问，并将查询结果用Record Writer写入 flowfile。
+
+### QueryIoTDBRecord的配置项
+
+| 配置项 | 描述 | 默认值 | 是否必填 |
+| ------------- |--------------------------------------------------------------------------------| ------ | -------- |
+| Host | IoTDB 的主机名 | null | true |
+| Port | IoTDB 的端口 | 6667 | true |
+| Username | IoTDB 的用户名 | null | true |
+| Password | IoTDB 的密码 | null | true |
+| Record Writer | 指定一个 Record Writer controller service 来写入数据。 | null | true |
+| iotdb-query | 需要执行的IoTDB query <bbr>。 Note: 如果有连入侧的连接那么查询会从FlowFile的内容中提取，否则使用当前配置的属性 | null | false |
+| iotdb-query-chunk-size | 返回的结果可以进行分块，数据流中会返回一批按设置大小切分的数据，而不是一个单一的响应. 分块查询可以返回无限量的行。 注意: 数据分块只有在设置不为0时启用 | 0 | false |
+
+
## Relationships
| relationship | 描述 |