You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Xuan Ronaldo (Jira)" <ji...@apache.org> on 2022/07/25 07:42:00 UTC
[jira] [Comment Edited] (NIFI-10234) implement a processor of Apache IoTDB
[ https://issues.apache.org/jira/browse/NIFI-10234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570606#comment-17570606 ]
Xuan Ronaldo edited comment on NIFI-10234 at 7/25/22 7:41 AM:
--------------------------------------------------------------
Thanks for your suggestions!!!
BTW, sorry for not replying to your message in time, because I hava had a vacation last week.
h1. *Here is a userguide about the processor*
h2. PutIoTDBProcessor
h3. Properties of PutIoTDBProcessor
||property||description||default value||necessary||
|IoTDB Host|The host of IoTDB.|127.0.0.1|false|
|IoTDB Port|The port of IoTDB.|6667|false|
|Username|Username to access the IoTDB.|root|false|
|Password|Password to access the IoTDB.|root|false|
|Record Reader|Specifies the type of Record Reader controller service to use for parsing the incoming data and determining the schema.|null|true|
h3. Schema of flowfile
There are a couple of rules about flowfile:
# the flowfile can be read by {{{}Record Reader{}}}.
# the schema of flowfile must contains a field {{{}Time{}}}.
# the data type of field {{Time}} must be {{STRING}} or {{{}LONG{}}}.
# fields excepted {{Time}} must start with {{{}root.{}}}.
# the supported data types are {{{}INT{}}}, {{{}LONG{}}}, {{{}FLOAT{}}}, {{{}DOUBLE{}}}, {{{}BOOLEAN{}}}, {{{}TEXT{}}}.
h3. Attributes of flowfile
||attribute||description||default value||
|schema|The schema that IoTDB needs doesn't support good by NiFi. Therefore, you can define the schema here. Besides, you can set encoding type and compression type by this method.
If you don't set this attribute, the inferred schema will be used.|null|
|aligned|Whether using aligned interface?|false|
|maxRowNumber|Specifies the max row number of each tablet.|1024|
h3. Convert schema by attribute
As mentioned above, converting schema by attribute which is more flexible and stronger than native schema.
The structure of attribute:
{code:java}
{
"timeType": "long",
"field": [{
"tsName": "root.sg.d1.s1",
"dataType": "INT32",
"encoding": "RLE",
"compressionType": "GZIP"
}, {
"tsName": "root.sg.d1.s2",
"dataType": "INT64",
"encoding": "RLE",
"compressionType": "GZIP"
}]
} {code}
*Note*
# The JSON of schema must contain {{timeType}} and {{{}fields{}}}.
# There are only two options {{long}} and {{string}} for {{{}timeType{}}}.
# The columns {{tsName}} and {{dataType}} must be set.
# The tsName must start with {{{}root.{}}}.
# The supported {{dataTypes}} are {{{}INT32{}}}, {{{}INT64{}}}, {{{}FLOAT{}}}, {{{}DOUBLE{}}}, {{{}BOOLEAN{}}}, {{{}TEXT{}}}.
# The supported {{encoding}} are {{{}PLAIN{}}}, {{{}DICTIONARY{}}}, {{{}RLE{}}}, {{{}DIFF{}}}, {{{}TS_2DIFF{}}}, {{{}BITMAP{}}}, {{{}GORILLA_V1{}}}, {{{}REGULAR{}}}, {{{}GORILLA{}}}.
# The supported {{compressionType}} are {{{}UNCOMPRESSED{}}}, {{{}SNAPPY{}}}, {{{}GZIP{}}}, {{{}LZO{}}}, {{{}SDT{}}}, {{{}PAA{}}}, {{{}PLA{}}}, {{{}LZ4{}}}.
h3. Relationships
||relationship||description||
|success|Data can be written correctly or flow file is empty.|
|failure|The shema or flow file is abnormal.|
was (Author: xuanronaldo):
Thanks for your suggestions!!!
BTW, sorry for not replying to your message in time, because I hava had a vacation last week.
h1. *Here is a userguide about the processor*
h2. PutIoTDBProcessor
h3. Properties of PutIoTDBProcessor
||property||description||default value||necessary||
|IoTDB Host|The host of IoTDB.|127.0.0.1|false|
|IoTDB Port|The port of IoTDB.|6667|false|
|Username|Username to access the IoTDB.|root|false|
|Password|Password to access the IoTDB.|root|false|
|Record Reader|Specifies the type of Record Reader controller service to use for parsing the incoming data and determining the schema.|null|true|
h3. Schema of flowfile
There are a couple of rules about flowfile:
# the flowfile can be read by {{Record Reader}}.
# the schema of flowfile must contains a field {{Time}}.
# the data type of field {{Time}} must be {{STRING}} or {{LONG}}.
# fields excepted {{Time}} must start with {{root.}}.
# the supported data types are {{INT}}, {{LONG}}, {{FLOAT}}, {{DOUBLE}}, {{BOOLEAN}}, {{TEXT}}.
h3. Attributes of flowfile
||attribute||description||default value||
|schema|The schema that IoTDB needs doesn't support good by NiFi. Therefore, you can define the schema here. Besides, you can set encoding type and compression type by this method.
If you don't set this attribute, the inferred schema will be used.|null|
|aligned|Whether using aligned interface?|false|
|maxRowNumber|Specifies the max row number of each tablet.|1024|
h3. Convert schema by attribute
As mentioned above, converting schema by attribute which is more flexible and stronger than native schema.
The structure of attribute:
{code:java}
{
"timeType": "long",
"field": [{
"tsName": "root.sg.d1.s1",
"dataType": "INT32",
"encoding": "RLE",
"compressionType": "GZIP"
}, {
"tsName": "root.sg.d1.s2",
"dataType": "INT64",
"encoding": "RLE",
"compressionType": "GZIP"
}]
} {code}
*Note*
# The JSON of schema must contain {{timeType}} and {{fields}}.
# There are only two options {{long}} and {{string}} for {{timeType}}.
# The columns {{tsName}} and {{dataType}} must be set.
# The tsName must start with {{root.}}.
# The supported {{dataTypes}} are {{INT32}}, {{INT64}}, {{FLOAT}}, {{DOUBLE}}, {{BOOLEAN}}, {{TEXT}}.
# The supported {{encoding}} are {{PLAIN}}, {{DICTIONARY}}, {{RLE}}, {{DIFF}}, {{TS_2DIFF}}, {{BITMAP}}, {{GORILLA_V1}}, {{REGULAR}}, {{GORILLA}}.
# The supported {{compressionType}} are {{UNCOMPRESSED}}, {{SNAPPY}}, {{GZIP}}, {{LZO}}, {{SDT}}, {{PAA}}, {{PLA}}, {{LZ4}}.
h3. Relationships
||relationship||description||
|success|Data can be written correctly or flow file is empty.|
|failure|The shema or flow file is abnormal.|
> implement a processor of Apache IoTDB
> -------------------------------------
>
> Key: NIFI-10234
> URL: https://issues.apache.org/jira/browse/NIFI-10234
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Xuan Ronaldo
> Priority: Major
>
> Hi, folks
>
> I'm a contributer of Apache IoTDB. Recently, I have implemented a processor which can write data to IoTDB. I'd like to submit it to the NiFi as a build-in processor. Besides, there are more processors or controller services will be developed.
>
> Regards,
> Xuan Ronaldo
--
This message was sent by Atlassian Jira
(v8.20.10#820010)