You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2022/07/11 10:09:28 UTC
[GitHub] [nifi] CloudWise-Lukemiao opened a new pull request, #6190: NIFI-10210 Processor supporting Apache iotdb database
CloudWise-Lukemiao opened a new pull request, #6190:
URL: https://github.com/apache/nifi/pull/6190
Iotdb process supports SQL insertion, session insertion and SQL query
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] CloudWise-Lukemiao commented on pull request #6190: NIFI-10210 Processor supporting Apache iotdb database
Posted by GitBox <gi...@apache.org>.
CloudWise-Lukemiao commented on PR #6190:
URL: https://github.com/apache/nifi/pull/6190#issuecomment-1196571706
> > Thanks for pointing out the problem. Next, I will change the query interface mode of JDBC to the session mode of iotdb, and I will remove the JDBC insertion mode. I hope to discuss the options and general design on the associate JIRA issue would be a great next step
>
> Thanks for the response @CloudWise-Lukemiao, that sounds like a good next step. Including unit tests for a new processor is also important.
> > Thanks for pointing out the problem. Next, I will change the query interface mode of JDBC to the session mode of iotdb, and I will remove the JDBC insertion mode. I hope to discuss the options and general design on the associate JIRA issue would be a great next step
>
> Thanks for the response @CloudWise-Lukemiao, that sounds like a good next step. Including unit tests for a new processor is also important.
hi,There is a problem I want to discuss. In addition to the data source, the putiotdb processor interface also needs a devicesid configuration (string), measurements (array), datatypes (array) and measurements, and batchsize (submitted quantity). As shown in the figure。
<img width="746" alt="image" src="https://user-images.githubusercontent.com/76942485/181229086-8bb677b2-3fa4-4ef7-8f38-4ffaee29057a.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] CloudWise-Lukemiao commented on pull request #6190: NIFI-10210 Processor supporting Apache iotdb database
Posted by GitBox <gi...@apache.org>.
CloudWise-Lukemiao commented on PR #6190:
URL: https://github.com/apache/nifi/pull/6190#issuecomment-1181217095
> Thanks for the contribution @CloudWise-Lukemiao!
>
> The implementation appears to use both the JDBC and Native Java library for interacting with IoTDB. JDBC access is already possible using existing components and leveraging the `DBCPConnnectionPool` or `HikariCPConnectionPool` services.
>
> The [IoTDB documentation for JDBC](https://iotdb.apache.org/UserGuide/Master/API/Programming-JDBC.html) notes that using JDBC is not recommended for high throughput, so there could be value in new NiFi components that leverage the [Java Native API](https://iotdb.apache.org/UserGuide/Master/API/Programming-Java-Native-API.html).
>
> With that background, the `PutIoTDB` processor includes a direct dependency on JSON input, which is problematic in terms of flexibility. It would be much better to evaluate an approach that leverages record-oriented processing using NiFi Record Reader services. It may also be useful to create a Controller Service that provides reusable IoTDB Sessions, which could support multiple processors.
>
> If you are willing to work through more of these details, discussing the options and general design on the associate Jira issue would be a great next step.
> Thanks for the contribution @CloudWise-Lukemiao!
>
> The implementation appears to use both the JDBC and Native Java library for interacting with IoTDB. JDBC access is already possible using existing components and leveraging the `DBCPConnnectionPool` or `HikariCPConnectionPool` services.
>
> The [IoTDB documentation for JDBC](https://iotdb.apache.org/UserGuide/Master/API/Programming-JDBC.html) notes that using JDBC is not recommended for high throughput, so there could be value in new NiFi components that leverage the [Java Native API](https://iotdb.apache.org/UserGuide/Master/API/Programming-Java-Native-API.html).
>
> With that background, the `PutIoTDB` processor includes a direct dependency on JSON input, which is problematic in terms of flexibility. It would be much better to evaluate an approach that leverages record-oriented processing using NiFi Record Reader services. It may also be useful to create a Controller Service that provides reusable IoTDB Sessions, which could support multiple processors.
>
> If you are willing to work through more of these details, discussing the options and general design on the associate Jira issue would be a great next step.
Thanks for pointing out the problem. Next, I will change the query interface mode of JDBC to the session mode of iotdb, and I will remove the JDBC insertion mode. I hope to discuss the options and general design on the associate JIRA issue would be a great next step
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6190: NIFI-10210 Processor supporting Apache iotdb database
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on PR #6190:
URL: https://github.com/apache/nifi/pull/6190#issuecomment-1295308526
@CloudWise-Lukemiao as mentioned earlier, there is more implementation discussion on [NIFI-10234](https://issues.apache.org/jira/browse/NIFI-10234) and ongoing work for a PutIoTDB Processor in PR #6416.
With that background, it seems best to close this PR for now. Feel free to provide feedback on the other PR, as there are still some issues to be resolved in terms for record field mapping.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory closed pull request #6190: NIFI-10210 Processor supporting Apache iotdb database
Posted by GitBox <gi...@apache.org>.
exceptionfactory closed pull request #6190: NIFI-10210 Processor supporting Apache iotdb database
URL: https://github.com/apache/nifi/pull/6190
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6190: NIFI-10210 Processor supporting Apache iotdb database
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on PR #6190:
URL: https://github.com/apache/nifi/pull/6190#issuecomment-1198633779
@CloudWise-Lukemiao There is some discussion on [NIFI-10234](https://issues.apache.org/jira/browse/NIFI-10234), so working through the details on that issue would be helpful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #6190: NIFI-10210 Processor supporting Apache iotdb database
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on PR #6190:
URL: https://github.com/apache/nifi/pull/6190#issuecomment-1185723848
> Thanks for pointing out the problem. Next, I will change the query interface mode of JDBC to the session mode of iotdb, and I will remove the JDBC insertion mode. I hope to discuss the options and general design on the associate JIRA issue would be a great next step
Thanks for the response @CloudWise-Lukemiao, that sounds like a good next step. Including unit tests for a new processor is also important.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] CloudWise-Lukemiao commented on pull request #6190: NIFI-10210 Processor supporting Apache iotdb database
Posted by GitBox <gi...@apache.org>.
CloudWise-Lukemiao commented on PR #6190:
URL: https://github.com/apache/nifi/pull/6190#issuecomment-1181284533
>
> > Thanks for the contribution @CloudWise-Lukemiao!
> > The implementation appears to use both the JDBC and Native Java library for interacting with IoTDB. JDBC access is already possible using existing components and leveraging the `DBCPConnnectionPool` or `HikariCPConnectionPool` services.
> > The [IoTDB documentation for JDBC](https://iotdb.apache.org/UserGuide/Master/API/Programming-JDBC.html) notes that using JDBC is not recommended for high throughput, so there could be value in new NiFi components that leverage the [Java Native API](https://iotdb.apache.org/UserGuide/Master/API/Programming-Java-Native-API.html).
> > With that background, the `PutIoTDB` processor includes a direct dependency on JSON input, which is problematic in terms of flexibility. It would be much better to evaluate an approach that leverages record-oriented processing using NiFi Record Reader services. It may also be useful to create a Controller Service that provides reusable IoTDB Sessions, which could support multiple processors.
> > If you are willing to work through more of these details, discussing the options and general design on the associate Jira issue would be a great next step.
>
> > Thanks for the contribution @CloudWise-Lukemiao!
> > The implementation appears to use both the JDBC and Native Java library for interacting with IoTDB. JDBC access is already possible using existing components and leveraging the `DBCPConnnectionPool` or `HikariCPConnectionPool` services.
> > The [IoTDB documentation for JDBC](https://iotdb.apache.org/UserGuide/Master/API/Programming-JDBC.html) notes that using JDBC is not recommended for high throughput, so there could be value in new NiFi components that leverage the [Java Native API](https://iotdb.apache.org/UserGuide/Master/API/Programming-Java-Native-API.html).
> > With that background, the `PutIoTDB` processor includes a direct dependency on JSON input, which is problematic in terms of flexibility. It would be much better to evaluate an approach that leverages record-oriented processing using NiFi Record Reader services. It may also be useful to create a Controller Service that provides reusable IoTDB Sessions, which could support multiple processors.
> > If you are willing to work through more of these details, discussing the options and general design on the associate Jira issue would be a great next step.
>
> Thanks for pointing out the problem. Next, I will change the query interface mode of JDBC to the session mode of iotdb, and I will remove the JDBC insertion mode. I hope to discuss the options and general design on the associate JIRA issue would be a great next step.I use record render to re optimize
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org