You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/04/07 09:29:57 UTC
[GitHub] [pulsar] vagetablechicken opened a new pull request, #15064: [feat][connector] Support openmldb jdbc sink connector
vagetablechicken opened a new pull request, #15064:
URL: https://github.com/apache/pulsar/pull/15064
<!--
### Contribution Checklist
- PR title format should be *[type][component] summary*. For details, see *[Guideline - Pulsar PR Naming Convention](https://docs.google.com/document/d/1d8Pw6ZbWk-_pCKdOmdvx9rnhPiyuxwq60_TrD68d7BA/edit#heading=h.trs9rsex3xom)*.
- Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
- Each pull request should address only one issue, not mix up code from multiple issues.
- Each commit in the pull request has a meaningful commit message
- Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
-->
### Motivation
Support sink to OpenMLDB.
### Modifications
Add a module 'openmldb 'in pulsar-io/jdbc
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
If we want to test this connector, we need to start an OpenMLDB cluster. So it can't be tested simply. But jdbc have tests.
### Does this pull request potentially affect one of the following parts:
No
### Documentation
Check the box below or label this PR directly.
Need to update docs?
- [x] `doc`
(Your PR contains doc changes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] RobertIndie commented on a diff in pull request #15064: [feat][connector] Support 4paradigm/OpenMLDB jdbc sink connector
Posted by GitBox <gi...@apache.org>.
RobertIndie commented on code in PR #15064:
URL: https://github.com/apache/pulsar/pull/15064#discussion_r846910096
##########
pulsar-io/jdbc/openmldb/pom.xml:
##########
@@ -0,0 +1,62 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>pulsar-io-jdbc</artifactId>
+ <groupId>org.apache.pulsar</groupId>
+ <version>2.10.0-SNAPSHOT</version>
Review Comment:
```suggestion
<version>2.11.0-SNAPSHOT</version>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] RobertIndie commented on pull request #15064: [feat][connector] Support 4paradigm/OpenMLDB jdbc sink connector
Posted by GitBox <gi...@apache.org>.
RobertIndie commented on PR #15064:
URL: https://github.com/apache/pulsar/pull/15064#issuecomment-1098814307
The `openmldb-jdbc` depends on log4j 1.x and it causes the OWASP dependency check to fail. We need a new release of openmldb-jdbc to fix this security issue. This PR disables OWASP check for the opemmldb connector for now. And it will be reenabled until this security issue has been fixed.
Besides, the [current owasp dependency check failure](https://github.com/apache/pulsar/runs/6018518487?check_suite_focus=true) is not related to this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] vagetablechicken commented on pull request #15064: [feat][connector] Support 4paradigm/OpenMLDB jdbc sink connector
Posted by GitBox <gi...@apache.org>.
vagetablechicken commented on PR #15064:
URL: https://github.com/apache/pulsar/pull/15064#issuecomment-1094483735
assemble/io.xml fixed. And I'll add integration test in the next pr.
Plz help me to pass the ci. Need rerun ci? @RobertIndie
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] RobertIndie commented on pull request #15064: [feat][connector] Support 4paradigm/OpenMLDB jdbc sink connector
Posted by GitBox <gi...@apache.org>.
RobertIndie commented on PR #15064:
URL: https://github.com/apache/pulsar/pull/15064#issuecomment-1094675039
While testing, I found this exception when starting the connector. Could you please take a look? @vagetablechicken
```
2022-04-11T15:58:55,371+0800 [function-timer-thread-17-1] ERROR org.apache.pulsar.functions.runtime.RuntimeSpawner - public/default/jdbc-openmldb-sink Function Container is dead with following exception. Restarting.
java.lang.UnsatisfiedLinkError: 'long com._4paradigm.openmldb.sql_router_sdkJNI.new_SQLRouterOptions()'
at com._4paradigm.openmldb.sql_router_sdkJNI.new_SQLRouterOptions(Native Method) ~[?:?]
at com._4paradigm.openmldb.SQLRouterOptions.<init>(SQLRouterOptions.java:56) ~[?:?]
at com._4paradigm.openmldb.sdk.impl.SqlClusterExecutor.<init>(SqlClusterExecutor.java:66) ~[?:?]
at com._4paradigm.openmldb.sdk.impl.SqlClusterExecutor.<init>(SqlClusterExecutor.java:90) ~[?:?]
at com._4paradigm.openmldb.jdbc.SQLDriver.connect(SQLDriver.java:54) ~[?:?]
at java.sql.DriverManager.getConnection(DriverManager.java:677) ~[java.sql:?]
at java.sql.DriverManager.getConnection(DriverManager.java:189) ~[java.sql:?]
at org.apache.pulsar.io.jdbc.JdbcAbstractSink.open(JdbcAbstractSink.java:93) ~[?:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:857) ~[classes/:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:223) ~[classes/:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:258) ~[classes/:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
```
<details>
<summary>Full exception context</summary>
<code><pre>
2022-04-11T15:58:13,069+0800 [main] INFO org.apache.pulsar.functions.utils.io.ConnectorUtils - Searching for connectors in /Users/aaronrobert/codebase/pulsar/connectors
2022-04-11T15:58:13,097+0800 [main] WARN org.apache.pulsar.functions.utils.io.ConnectorUtils - Connectors archive directory not found
2022-04-11T15:58:13,128+0800 [main] INFO org.apache.pulsar.common.nar.NarUnpacker - Created directory /var/folders/sx/ws7wtlmn3t1bhrdlsfbmlv6r0000gn/T/pulsar_localrunner_nars_3526555380500249181/pulsar-nar/pulsar-io-jdbc-openmldb-2.11.0-SNAPSHOT.nar-unpacked
2022-04-11T15:58:15,559+0800 [main] INFO org.apache.pulsar.common.nar.NarUnpacker - Extracting /Users/aaronrobert/codebase/pulsar/pulsar-io/jdbc/openmldb/target/pulsar-io-jdbc-openmldb-2.11.0-SNAPSHOT.nar to /var/folders/sx/ws7wtlmn3t1bhrdlsfbmlv6r0000gn/T/pulsar_localrunner_nars_3526555380500249181/pulsar-nar/pulsar-io-jdbc-openmldb-2.11.0-SNAPSHOT.nar-unpacked/ABc4BPN6ZjOQMHeC_JQ1xg
2022-04-11T15:58:19,304+0800 [main] WARN io.netty.resolver.dns.DnsServerAddressStreamProviders - Can not find io.netty.resolver.dns.macos.MacOSDnsServerAddressStreamProvider in the classpath, fallback to system defaults. This may result in incorrect DNS resolutions on MacOS.
2022-04-11T15:58:24,580+0800 [main] INFO org.apache.pulsar.functions.runtime.RuntimeSpawner - public/default/jdbc-openmldb-sink-0 RuntimeSpawner starting function
2022-04-11T15:58:24,583+0800 [main] INFO org.apache.pulsar.common.nar.FileUtils - Jar file /Users/aaronrobert/codebase/pulsar/pulsar-io/jdbc/openmldb/target/pulsar-io-jdbc-openmldb-2.11.0-SNAPSHOT.nar contains META-INF/bundled-dependencies, it may be a NAR file
2022-04-11T15:58:24,583+0800 [main] INFO org.apache.pulsar.functions.runtime.thread.ThreadRuntime - Trying Loading file as NAR file: /Users/aaronrobert/codebase/pulsar/pulsar-io/jdbc/openmldb/target/pulsar-io-jdbc-openmldb-2.11.0-SNAPSHOT.nar
2022-04-11T15:58:25,331+0800 [main] INFO org.apache.pulsar.functions.runtime.thread.ThreadRuntime - Initialize function class loader for function jdbc-openmldb-sink at function cache manager, functionClassLoader: org.apache.pulsar.common.nar.NarClassLoader[/var/folders/sx/ws7wtlmn3t1bhrdlsfbmlv6r0000gn/T/pulsar_localrunner_nars_3526555380500249181/pulsar-nar/pulsar-io-jdbc-openmldb-2.11.0-SNAPSHOT.nar-unpacked/ABc4BPN6ZjOQMHeC_JQ1xg]
2022-04-11T15:58:25,361+0800 [main] INFO org.apache.pulsar.functions.runtime.thread.ThreadRuntime - ThreadContainer starting function with instanceId 0 functionId 9c836a56-83a4-416f-b9b7-bbabedad787f namespace default
2022-04-11T15:58:55,371+0800 [function-timer-thread-17-1] ERROR org.apache.pulsar.functions.runtime.RuntimeSpawner - public/default/jdbc-openmldb-sink Function Container is dead with following exception. Restarting.
java.lang.UnsatisfiedLinkError: 'long com._4paradigm.openmldb.sql_router_sdkJNI.new_SQLRouterOptions()'
at com._4paradigm.openmldb.sql_router_sdkJNI.new_SQLRouterOptions(Native Method) ~[?:?]
at com._4paradigm.openmldb.SQLRouterOptions.<init>(SQLRouterOptions.java:56) ~[?:?]
at com._4paradigm.openmldb.sdk.impl.SqlClusterExecutor.<init>(SqlClusterExecutor.java:66) ~[?:?]
at com._4paradigm.openmldb.sdk.impl.SqlClusterExecutor.<init>(SqlClusterExecutor.java:90) ~[?:?]
at com._4paradigm.openmldb.jdbc.SQLDriver.connect(SQLDriver.java:54) ~[?:?]
at java.sql.DriverManager.getConnection(DriverManager.java:677) ~[java.sql:?]
at java.sql.DriverManager.getConnection(DriverManager.java:189) ~[java.sql:?]
at org.apache.pulsar.io.jdbc.JdbcAbstractSink.open(JdbcAbstractSink.java:93) ~[?:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:857) ~[classes/:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:223) ~[classes/:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:258) ~[classes/:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2022-04-11T15:58:55,395+0800 [function-timer-thread-17-1] INFO org.apache.pulsar.functions.runtime.thread.ThreadRuntime - Unloading JAR files for instanceId 0 functionId 9c836a56-83a4-416f-b9b7-bbabedad787f namespace default
2022-04-11T15:58:55,397+0800 [function-timer-thread-17-1] INFO org.apache.pulsar.common.nar.FileUtils - Jar file /Users/aaronrobert/codebase/pulsar/pulsar-io/jdbc/openmldb/target/pulsar-io-jdbc-openmldb-2.11.0-SNAPSHOT.nar contains META-INF/bundled-dependencies, it may be a NAR file
2022-04-11T15:58:55,397+0800 [function-timer-thread-17-1] INFO org.apache.pulsar.functions.runtime.thread.ThreadRuntime - Trying Loading file as NAR file: /Users/aaronrobert/codebase/pulsar/pulsar-io/jdbc/openmldb/target/pulsar-io-jdbc-openmldb-2.11.0-SNAPSHOT.nar
2022-04-11T15:58:55,920+0800 [function-timer-thread-17-1] INFO org.apache.pulsar.functions.runtime.thread.ThreadRuntime - Initialize function class loader for function jdbc-openmldb-sink at function cache manager, functionClassLoader: org.apache.pulsar.common.nar.NarClassLoader[/var/folders/sx/ws7wtlmn3t1bhrdlsfbmlv6r0000gn/T/pulsar_localrunner_nars_3526555380500249181/pulsar-nar/pulsar-io-jdbc-openmldb-2.11.0-SNAPSHOT.nar-unpacked/ABc4BPN6ZjOQMHeC_JQ1xg]
^C2022-04-11T15:58:59,284+0800 [Thread-0] INFO org.apache.pulsar.functions.runtime.thread.ThreadRuntime - Unloading JAR files for instanceId 0 functionId 9c836a56-83a4-416f-b9b7-bbabedad787f namespace default
2022-04-11T15:58:59,285+0800 [Thread-0] INFO org.apache.pulsar.client.impl.PulsarClientImpl - Client closing. URL: pulsar://localhost:6650
➜ openmldb git:(openmldb-connector) ✗ 2022-04-11T15:59:00,930+0800 [function-timer-thread-17-1] INFO org.apache.pulsar.functions.runtime.thread.ThreadRuntime - ThreadContainer starting function with instanceId 0 functionId 9c836a56-83a4-416f-b9b7-bbabedad787f namespace default
</pre></code>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #15064: [feat][connector] Support 4paradigm/OpenMLDB jdbc sink connector
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on PR #15064:
URL: https://github.com/apache/pulsar/pull/15064#issuecomment-1097501546
@vagetablechicken could you please help check the failed `OWASP Dependency Check` test? I'm not sure if it was introduced by new dependencies, maybe you can try to rebase to the master branch first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] RobertIndie commented on a diff in pull request #15064: [feat][connector] Support 4paradigm/OpenMLDB jdbc sink connector
Posted by GitBox <gi...@apache.org>.
RobertIndie commented on code in PR #15064:
URL: https://github.com/apache/pulsar/pull/15064#discussion_r845801143
##########
pulsar-io/jdbc/openmldb/src/main/java/org/apache/pulsar/io/jdbc/OpenMLDBJdbcAutoSchemaSink.java:
##########
@@ -0,0 +1,36 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pulsar.io.jdbc;
+
+import org.apache.pulsar.io.core.annotations.Connector;
+import org.apache.pulsar.io.core.annotations.IOType;
+
+/**
Review Comment:
This seems to be an invalid comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a diff in pull request #15064: [feat][connector] Support 4paradigm/OpenMLDB jdbc sink connector
Posted by GitBox <gi...@apache.org>.
momo-jun commented on code in PR #15064:
URL: https://github.com/apache/pulsar/pull/15064#discussion_r847858389
##########
site2/docs/io-jdbc-sink.md:
##########
@@ -15,16 +15,16 @@ The configuration of all JDBC sink connectors has the following properties.
### Property
-| Name | Type|Required | Default | Description
-|------|----------|----------|---------|-------------|
-| `userName` | String|false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.**|
-| `password` | String|false | " " (empty string)| The password used to connect to the database specified by `jdbcUrl`. <br><br>**Note: `password` is case-sensitive.**|
-| `jdbcUrl` | String|true | " " (empty string) | The JDBC URL of the database to which the connector connects. |
-| `tableName` | String|true | " " (empty string) | The name of the table to which the connector writes. |
-| `nonKey` | String|false | " " (empty string) | A comma-separated list contains the fields used in updating events. |
-| `key` | String|false | " " (empty string) | A comma-separated list contains the fields used in `where` condition of updating and deleting events. |
-| `timeoutMs` | int| false|500 | The JDBC operation timeout in milliseconds. |
-| `batchSize` | int|false | 200 | The batch size of updates made to the database. |
+| Name | Type | Required | Default | Description |
+|-------------|--------|----------|--------------------|----------------------------------------------------------------------------------------------------------------------|
+| `userName` | String | false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.** |
+| `password` | String | false | " " (empty string) | The password used to connect to the database specified by `jdbcUrl`. <br><br>**Note: `password` is case-sensitive.** |
+| `jdbcUrl` | String | true | " " (empty string) | The JDBC URL of the database to which the connector connects. |
+| `tableName` | String | true | " " (empty string) | The name of the table to which the connector writes. |
Review Comment:
```suggestion
| `tableName` | String | true | " " (empty string) | The name of the table that the connector writes to. |
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] momo-jun commented on a diff in pull request #15064: [feat][connector] Support 4paradigm/OpenMLDB jdbc sink connector
Posted by GitBox <gi...@apache.org>.
momo-jun commented on code in PR #15064:
URL: https://github.com/apache/pulsar/pull/15064#discussion_r847858077
##########
site2/docs/io-jdbc-sink.md:
##########
@@ -15,16 +15,16 @@ The configuration of all JDBC sink connectors has the following properties.
### Property
-| Name | Type|Required | Default | Description
-|------|----------|----------|---------|-------------|
-| `userName` | String|false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.**|
-| `password` | String|false | " " (empty string)| The password used to connect to the database specified by `jdbcUrl`. <br><br>**Note: `password` is case-sensitive.**|
-| `jdbcUrl` | String|true | " " (empty string) | The JDBC URL of the database to which the connector connects. |
-| `tableName` | String|true | " " (empty string) | The name of the table to which the connector writes. |
-| `nonKey` | String|false | " " (empty string) | A comma-separated list contains the fields used in updating events. |
-| `key` | String|false | " " (empty string) | A comma-separated list contains the fields used in `where` condition of updating and deleting events. |
-| `timeoutMs` | int| false|500 | The JDBC operation timeout in milliseconds. |
-| `batchSize` | int|false | 200 | The batch size of updates made to the database. |
+| Name | Type | Required | Default | Description |
+|-------------|--------|----------|--------------------|----------------------------------------------------------------------------------------------------------------------|
+| `userName` | String | false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.** |
+| `password` | String | false | " " (empty string) | The password used to connect to the database specified by `jdbcUrl`. <br><br>**Note: `password` is case-sensitive.** |
+| `jdbcUrl` | String | true | " " (empty string) | The JDBC URL of the database to which the connector connects. |
Review Comment:
```suggestion
| `jdbcUrl` | String | true | " " (empty string) | The JDBC URL of the database that the connector connects to. |
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on a diff in pull request #15064: [feat][connector] Support 4paradigm/OpenMLDB jdbc sink connector
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on code in PR #15064:
URL: https://github.com/apache/pulsar/pull/15064#discussion_r847899347
##########
site2/docs/io-jdbc-sink.md:
##########
@@ -15,16 +15,16 @@ The configuration of all JDBC sink connectors has the following properties.
### Property
-| Name | Type|Required | Default | Description
-|------|----------|----------|---------|-------------|
-| `userName` | String|false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.**|
-| `password` | String|false | " " (empty string)| The password used to connect to the database specified by `jdbcUrl`. <br><br>**Note: `password` is case-sensitive.**|
-| `jdbcUrl` | String|true | " " (empty string) | The JDBC URL of the database to which the connector connects. |
-| `tableName` | String|true | " " (empty string) | The name of the table to which the connector writes. |
-| `nonKey` | String|false | " " (empty string) | A comma-separated list contains the fields used in updating events. |
-| `key` | String|false | " " (empty string) | A comma-separated list contains the fields used in `where` condition of updating and deleting events. |
-| `timeoutMs` | int| false|500 | The JDBC operation timeout in milliseconds. |
-| `batchSize` | int|false | 200 | The batch size of updates made to the database. |
+| Name | Type | Required | Default | Description |
+|-------------|--------|----------|--------------------|----------------------------------------------------------------------------------------------------------------------|
+| `userName` | String | false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.** |
+| `password` | String | false | " " (empty string) | The password used to connect to the database specified by `jdbcUrl`. <br><br>**Note: `password` is case-sensitive.** |
Review Comment:
same
##########
site2/docs/io-jdbc-sink.md:
##########
@@ -15,16 +15,16 @@ The configuration of all JDBC sink connectors has the following properties.
### Property
-| Name | Type|Required | Default | Description
-|------|----------|----------|---------|-------------|
-| `userName` | String|false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.**|
-| `password` | String|false | " " (empty string)| The password used to connect to the database specified by `jdbcUrl`. <br><br>**Note: `password` is case-sensitive.**|
-| `jdbcUrl` | String|true | " " (empty string) | The JDBC URL of the database to which the connector connects. |
-| `tableName` | String|true | " " (empty string) | The name of the table to which the connector writes. |
-| `nonKey` | String|false | " " (empty string) | A comma-separated list contains the fields used in updating events. |
-| `key` | String|false | " " (empty string) | A comma-separated list contains the fields used in `where` condition of updating and deleting events. |
-| `timeoutMs` | int| false|500 | The JDBC operation timeout in milliseconds. |
-| `batchSize` | int|false | 200 | The batch size of updates made to the database. |
+| Name | Type | Required | Default | Description |
+|-------------|--------|----------|--------------------|----------------------------------------------------------------------------------------------------------------------|
+| `userName` | String | false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.** |
Review Comment:
```suggestion
| `userName` | String | false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br /><br />**Note: `userName` is case-sensitive.** |
```
syntax: https://docs.google.com/document/d/1IV35SI_F8G8cL-Vuzknc6RTGLK9_edRMpZpnrHvAWNs/edit#bookmark=id.wcmnciyvvq6p
##########
site2/docs/io-jdbc-sink.md:
##########
@@ -15,16 +15,16 @@ The configuration of all JDBC sink connectors has the following properties.
### Property
-| Name | Type|Required | Default | Description
-|------|----------|----------|---------|-------------|
-| `userName` | String|false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.**|
-| `password` | String|false | " " (empty string)| The password used to connect to the database specified by `jdbcUrl`. <br><br>**Note: `password` is case-sensitive.**|
-| `jdbcUrl` | String|true | " " (empty string) | The JDBC URL of the database to which the connector connects. |
-| `tableName` | String|true | " " (empty string) | The name of the table to which the connector writes. |
-| `nonKey` | String|false | " " (empty string) | A comma-separated list contains the fields used in updating events. |
-| `key` | String|false | " " (empty string) | A comma-separated list contains the fields used in `where` condition of updating and deleting events. |
-| `timeoutMs` | int| false|500 | The JDBC operation timeout in milliseconds. |
-| `batchSize` | int|false | 200 | The batch size of updates made to the database. |
+| Name | Type | Required | Default | Description |
+|-------------|--------|----------|--------------------|----------------------------------------------------------------------------------------------------------------------|
+| `userName` | String | false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.** |
+| `password` | String | false | " " (empty string) | The password used to connect to the database specified by `jdbcUrl`. <br><br>**Note: `password` is case-sensitive.** |
+| `jdbcUrl` | String | true | " " (empty string) | The JDBC URL of the database to which the connector connects. |
+| `tableName` | String | true | " " (empty string) | The name of the table to which the connector writes. |
+| `nonKey` | String | false | " " (empty string) | A comma-separated list contains the fields used in updating events. |
Review Comment:
```suggestion
| `nonKey` | String | false | " " (empty string) | A comma-separated list containing the fields used in updating events. |
```
##########
site2/docs/io-jdbc-sink.md:
##########
@@ -15,16 +15,16 @@ The configuration of all JDBC sink connectors has the following properties.
### Property
-| Name | Type|Required | Default | Description
-|------|----------|----------|---------|-------------|
-| `userName` | String|false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.**|
-| `password` | String|false | " " (empty string)| The password used to connect to the database specified by `jdbcUrl`. <br><br>**Note: `password` is case-sensitive.**|
-| `jdbcUrl` | String|true | " " (empty string) | The JDBC URL of the database to which the connector connects. |
-| `tableName` | String|true | " " (empty string) | The name of the table to which the connector writes. |
-| `nonKey` | String|false | " " (empty string) | A comma-separated list contains the fields used in updating events. |
-| `key` | String|false | " " (empty string) | A comma-separated list contains the fields used in `where` condition of updating and deleting events. |
-| `timeoutMs` | int| false|500 | The JDBC operation timeout in milliseconds. |
-| `batchSize` | int|false | 200 | The batch size of updates made to the database. |
+| Name | Type | Required | Default | Description |
+|-------------|--------|----------|--------------------|----------------------------------------------------------------------------------------------------------------------|
+| `userName` | String | false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.** |
+| `password` | String | false | " " (empty string) | The password used to connect to the database specified by `jdbcUrl`. <br><br>**Note: `password` is case-sensitive.** |
+| `jdbcUrl` | String | true | " " (empty string) | The JDBC URL of the database to which the connector connects. |
+| `tableName` | String | true | " " (empty string) | The name of the table to which the connector writes. |
+| `nonKey` | String | false | " " (empty string) | A comma-separated list contains the fields used in updating events. |
Review Comment:
or `A comma-separated list that contains the fields used in updating events`
##########
site2/docs/io-jdbc-sink.md:
##########
@@ -15,16 +15,16 @@ The configuration of all JDBC sink connectors has the following properties.
### Property
-| Name | Type|Required | Default | Description
-|------|----------|----------|---------|-------------|
-| `userName` | String|false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.**|
-| `password` | String|false | " " (empty string)| The password used to connect to the database specified by `jdbcUrl`. <br><br>**Note: `password` is case-sensitive.**|
-| `jdbcUrl` | String|true | " " (empty string) | The JDBC URL of the database to which the connector connects. |
-| `tableName` | String|true | " " (empty string) | The name of the table to which the connector writes. |
-| `nonKey` | String|false | " " (empty string) | A comma-separated list contains the fields used in updating events. |
-| `key` | String|false | " " (empty string) | A comma-separated list contains the fields used in `where` condition of updating and deleting events. |
-| `timeoutMs` | int| false|500 | The JDBC operation timeout in milliseconds. |
-| `batchSize` | int|false | 200 | The batch size of updates made to the database. |
+| Name | Type | Required | Default | Description |
+|-------------|--------|----------|--------------------|----------------------------------------------------------------------------------------------------------------------|
+| `userName` | String | false | " " (empty string) | The username used to connect to the database specified by `jdbcUrl`.<br><br>**Note: `userName` is case-sensitive.** |
+| `password` | String | false | " " (empty string) | The password used to connect to the database specified by `jdbcUrl`. <br><br>**Note: `password` is case-sensitive.** |
+| `jdbcUrl` | String | true | " " (empty string) | The JDBC URL of the database to which the connector connects. |
+| `tableName` | String | true | " " (empty string) | The name of the table to which the connector writes. |
+| `nonKey` | String | false | " " (empty string) | A comma-separated list contains the fields used in updating events. |
+| `key` | String | false | " " (empty string) | A comma-separated list contains the fields used in `where` condition of updating and deleting events. |
Review Comment:
same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] vagetablechicken commented on pull request #15064: [feat][connector] Support 4paradigm/OpenMLDB jdbc sink connector
Posted by GitBox <gi...@apache.org>.
vagetablechicken commented on PR #15064:
URL: https://github.com/apache/pulsar/pull/15064#issuecomment-1095930351
> While testing, I found this exception when starting the connector. Could you please take a look? @vagetablechicken
>
> ```
> 2022-04-11T15:58:55,371+0800 [function-timer-thread-17-1] ERROR org.apache.pulsar.functions.runtime.RuntimeSpawner - public/default/jdbc-openmldb-sink Function Container is dead with following exception. Restarting.
> java.lang.UnsatisfiedLinkError: 'long com._4paradigm.openmldb.sql_router_sdkJNI.new_SQLRouterOptions()'
> at com._4paradigm.openmldb.sql_router_sdkJNI.new_SQLRouterOptions(Native Method) ~[?:?]
> at com._4paradigm.openmldb.SQLRouterOptions.<init>(SQLRouterOptions.java:56) ~[?:?]
> at com._4paradigm.openmldb.sdk.impl.SqlClusterExecutor.<init>(SqlClusterExecutor.java:66) ~[?:?]
> at com._4paradigm.openmldb.sdk.impl.SqlClusterExecutor.<init>(SqlClusterExecutor.java:90) ~[?:?]
> at com._4paradigm.openmldb.jdbc.SQLDriver.connect(SQLDriver.java:54) ~[?:?]
> at java.sql.DriverManager.getConnection(DriverManager.java:677) ~[java.sql:?]
> at java.sql.DriverManager.getConnection(DriverManager.java:189) ~[java.sql:?]
> at org.apache.pulsar.io.jdbc.JdbcAbstractSink.open(JdbcAbstractSink.java:93) ~[?:?]
> at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupOutput(JavaInstanceRunnable.java:857) ~[classes/:?]
> at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:223) ~[classes/:?]
> at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:258) ~[classes/:?]
> at java.lang.Thread.run(Thread.java:829) [?:?]
> ```
>
> Full exception context
Fixed, use openmldb-native allinone version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] RobertIndie merged pull request #15064: [feat][connector] Support 4paradigm/OpenMLDB jdbc sink connector
Posted by GitBox <gi...@apache.org>.
RobertIndie merged PR #15064:
URL: https://github.com/apache/pulsar/pull/15064
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org