You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "AntiBit82 (via GitHub)" <gi...@apache.org> on 2023/03/31 14:49:21 UTC
[GitHub] [pulsar] AntiBit82 opened a new issue, #19986: [Bug] JdbcSink: NullPointerException for PreparedStatement when updating/deleting record
AntiBit82 opened a new issue, #19986:
URL: https://github.com/apache/pulsar/issues/19986
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
Pulsar 2.11
JDBC Sink Connecrtor 2.11
OS: MacOS Ventura
### Minimal reproduce step
Simple class Customer:
`public class Customer {
private int id;
private String name;
private String address;
private int age;
public Customer() {}
public Customer(int id, String name, String address, int age) {
this.id = id;
this.name = name;
this.address = address;
this.age = age;
}
//getters/setters/toString, etc.
}
`
I have a Postgres DB running with resp. customer table.
In Pulsar standalone I create a JDBC PostgreSQL sink:
`bin/pulsar-admin sinks create --sink-config-file conf/sinkcustomer.yaml`
Here's the config:
`tenant: "mynamespace"
namespace: "test"
name: "customer-sink"
inputs: [ "persistent://mynamespace/test/customer" ]
key: "id"
nonKey: "name,address,age"
sinkType: "jdbc-postgres"
configs:
userName: "postgres"
password: "postgres"
jdbcUrl: "jdbc:postgresql://localhost:5432/postgres"
tableName: "customer"`
Now produce some events:
`
Producer<Customer> pc = ...
pc.newMessage(Schema.AVRO(Customer.class)).value(new Customer(1,"XXX","Snowy Mountain 1",40)).send();
pc.newMessage(Schema.AVRO(Customer.class)).value(new Customer(1, "YYY", "Sunny Greek Beach", 40))
.property("ACTION","DELETE").send();
// You could also use "UPDATE", same problem will appear
`
First insert of the customer works fine.
Second delete / update throws NPE.
### What did you expect to see?
Expect the customer to be updated or deleted
### What did you see instead?
`[pool-5-thread-1] ERROR org.apache.pulsar.io.jdbc.JdbcAbstractSink - Got exception
java.lang.NullPointerException: Cannot invoke "java.sql.PreparedStatement.execute()" because "this.deleteStatement" is null
at org.apache.pulsar.io.jdbc.JdbcAbstractSink.flush(JdbcAbstractSink.java:241) ~[pulsar-io-jdbc-core-2.11.0.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at java.lang.Thread.run(Thread.java:833) ~[?:?]
`
or same message with 'this.updateStatement is null'
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] AntiBit82 commented on issue #19986: [Bug] JdbcSink: NullPointerException for PreparedStatement when updating/deleting record
Posted by "AntiBit82 (via GitHub)" <gi...@apache.org>.
AntiBit82 commented on issue #19986:
URL: https://github.com/apache/pulsar/issues/19986#issuecomment-1492119550
> it must go under the `.configs` section. Also `nonKey`
>
> ```
> tenant: "mynamespace"
> namespace: "test"
> name: "customer-sink"
> inputs: [ "persistent://mynamespace/test/customer" ]
> sinkType: "jdbc-postgres"
> configs:
> userName: "postgres"
> password: "postgres"
> jdbcUrl: "jdbc:postgresql://localhost:5432/postgres"
> tableName: "customer"
> key: "id"
> nonKey: "name,address,age"
> ```
Ahh wrong section, thanks! I will try that and report back
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] AntiBit82 closed issue #19986: [Bug] JdbcSink: NullPointerException for PreparedStatement when updating/deleting record
Posted by "AntiBit82 (via GitHub)" <gi...@apache.org>.
AntiBit82 closed issue #19986: [Bug] JdbcSink: NullPointerException for PreparedStatement when updating/deleting record
URL: https://github.com/apache/pulsar/issues/19986
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] AntiBit82 commented on issue #19986: [Bug] JdbcSink: NullPointerException for PreparedStatement when updating/deleting record
Posted by "AntiBit82 (via GitHub)" <gi...@apache.org>.
AntiBit82 commented on issue #19986:
URL: https://github.com/apache/pulsar/issues/19986#issuecomment-1492132463
@nicoloboschi
Can confirm, now it's working as intended!
Maybe you could add an error message and also update your docs on the website with an example, since these 2 properties are common between all the JDBCSinks I thought it had to go in the common upper section of the config.
Anyways, thx for the quick help!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nicoloboschi commented on issue #19986: [Bug] JdbcSink: NullPointerException for PreparedStatement when updating/deleting record
Posted by "nicoloboschi (via GitHub)" <gi...@apache.org>.
nicoloboschi commented on issue #19986:
URL: https://github.com/apache/pulsar/issues/19986#issuecomment-1492080642
@AntiBit82 thanks for reporting the issue, we'll have to add an error message.
You need to set `key` in the config containing the primary key columns of the table (separated by a comma if multiple columns)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nicoloboschi commented on issue #19986: [Bug] JdbcSink: NullPointerException for PreparedStatement when updating/deleting record
Posted by "nicoloboschi (via GitHub)" <gi...@apache.org>.
nicoloboschi commented on issue #19986:
URL: https://github.com/apache/pulsar/issues/19986#issuecomment-1492117786
it must go under the `.configs` section. Also `nonKey`
```
tenant: "mynamespace"
namespace: "test"
name: "customer-sink"
inputs: [ "persistent://mynamespace/test/customer" ]
sinkType: "jdbc-postgres"
configs:
userName: "postgres"
password: "postgres"
jdbcUrl: "jdbc:postgresql://localhost:5432/postgres"
tableName: "customer"
key: "id"
nonKey: "name,address,age"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] AntiBit82 commented on issue #19986: [Bug] JdbcSink: NullPointerException for PreparedStatement when updating/deleting record
Posted by "AntiBit82 (via GitHub)" <gi...@apache.org>.
AntiBit82 commented on issue #19986:
URL: https://github.com/apache/pulsar/issues/19986#issuecomment-1492114108
> @AntiBit82 thanks for reporting the issue, we'll have to add an error message. You need to set `key` in the config containing the primary key columns of the table (separated by a comma if multiple columns)
The pk is there, you can see it in the config above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org