You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "Sandman-T (via GitHub)" <gi...@apache.org> on 2025/07/03 03:13:06 UTC
[I] [Bug] [Module Name] vertica sink 使用generate_sink_sql无法通过主键进行更新 [seatunnel]
Sandman-T opened a new issue, #9534:
URL: https://github.com/apache/seatunnel/issues/9534
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
sink config中:
generate_sink_sql = true
primary_keys = ["id"]
vertica jdbc使用8.x、12.x、24.x,都会在实际插入数据时,不管主键是否存在重复值,都会报错:
Caused by: com.vertica.util.ClientSyntaxErrorException: [Vertica][VJDBC](2848) ERROR: Could not determine data type of parameter $1
上述报错是在将表中字段全部改成string之后的,若是表中存在int类型或date类型字段,报错稍有不同:
Caused by: java.sql.SQLException: [Vertica][VJDBC](https://github.com/apache/seatunnel/3376) ERROR: Failed to find conversion function from unknown to date
或
Caused by: com.vertica.support.exceptions.ErrorException: [Vertica][VJDBC](https://github.com/apache/seatunnel/3376) ERROR: Failed to find conversion function from unknown to int
### SeaTunnel Version
2.3.9
### SeaTunnel Config
```conf
env {
parallelism = 1
checkpoint.interval = 120000
job.name= "kafka-seatunnel-vertia"
job.mode= "STREAMING"
}
source {
kafka {
plugin_output = "kafka_out"
bootstrap.servers = "127.0.0.1:9092,127.0.0.2:9092,127.0.0.3:9092"
topic = "input-test"
consumer.group = "test-group"
format = json
partition-discovery.interval-millis = 60000
commit_on_checkpoint = true
start_mode = group_offsets
kafka.config = {
client.id = test-group
max.poll.records = 10
auto.offset.reset = "latest"
enable.auto.commit = "false"
}
schema = {
fields {
id = "string"
gtoken = "string"
gname = "string"
device_score = "string"
device_level = "string"
detecttime = "string"
}
}
}
}
sink {
jdbc {
plugin_input = "kafka_out"
url = "jdbc:vertica://127.0.0.1:5433/dev"
driver = "com.vertica.jdbc.Driver"
user = "xxxxxxxx"
password = "xxxxxxxxx"
generate_sink_sql = true
primary_keys = ["id"]
database = "tmp"
table = "test_table"
batch_size = 1
batch_interval_ms = 30000
}
}
```
### Running Command
```shell
./bin/seatunnel.sh --config ./config/kafka.template -m local
```
### Error Exception
```log
org.apache.seatunnel.connectors.seatunnel.jdbc.exception.JdbcConnectorException: ErrorCode:[COMMON-08], ErrorDescription:[Sql operation failed, such as (execute,addBatch,close) etc...] - unable to open JDBC writer
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.createAndOpenStatementExecutor(JdbcOutputFormat.java:107) ~[connector-jdbc-2.3.9-2.12.15-em.jar:2.3.9]
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.open(JdbcOutputFormat.java:79) ~[connector-jdbc-2.3.9-2.12.15-em.jar:2.3.9]
at org.apache.seatunnel.connectors.seatunnel.jdbc.sink.JdbcSinkWriter.tryOpen(JdbcSinkWriter.java:122) ~[connector-jdbc-2.3.9-2.12.15-em.jar:2.3.9]
at org.apache.seatunnel.connectors.seatunnel.jdbc.sink.JdbcSinkWriter.write(JdbcSinkWriter.java:133) ~[connector-jdbc-2.3.9-2.12.15-em.jar:2.3.9]
at org.apache.seatunnel.connectors.seatunnel.jdbc.sink.JdbcSinkWriter.write(JdbcSinkWriter.java:43) ~[connector-jdbc-2.3.9-2.12.15-em.jar:2.3.9]
at org.apache.seatunnel.api.sink.multitablesink.MultiTableWriterRunnable.run(MultiTableWriterRunnable.java:67) [seatunnel-starter.jar:2.3.9]
at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:43) [seatunnel-starter.jar:2.3.9]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_111]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_111]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_111]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_111]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]
Caused by: java.sql.SQLSyntaxErrorException: [Vertica][VJDBC](2848) ERROR: Could not determine data type of parameter $1
at com.vertica.util.ServerErrorData.buildException(Unknown Source) ~[vertica-jdbc-24.2.0-1.jar:?]
at com.vertica.io.ProtocolStream.readExpectedMessage(Unknown Source) ~[vertica-jdbc-24.2.0-1.jar:?]
at com.vertica.dataengine.VDataEngine.prepareImpl(Unknown Source) ~[vertica-jdbc-24.2.0-1.jar:?]
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source) ~[vertica-jdbc-24.2.0-1.jar:?]
at com.vertica.dataengine.VDataEngine.prepare(Unknown Source) ~[vertica-jdbc-24.2.0-1.jar:?]
at com.vertica.jdbc.common.SPreparedStatement.<init>(SPreparedStatement.java:310) ~[vertica-jdbc-24.2.0-1.jar:?]
at com.vertica.jdbc.jdbc41.S41PreparedStatement.<init>(S41PreparedStatement.java:91) ~[vertica-jdbc-24.2.0-1.jar:?]
at com.vertica.jdbc.jdbc42.S42PreparedStatement.<init>(S42PreparedStatement.java:77) ~[vertica-jdbc-24.2.0-1.jar:?]
at com.vertica.jdbc.VerticaJdbc42PreparedStatementImpl.<init>(Unknown Source) ~[vertica-jdbc-24.2.0-1.jar:?]
at com.vertica.jdbc.VJDBCObjectFactory.createPreparedStatement(Unknown Source) ~[vertica-jdbc-24.2.0-1.jar:?]
at com.vertica.jdbc.common.SConnection.prepareStatement(SConnection.java:1309) ~[vertica-jdbc-24.2.0-1.jar:?]
at com.vertica.jdbc.common.SConnection.prepareStatement(SConnection.java:1263) ~[vertica-jdbc-24.2.0-1.jar:?]
at org.apache.seatunnel.shade.com.zaxxer.hikari.pool.ProxyConnection.prepareStatement(ProxyConnection.java:337) ~[connector-jdbc-2.3.9-2.12.15-em.jar:2.3.9]
at org.apache.seatunnel.shade.com.zaxxer.hikari.pool.HikariProxyConnection.prepareStatement(HikariProxyConnection.java) ~[connector-jdbc-2.3.9-2.12.15-em.jar:2.3.9]
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.FieldNamedPreparedStatement.prepareStatement(FieldNamedPreparedStatement.java:673) ~[connector-jdbc-2.3.9-2.12.15-em.jar:2.3.9]
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormatBuilder.lambda$createSimpleExecutor$6(JdbcOutputFormatBuilder.java:348) ~[connector-jdbc-2.3.9-2.12.15-em.jar:2.3.9]
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.SimpleBatchStatementExecutor.prepareStatements(SimpleBatchStatementExecutor.java:43) ~[connector-jdbc-2.3.9-2.12.15-em.jar:2.3.9]
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.executor.BufferReducedBatchStatementExecutor.prepareStatements(BufferReducedBatchStatementExecutor.java:50) ~[connector-jdbc-2.3.9-2.12.15-em.jar:2.3.9]
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcOutputFormat.createAndOpenStatementExecutor(JdbcOutputFormat.java:105) ~[connector-jdbc-2.3.9-2.12.15-em.jar:2.3.9]
... 11 more
```
### Zeta or Flink or Spark Version
zeta
### Java or Scala Version
_No response_
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Module Name] vertica sink 使用generate_sink_sql无法通过主键进行更新 [seatunnel]
Posted by "corgy-w (via GitHub)" <gi...@apache.org>.
corgy-w closed issue #9534: [Bug] [Module Name] vertica sink 使用generate_sink_sql无法通过主键进行更新
URL: https://github.com/apache/seatunnel/issues/9534
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org