You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/10/14 03:16:45 UTC
[GitHub] [incubator-seatunnel] matesoul opened a new issue, #3095: [Bug] [Module Name] Bug title
matesoul opened a new issue, #3095:
URL: https://github.com/apache/incubator-seatunnel/issues/3095
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
When I use flink-jdbc sink to write hive, Throw me an error：
No suitable driver found for jdbc:hive2://192.168.1.1:10000/default
I am sure hive-jdbc-{version}.jar file is in ${FLINK_HOME}/lib dir
### SeaTunnel Version
2.1.3
### SeaTunnel Config
```conf
env {
parallelism.default = 2
execution.checkpointing.interval = 1000
state.checkpoints.dir = "hdfs://nameservice1/seatunnel-task/checkpoint"
execution.checkpointing.timeout = 1500
}
source {
JdbcSource {
driver = "com.mysql.cj.jdbc.Driver"
url = "jdbc:mysql://192.168.30.48:3306/y_test"
username = "root"
password = "Root@123"
query = "select id,goodsid,goodsname from goods_source where id > ${id}"
partition_column = "id"
result_table_name = "goods_source"
parallelism = "1"
}
}
transform {
sql {
source_table_name = goods_source
sql = "select id as id,goodsid as goodid,goodsname as goodname from goods_source"
result_table_name = transform_result_goods_source
}
}
sink {
JdbcSink {
driver = org.apache.hive.jdbc.HiveDriver
url = "jdbc:hive2://k8s003:10000/default"
username = "hive"
password = "hive"
query = "INSERT INTO goods (id, goodid, goodname) VALUES(?,?,?)"
batch_size = "1000"
source_table_name = "transform_result_goods_source"
pre_sql = "truncate table goods"
ignore_post_sql_exceptions = "false"
parallelism = "1"
}
}
```
### Running Command
```shell
./bin/start-seatunnel-sql.sh -t yarn-per-job \
--detached \
--config ./config/flink.sql.mysql-hive.conf \
-Dyarn.application.name=test-mysql-hive-batch \
-Dparallelism.default=2 \
-Djobmanager.memory.process.size=1024mb \
-Dtaskmanager.memory.process.size=1024mb
```
### Error Exception
```log
org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Execute Flink task error
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:812)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:246)
at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)
at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
Caused by: org.apache.seatunnel.core.base.exception.CommandExecuteException: Execute Flink task error
at org.apache.seatunnel.core.flink.command.FlinkApiTaskExecuteCommand.execute(FlinkApiTaskExecuteCommand.java:85)
at org.apache.seatunnel.core.base.Seatunnel.run(Seatunnel.java:40)
at org.apache.seatunnel.core.flink.SeatunnelFlink.main(SeatunnelFlink.java:34)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
... 11 more
Caused by: java.lang.RuntimeException: java.sql.SQLException: No suitable driver found for jdbc:hive2://192.168.1.1:10000/default
at org.apache.seatunnel.flink.jdbc.sink.JdbcSink.executePreSql(JdbcSink.java:197)
at org.apache.seatunnel.flink.jdbc.sink.JdbcSink.outputBatch(JdbcSink.java:167)
at org.apache.seatunnel.flink.batch.FlinkBatchExecution.start(FlinkBatchExecution.java:71)
at org.apache.seatunnel.core.flink.command.FlinkApiTaskExecuteCommand.execute(FlinkApiTaskExecuteCommand.java:82)
... 18 more
Caused by: java.sql.SQLException: No suitable driver found for jdbc:hive2://192.168.1.1:10000/default
at java.sql.DriverManager.getConnection(DriverManager.java:689)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.seatunnel.flink.jdbc.sink.JdbcSink.executeSql(JdbcSink.java:217)
at org.apache.seatunnel.flink.jdbc.sink.JdbcSink.executePreSql(JdbcSink.java:194)
... 21 more
```
### Flink or Spark Version
flink 1.13.6
### Java or Scala Version
idk 1.8
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] closed issue #3095: [Bug] [seatunnel-connector-flink-jdbc] No suitable driver found for hive2
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #3095: [Bug] [seatunnel-connector-flink-jdbc] No suitable driver found for hive2
URL: https://github.com/apache/incubator-seatunnel/issues/3095
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] matesoul commented on issue #3095: [Bug] [seatunnel-connector-flink-jdbc] No suitable driver found for hive2
Posted by GitBox <gi...@apache.org>.
matesoul commented on issue #3095:
URL: https://github.com/apache/incubator-seatunnel/issues/3095#issuecomment-1278753638
Tips：hive veriosn is 1.1.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #3095: [Bug] [seatunnel-connector-flink-jdbc] No suitable driver found for hive2
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #3095:
URL: https://github.com/apache/incubator-seatunnel/issues/3095#issuecomment-1321292332
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] github-actions[bot] commented on issue #3095: [Bug] [seatunnel-connector-flink-jdbc] No suitable driver found for hive2
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #3095:
URL: https://github.com/apache/incubator-seatunnel/issues/3095#issuecomment-1312875461
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] CalvinKirs commented on issue #3095: [Bug] [seatunnel-connector-flink-jdbc] No suitable driver found for hive2
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on issue #3095:
URL: https://github.com/apache/incubator-seatunnel/issues/3095#issuecomment-1278755295
As a reminder, sensitive information is best hidden
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org