You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/07/08 08:46:28 UTC
[GitHub] [incubator-seatunnel] h472591847 opened a new issue, #2152: I ran into problems when deploying JdbcSource to collect data in kubernetes
h472591847 opened a new issue, #2152:
URL: https://github.com/apache/incubator-seatunnel/issues/2152
### Search before asking
- [X] I had searched in the [feature](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
### Description
I ran into problems when deploying JdbcSource to collect data in kubernetes,
Is it related to JDBC connector dependencies? How do I add JDBC dependencies
`2022-07-08 08:24:31,585 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - Calculating tasks to restart to recover the failed task c83bdbe46e4f0685457e9e7370d7e8d3_0.
2022-07-08 08:24:31,587 INFO org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy [] - 2 tasks should be restarted to recover the failed task c83bdbe46e4f0685457e9e7370d7e8d3_0.
2022-07-08 08:24:31,589 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Job seatunnel (3d9899f7326b0bbd707daa6252f46fc8) switched from state RUNNING to FAILING.
org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:138) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:82) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:216) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:206) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:197) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:682) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:79) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:435) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_332]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_332]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_332]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_332]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) [flink-dist_2.12-1.13.6.jar:1.13.6]
at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [flink-dist_2.12-1.13.6.jar:1.13.6]
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) [flink-dist_2.12-1.13.6.jar:1.13.6]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.12-1.13.6.jar:1.13.6]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-dist_2.12-1.13.6.jar:1.13.6]
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.actor.Actor.aroundReceive(Actor.scala:517) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.actor.Actor.aroundReceive$(Actor.scala:515) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.12-1.13.6.jar:1.13.6]
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.12-1.13.6.jar:1.13.6]
Caused by: java.lang.IllegalStateException: The library registration references a different set of library BLOBs than previous registrations for this job:
old:[file:/opt/seatunnel/connectors/flink/seatunnel-connector-flink-jdbc-2.1.2.jar]
new:[file:/opt/seatunnel/connectors/flink/seatunnel-connector-flink-jdbc-2.1.2.jar, file:/opt/seatunnel/connectors/flink/seatunnel-connector-flink-jdbc-2.1.2.jar]
at org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$ResolvedClassLoader.verifyClassLoader(BlobLibraryCacheManager.java:431) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$ResolvedClassLoader.access$500(BlobLibraryCacheManager.java:356) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$LibraryCacheEntry.getOrResolveClassLoader(BlobLibraryCacheManager.java:232) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$LibraryCacheEntry.access$1100(BlobLibraryCacheManager.java:199) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.execution.librarycache.BlobLibraryCacheManager$DefaultClassLoaderLease.getOrResolveClassLoader(BlobLibraryCacheManager.java:333) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.taskmanager.Task.createUserCodeClassloader(Task.java:1024) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:628) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566) ~[flink-dist_2.12-1.13.6.jar:1.13.6]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_332]
2022-07-08 08:24:31,592 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - CHAIN DataSource (at createInput(ExecutionEnvironment.java:655) (org.apache.seatunnel.flink.jdbc.input.JdbcInputFormat)) -> Map (to: Row) (1/1) (cafd9fcbc31e56685edd718f809a8593) switched from RUNNING to CANCELING.`
flink.streaming.conf
` env {
execution.parallelism = 1
}
source {
JdbcSource {
driver = com.mysql.cj.jdbc.Driver
url = "jdbc:mysql://*.*.*.*:3306/test"
username = root
password = xxxxxx
query = "select nick,pass from users_source"
result_table_name = "users_source"
}
}
transform {
sql {
sql = "select nick,pass from users_source"
}
}
sink {
JdbcSink {
source_table_name = users_source
driver = "com.mysql.cj.jdbc.Driver"
url = "jdbc:mysql://*.*.*.*:3306/test"
username = root
password = xxxxxx
query = "insert into users_sink(nick,pass) values(?,?)"
batch_size = 2
}
}`
### Usage Scenario
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] h472591847 commented on issue #2152: I ran into problems when deploying JdbcSource to collect data in kubernetes
Posted by GitBox <gi...@apache.org>.
h472591847 commented on issue #2152:
URL: https://github.com/apache/incubator-seatunnel/issues/2152#issuecomment-1184156354
> Hi, this is bug in 2.1.2. already fixed in dev. You can try use dev version
THX,I'm going to have a try at it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on issue #2152: I ran into problems when deploying JdbcSource to collect data in kubernetes
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on issue #2152:
URL: https://github.com/apache/incubator-seatunnel/issues/2152#issuecomment-1182686733
Hi, this is bug in 2.1.2. already fixed in dev. You can try use dev version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X closed issue #2152: I ran into problems when deploying JdbcSource to collect data in kubernetes
Posted by GitBox <gi...@apache.org>.
Hisoka-X closed issue #2152: I ran into problems when deploying JdbcSource to collect data in kubernetes
URL: https://github.com/apache/incubator-seatunnel/issues/2152
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] adudu137 commented on issue #2152: I ran into problems when deploying JdbcSource to collect data in kubernetes
Posted by GitBox <gi...@apache.org>.
adudu137 commented on issue #2152:
URL: https://github.com/apache/incubator-seatunnel/issues/2152#issuecomment-1178748819
报错的信息直接翻译过来是，你现在注册的比之前注册的多一个jar包，是不是作业配置多配置了一个重复路径
Caused by: java.lang.IllegalStateException: The library registration references a different set of library BLOBs than previous registrations for this job:
old:[file:/opt/seatunnel/connectors/flink/seatunnel-connector-flink-jdbc-2.1.2.jar]
new:[file:/opt/seatunnel/connectors/flink/seatunnel-connector-flink-jdbc-2.1.2.jar, file:/opt/seatunnel/connectors/flink/seatunnel-connector-flink-jdbc-2.1.2.jar]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org