You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2022/05/06 02:03:20 UTC
[GitHub] [shardingsphere] nickfan commented on issue #17299: ShardingSphere-Proxy Unable to work with apache flink cdc connector.
nickfan commented on issue #17299:
URL: https://github.com/apache/shardingsphere/issues/17299#issuecomment-1119195695
> Hi @nickfan The support for `information_schema` is being improved, please pay attention to the subsequent version releases. Related issue: #16234
i upgraded shardingsphere to 5.1.1, still cause the same issue (with information_schema included):
```log
jobmanager_1 | 2022-05-06T01:54:19.211855085Z 2022-05-06 01:54:19,211 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - Starting execution of job insert-into_myhive.default.es_spl_queue_log (3b8fad9f3ed3a1d1d6ba4cb09583f2f9) under job master id 00000000000000000000000000000000.
jobmanager_1 | 2022-05-06T01:54:19.213168090Z 2022-05-06 01:54:19,213 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Starting split enumerator for source Source: TableSourceScan(table=[[myhive, default, cdc_init_spl_queue_log]], fields=[id, org_id, spl_order_id, tracking_code]) -> DropUpdateBefore.
jobmanager_1 | 2022-05-06T01:54:19.219152022Z 2022-05-06 01:54:19,219 INFO com.ververica.cdc.connectors.mysql.source.connection.JdbcConnectionPools [] - Create and register connection pool 172.16.0.33:3308
jobmanager_1 | 2022-05-06T01:54:19.220544261Z 2022-05-06 01:54:19,220 INFO com.ververica.cdc.connectors.shaded.com.zaxxer.hikari.HikariDataSource [] - connection-pool-172.16.0.33:3308 - Starting...
jobmanager_1 | 2022-05-06T01:54:19.306373014Z 2022-05-06 01:54:19,306 INFO com.ververica.cdc.connectors.shaded.com.zaxxer.hikari.HikariDataSource [] - connection-pool-172.16.0.33:3308 - Start completed.
jobmanager_1 | 2022-05-06T01:54:19.341044487Z 2022-05-06 01:54:19,340 INFO io.debezium.jdbc.JdbcConnection [] - Connection gracefully closed
jobmanager_1 | 2022-05-06T01:54:19.341182589Z 2022-05-06 01:54:19,341 INFO com.ververica.cdc.connectors.mysql.MySqlValidator [] - MySQL validation passed.
jobmanager_1 | 2022-05-06T01:54:19.348825206Z 2022-05-06 01:54:19,348 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - Read list of available databases
jobmanager_1 | 2022-05-06T01:54:19.351583669Z 2022-05-06 01:54:19,351 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - list of available databases is: [sharding_db, mysql, information_schema, performance_schema, sys]
jobmanager_1 | 2022-05-06T01:54:19.351597610Z 2022-05-06 01:54:19,351 INFO com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - Read list of available tables in each database
jobmanager_1 | 2022-05-06T01:54:19.369729366Z 2022-05-06 01:54:19,369 WARN com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - skipping database 'sharding_db' due to error reading tables: No database selected
jobmanager_1 | 2022-05-06T01:54:19.371206896Z 2022-05-06 01:54:19,371 WARN com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - skipping database 'mysql' due to error reading tables: No database selected
jobmanager_1 | 2022-05-06T01:54:19.372318346Z 2022-05-06 01:54:19,372 WARN com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - skipping database 'information_schema' due to error reading tables: No database selected
jobmanager_1 | 2022-05-06T01:54:19.373209226Z 2022-05-06 01:54:19,373 WARN com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - skipping database 'performance_schema' due to error reading tables: No database selected
jobmanager_1 | 2022-05-06T01:54:19.374866886Z 2022-05-06 01:54:19,374 WARN com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - skipping database 'sys' due to error reading tables: No database selected
jobmanager_1 | 2022-05-06T01:54:19.376215140Z 2022-05-06 01:54:19,376 INFO io.debezium.jdbc.JdbcConnection [] - Connection gracefully closed
jobmanager_1 | 2022-05-06T01:54:19.381521176Z 2022-05-06 01:54:19,378 ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Failed to create Source Enumerator for source Source: TableSourceScan(table=[[myhive, default, cdc_init_spl_queue_log]], fields=[id, org_id, spl_order_id, tracking_code]) -> DropUpdateBefore
jobmanager_1 | 2022-05-06T01:54:19.381544918Z org.apache.flink.util.FlinkRuntimeException: Failed to discover captured tables for enumerator
jobmanager_1 | 2022-05-06T01:54:19.381548337Z at com.ververica.cdc.connectors.mysql.source.MySqlSource.createEnumerator(MySqlSource.java:170) ~[flink-sql-connector-mysql-cdc-2.1.0.jar:2.1.0]
jobmanager_1 | 2022-05-06T01:54:19.381551337Z at org.apache.flink.runtime.source.coordinator.SourceCoordinator.start(SourceCoordinator.java:128) ~[flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381554248Z at org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$DeferrableCoordinator.applyCall(RecreateOnResetOperatorCoordinator.java:291) ~[flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381557585Z at org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator.start(RecreateOnResetOperatorCoordinator.java:70) ~[flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381560391Z at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder.start(OperatorCoordinatorHolder.java:194) ~[flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381563002Z at org.apache.flink.runtime.scheduler.DefaultOperatorCoordinatorHandler.startAllOperatorCoordinators(DefaultOperatorCoordinatorHandler.java:85) ~[flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381565617Z at org.apache.flink.runtime.scheduler.SchedulerBase.startScheduling(SchedulerBase.java:592) ~[flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381568183Z at org.apache.flink.runtime.jobmaster.JobMaster.startScheduling(JobMaster.java:955) ~[flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381570771Z at org.apache.flink.runtime.jobmaster.JobMaster.startJobExecution(JobMaster.java:873) ~[flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381573286Z at org.apache.flink.runtime.jobmaster.JobMaster.onStart(JobMaster.java:383) ~[flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381575813Z at org.apache.flink.runtime.rpc.RpcEndpoint.internalCallOnStart(RpcEndpoint.java:181) ~[flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381579351Z at org.apache.flink.runtime.rpc.akka.AkkaRpcActor$StoppedState.start(AkkaRpcActor.java:605) ~[flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381582075Z at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleControlMessage(AkkaRpcActor.java:180) ~[flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381584698Z at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381587246Z at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381589746Z at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381592192Z at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381604319Z at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381607039Z at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381609600Z at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381612323Z at akka.actor.Actor.aroundReceive(Actor.scala:517) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381614900Z at akka.actor.Actor.aroundReceive$(Actor.scala:515) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381617444Z at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381619887Z at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381622342Z at akka.actor.ActorCell.invoke(ActorCell.scala:561) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381624803Z at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381627320Z at akka.dispatch.Mailbox.run(Mailbox.scala:225) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381629931Z at akka.dispatch.Mailbox.exec(Mailbox.scala:235) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381632520Z at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381635121Z at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381637742Z at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381640473Z at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [flink-dist_2.12-1.13.3.jar:1.13.3]
jobmanager_1 | 2022-05-06T01:54:19.381643027Z Caused by: java.lang.IllegalArgumentException: Can't find any matched tables, please check your configured database-name: [sharding_db] and table-name: [sharding_db.spl_queue_log]
jobmanager_1 | 2022-05-06T01:54:19.381645744Z at com.ververica.cdc.connectors.mysql.debezium.DebeziumUtils.discoverCapturedTables(DebeziumUtils.java:167) ~[flink-sql-connector-mysql-cdc-2.1.0.jar:2.1.0]
jobmanager_1 | 2022-05-06T01:54:19.381648582Z at com.ververica.cdc.connectors.mysql.source.MySqlSource.createEnumerator(MySqlSource.java:161) ~[flink-sql-connector-mysql-cdc-2.1.0.jar:2.1.0]
jobmanager_1 | 2022-05-06T01:54:19.381651603Z ... 31 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org