You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "zengqinchris (via GitHub)" <gi...@apache.org> on 2023/04/24 08:34:24 UTC
[GitHub] [incubator-seatunnel] zengqinchris opened a new issue, #4656: [Bug] [connector-jdbc] Specify offline mode to execute discovery as a streaming task
zengqinchris opened a new issue, #4656:
URL: https://github.com/apache/incubator-seatunnel/issues/4656
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
Specify offline mode to execute discovery as a streaming task, Perform approximately 10 tasks, which may occur once
### SeaTunnel Version
v2.3.1
### SeaTunnel Config
```conf
{
"env" : {
"execution.parallelism" : 1,
"job.mode" : "BATCH"
},
"source" : [
{
"password" : "111",
"driver" : "com.mysql.cj.jdbc.Driver",
"query" : " select id,use_name,use_pwd,load_ts from xxx where 1=1",
"connection_check_timeout_sec" : 1000,
"plugin_name" : "Jdbc",
"user" : "11",
"url" : "jdbc:mysql://0.0.0.0:3306/wcscreen"
}
],
"transform" : [],
"sink" : [
{
"password" : "111",
"driver" : "com.mysql.cj.jdbc.Driver",
"max_retries" : 3,
"query" : " replace into xxx(id,use_name,use_pwd,load_ts) values (?,?,?,?) ",
"plugin_name" : "jdbc",
"user" : "11",
"url" : "jdbc:mysql://0.0.0.0:3306/ydtest"
}
]
}
```
### Running Command
```shell
./bin/seatunnel.sh --config ./config/bb.conf
```
### Error Exception
```log
2023-04-24 15:41:34.041 INFO Program is substituting variables for you
2023-04-24 15:41:34.041 INFO Variables substitution ended successfully
2023-04-24 15:41:34.041 WARN The code you submit will not be limited by the limit
Job with jobId : nodeexecution_hadoop_ydstnnl_59 and execID : nodeexecution_hadoop_ydstnnl_59 submitted
2023-04-24 15:41:34.041 INFO You have submitted a new job, script code (after variable substitution) is
************************************SCRIPT CODE************************************
{"taskId":9,"script":"/YdBigData/user/seatunnel/config/9_319_MySQL_MySQL.conf","scriptType":"prod","source":"MySQL","sink":"MySQL","taskName":"m2m4","remark":"r","createTime":"2023-04-20 20:44:34","publishTime":"2023-04-20 20:59:17","projectId":"319","flowId":336,"projectName":"lz_project","flowName":"m2m4","nodeKey":"3ea09f0d-abd8-449d-b2c1-d92040f571ad"}
************************************SCRIPT CODE************************************
2023-04-24 15:41:34.041 INFO Your job is accepted, jobID is exec_id018014linkis-cg-entrancebigdata08:9104nodeexecution_hadoop_ydstnnl_59 and taskID is 1420 in ServiceInstance(linkis-cg-entrance, bigdata08:9104). Please wait it to be scheduled
job is scheduled.
2023-04-24 15:41:34.041 INFO Your job is Scheduled. Please wait it to run.
Your job is being scheduled by orchestrator.
2023-04-24 15:41:34.041 INFO job is running.
2023-04-24 15:41:34.041 INFO Your job is Running now. Please wait it to complete.
2023-04-24 15:41:34.041 INFO Job with jobGroupId : 1420 and subJobId : 1420 was submitted to Orchestrator.
2023-04-24 15:41:34.041 INFO Background is starting a new engine for you,execId astJob_136_codeExec_136 mark id is mark_136, it may take several seconds, please wait
2023-04-24 15:41:34.041 INFO EngineConn local log path: ServiceInstance(linkis-cg-engineconn, bigdata08:46624) /bigdata/datahorizon/application/linkis/dirs/hadoop/20230424/ydstnnl/14de5239-f2b5-41d5-98d1-e786df3979d1/logs
bigdata08:46624_0 >> /opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/bin/seatunnel.sh --config /appcom/tmp/ydstnnlConfDir/9_319_MySQL_MySQL.conf
JAVA_OPTS: -Dhazelcast.client.config=/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/config/hazelcast-client.yaml -Dseatunnel.config=/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/config/seatunnel.yaml -Dhazelcast.config=/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/config/hazelcast.yaml -Dlog4j2.configurationFile=/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/config/log4j2_client.properties -Dseatunnel.logs.path=/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/logs -Dseatunnel.logs.file_name=seatunnel-starter-client -Xms128m -Xmx512m -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/tmp/seatunnel-6639055162999374831 -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/data/seatunnel/
Apr 24, 2023 3:41:35 PM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Loading configuration '/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/config/seatunnel.yaml' from System property 'seatunnel.config'
Apr 24, 2023 3:41:35 PM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Using configuration file at /opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/config/seatunnel.yaml
Apr 24, 2023 3:41:35 PM org.apache.seatunnel.engine.common.config.SeaTunnelConfig
INFO: seatunnel.home is /home/hadoop
Apr 24, 2023 3:41:35 PM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Loading configuration '/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/config/hazelcast.yaml' from System property 'hazelcast.config'
Apr 24, 2023 3:41:35 PM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Using configuration file at /opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/config/hazelcast.yaml
Apr 24, 2023 3:41:36 PM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Loading configuration '/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/config/hazelcast-client.yaml' from System property 'hazelcast.client.config'
Apr 24, 2023 3:41:36 PM com.hazelcast.internal.config.AbstractConfigLocator
INFO: Using configuration file at /opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/config/hazelcast-client.yaml
2023-04-24 15:41:36,678 INFO com.hazelcast.client.impl.spi.ClientInvocationService - hz.client_1 [seatunnel] [5.1] Running with 2 response threads, dynamic=true
2023-04-24 15:41:36,753 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2023-04-24 15:41:36,754 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2023-04-24 15:41:36,777 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Trying to connect to cluster: seatunnel
2023-04-24 15:41:36,780 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Trying to connect to [bigdata04]:5801
2023-04-24 15:41:36,811 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2023-04-24 15:41:36,812 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Authenticated with server [bigdata04]:5801:3ea41eb6-06cd-470a-94fe-f8c485cfa2f2, server version: 5.1, local address: /172.27.10.253:35535
2023-04-24 15:41:36,813 INFO com.hazelcast.internal.diagnostics.Diagnostics - hz.client_1 [seatunnel] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2023-04-24 15:41:36,824 INFO com.hazelcast.client.impl.spi.ClientClusterService - hz.client_1 [seatunnel] [5.1]
Members [2] {
Member [bigdata03]:5801 - 438295ac-d767-43ab-ba13-e79495639ecc
Member [bigdata04]:5801 - 3ea41eb6-06cd-470a-94fe-f8c485cfa2f2
}
2023-04-24 15:41:36,826 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Authenticated with server [bigdata03]:5801:438295ac-d767-43ab-ba13-e79495639ecc, server version: 5.1, local address: /172.27.10.253:53423
2023-04-24 15:41:36,852 INFO com.hazelcast.client.impl.statistics.ClientStatisticsService - Client statistics is enabled with period 5 seconds.
2023-04-24 15:41:36,892 INFO org.apache.seatunnel.engine.client.job.JobExecutionEnvironment - add common jar in plugins :[file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/sqlite-jdbc-3.39.3.0.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/redshift-jdbc42-2.1.0.9.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/tablestore-jdbc-5.13.9.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/ojdbc8-12.2.0.1.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/db2jcc-db2jcc4.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/postgresql-42.4.3.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/DmJdbcDriver18-8.1.2.141.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/ali-phoenix-shaded-thin-client-5.2.5-HBase-2.x.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/mssql-jdbc-9.2.1.jre8.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/
lib/terajdbc4-17.20.00.12.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/mysql-connector-java-8.0.27.jar]
2023-04-24 15:41:36,898 INFO org.apache.seatunnel.core.starter.utils.ConfigBuilder - Loading config file from path: /appcom/tmp/ydstnnlConfDir/9_319_MySQL_MySQL.conf
2023-04-24 15:41:36,946 INFO org.apache.seatunnel.core.starter.utils.ConfigShadeUtils - Load config shade spi: [base64]
2023-04-24 15:41:36,989 INFO org.apache.seatunnel.core.starter.utils.ConfigBuilder - Parsed config file: {
"env" : {
"execution.parallelism" : 1,
"job.mode" : "BATCH"
},
"source" : [
{
"password" : "111",
"driver" : "com.mysql.cj.jdbc.Driver",
"query" : " select id,use_name,use_pwd,load_ts from xxx where 1=1",
"connection_check_timeout_sec" : 1000,
"plugin_name" : "Jdbc",
"user" : "11",
"url" : "jdbc:mysql://0.0.0.0:3306/wcscreen"
}
],
"transform" : [],
"sink" : [
{
"password" : "111",
"driver" : "com.mysql.cj.jdbc.Driver",
"max_retries" : 3,
"query" : " replace into xxx(id,use_name,use_pwd,load_ts) values (?,?,?,?) ",
"plugin_name" : "jdbc",
"user" : "11",
"url" : "jdbc:mysql://0.0.0.0:3306/ydtest"
}
]
}
2023-04-24 15:41:37,008 INFO org.apache.seatunnel.core.starter.utils.ConfigBuilder - Loading config file from path: /appcom/tmp/ydstnnlConfDir/9_319_MySQL_MySQL.conf
2023-04-24 15:41:37,013 INFO org.apache.seatunnel.core.starter.utils.ConfigBuilder - Parsed config file: {
"env" : {
"execution.parallelism" : 1,
"job.mode" : "BATCH"
},
"source" : [
{
"password" : "111",
"driver" : "com.mysql.cj.jdbc.Driver",
"query" : " select id,use_name,use_pwd,load_ts from xxx where 1=1",
"connection_check_timeout_sec" : 1000,
"plugin_name" : "Jdbc",
"user" : "11",
"url" : "jdbc:mysql://0.0.0.0:3306/wcscreen"
}
],
"transform" : [],
"sink" : [
{
"password" : "111",
"driver" : "com.mysql.cj.jdbc.Driver",
"max_retries" : 3,
"query" : " replace into xxx(id,use_name,use_pwd,load_ts) values (?,?,?,?) ",
"plugin_name" : "jdbc",
"user" : "11",
"url" : "jdbc:mysql://0.0.0.0:3306/ydtest"
}
]
}
2023-04-24 15:41:37,021 INFO org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load SeaTunnelSource Plugin from /opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/connectors/seatunnel
2023-04-24 15:41:37,031 INFO org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Discovery plugin jar: Jdbc at: file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/connectors/seatunnel/connector-jdbc-2.3.2-SNAPSHOT.jar
2023-04-24 15:41:37,045 INFO org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load plugin: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='Jdbc'} from path: file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/connectors/seatunnel/connector-jdbc-2.3.2-SNAPSHOT.jar use classloader: org.apache.seatunnel.engine.common.loader.SeaTunnelChildFirstClassLoader
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2023-04-24 15:41:38,450 INFO org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSource - The partition_column parameter is not configured, and the source parallelism is set to 1
2023-04-24 15:41:38,455 INFO org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load SeaTunnelSink Plugin from /opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/connectors/seatunnel
2023-04-24 15:41:38,456 INFO org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Discovery plugin jar: jdbc at: file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/connectors/seatunnel/connector-jdbc-2.3.2-SNAPSHOT.jar
2023-04-24 15:41:38,457 INFO org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load plugin: PluginIdentifier{engineType='seatunnel', pluginType='sink', pluginName='jdbc'} from classpath
2023-04-24 15:41:38,496 INFO org.apache.seatunnel.engine.client.job.ClientJobProxy - start submit job, job id: 702786240566525958, with plugin jar [file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/sqlite-jdbc-3.39.3.0.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/redshift-jdbc42-2.1.0.9.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/tablestore-jdbc-5.13.9.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/ojdbc8-12.2.0.1.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/db2jcc-db2jcc4.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/connectors/seatunnel/connector-jdbc-2.3.2-SNAPSHOT.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/postgresql-42.4.3.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/DmJdbcDriver18-8.1.2.141.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/ali-phoenix-shaded-thin-client-5.2.5-HBase-2.x.jar, file:/opt/usdp-srv/s
rv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/mssql-jdbc-9.2.1.jre8.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/terajdbc4-17.20.00.12.jar, file:/opt/usdp-srv/srv/udp/2.0.0.0/seatunnel/plugins/jdbc/lib/mysql-connector-java-8.0.27.jar]
2023-04-24 15:41:38,666 WARN org.apache.seatunnel.engine.client.job.JobMetricsRunner - Failed to get job metrics summary, it maybe first-run
2023-04-24 15:42:38,666 INFO org.apache.seatunnel.engine.client.job.JobMetricsRunner -
***********************************************
Job Progress Information
***********************************************
Job Id : 702786240566525958
Read Count So Far : 0
Write Count So Far : 0
Average Read Count : 0/s
Average Write Count : 0/s
Last Statistic Time : 2023-04-24 15:41:38
Current Statistic Time : 2023-04-24 15:42:38
***********************************************
2023-04-24 15:43:38,660 INFO org.apache.seatunnel.engine.client.job.JobMetricsRunner -
***********************************************
Job Progress Information
***********************************************
Job Id : 702786240566525958
Read Count So Far : 0
Write Count So Far : 0
Average Read Count : 0/s
Average Write Count : 0/s
Last Statistic Time : 2023-04-24 15:42:38
Current Statistic Time : 2023-04-24 15:43:38
***********************************************
2023-04-24 15:44:38,660 INFO org.apache.seatunnel.engine.client.job.JobMetricsRunner -
***********************************************
Job Progress Information
***********************************************
Job Id : 702786240566525958
Read Count So Far : 0
Write Count So Far : 0
Average Read Count : 0/s
Average Write Count : 0/s
Last Statistic Time : 2023-04-24 15:43:38
Current Statistic Time : 2023-04-24 15:44:38
***********************************************
2023-04-24 15:45:38,659 INFO org.apache.seatunnel.engine.client.job.JobMetricsRunner -
***********************************************
Job Progress Information
***********************************************
Job Id : 702786240566525958
Read Count So Far : 0
Write Count So Far : 0
Average Read Count : 0/s
Average Write Count : 0/s
Last Statistic Time : 2023-04-24 15:44:38
Current Statistic Time : 2023-04-24 15:45:38
***********************************************
2023-04-24 15:46:38,661 INFO org.apache.seatunnel.engine.client.job.JobMetricsRunner -
***********************************************
Job Progress Information
***********************************************
Job Id : 702786240566525958
Read Count So Far : 0
Write Count So Far : 0
Average Read Count : 0/s
Average Write Count : 0/s
Last Statistic Time : 2023-04-24 15:45:38
Current Statistic Time : 2023-04-24 15:46:38
***********************************************
2023-04-24 15:46:39,825 INFO org.apache.seatunnel.engine.client.job.ClientJobProxy - Job (702786240566525958) end with state FAILED
2023-04-24 15:46:39,826 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2023-04-24 15:46:39,832 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [bigdata04]:5801:3ea41eb6-06cd-470a-94fe-f8c485cfa2f2, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/172.27.10.253:35535->bigdata04/172.27.10.183:5801}, remoteAddress=[bigdata04]:5801, lastReadTime=2023-04-24 15:46:36.855, lastWriteTime=2023-04-24 15:46:36.855, closedTime=2023-04-24 15:46:39.828, connected server version=5.1}
2023-04-24 15:46:39,833 INFO com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel] [5.1] Removed connection to endpoint: [bigdata03]:5801:438295ac-d767-43ab-ba13-e79495639ecc, connection: ClientConnection{alive=false, connectionId=2, channel=NioChannel{/172.27.10.253:53423->bigdata03/172.27.10.152:5801}, remoteAddress=[bigdata03]:5801, lastReadTime=2023-04-24 15:46:39.812, lastWriteTime=2023-04-24 15:46:38.626, closedTime=2023-04-24 15:46:39.832, connected server version=5.1}
2023-04-24 15:46:39,833 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2023-04-24 15:46:39,835 INFO com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2023-04-24 15:46:39,835 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed SeaTunnel client......
2023-04-24 15:46:39,836 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed metrics executor service ......
2023-04-24 15:46:39,836 ERROR org.apache.seatunnel.core.starter.SeaTunnel -
===============================================================================
2023-04-24 15:46:39,836 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Fatal Error,
2023-04-24 15:46:39,836 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Please submit bug report in https://github.com/apache/incubator-seatunnel/issues
2023-04-24 15:46:39,836 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Reason:SeaTunnel job executed failed
2023-04-24 15:46:39,838 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: org.apache.seatunnel.engine.server.checkpoint.CheckpointException: CheckpointCoordinator inside have error.
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:215)
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:211)
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$reportedTask$2(CheckpointCoordinator.java:201)
at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:870)
at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:852)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1632)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.CompletionException: java.lang.RuntimeException: Execute given execution failed after retry 30 times
at com.hazelcast.spi.impl.AbstractInvocationFuture.returnOrThrowWithJoinConventions(AbstractInvocationFuture.java:819)
at com.hazelcast.spi.impl.AbstractInvocationFuture.resolveAndThrowWithJoinConvention(AbstractInvocationFuture.java:835)
at com.hazelcast.spi.impl.AbstractInvocationFuture.join(AbstractInvocationFuture.java:553)
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.restoreTaskState(CheckpointCoordinator.java:250)
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$reportedTask$1(CheckpointCoordinator.java:189)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
... 3 more
Caused by: java.lang.RuntimeException: Execute given execution failed after retry 30 times
at org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:75)
at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.run(NotifyTaskRestoreOperation.java:85)
at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189)
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273)
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)
at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
Caused by: org.apache.seatunnel.engine.server.exception.TaskGroupContextNotFoundException: task group TaskGroupLocation{jobId=702786240566525958, pipelineId=1, taskGroupId=1} not found.
at org.apache.seatunnel.engine.server.TaskExecutionService.getExecutionContext(TaskExecutionService.java:168)
at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$run$0(NotifyTaskRestoreOperation.java:90)
at org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:48)
... 9 more
at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:119)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:174)
... 2 more
2023-04-24 15:46:39,838 ERROR org.apache.seatunnel.core.starter.SeaTunnel -
===============================================================================
Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: org.apache.seatunnel.engine.server.checkpoint.CheckpointException: CheckpointCoordinator inside have error.
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:215)
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.handleCoordinatorError(CheckpointCoordinator.java:211)
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$reportedTask$2(CheckpointCoordinator.java:201)
at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:870)
at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:852)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1632)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.CompletionException: java.lang.RuntimeException: Execute given execution failed after retry 30 times
at com.hazelcast.spi.impl.AbstractInvocationFuture.returnOrThrowWithJoinConventions(AbstractInvocationFuture.java:819)
at com.hazelcast.spi.impl.AbstractInvocationFuture.resolveAndThrowWithJoinConvention(AbstractInvocationFuture.java:835)
at com.hazelcast.spi.impl.AbstractInvocationFuture.join(AbstractInvocationFuture.java:553)
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.restoreTaskState(CheckpointCoordinator.java:250)
at org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator.lambda$reportedTask$1(CheckpointCoordinator.java:189)
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
... 3 more
Caused by: java.lang.RuntimeException: Execute given execution failed after retry 30 times
at org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:75)
at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.run(NotifyTaskRestoreOperation.java:85)
at com.hazelcast.spi.impl.operationservice.Operation.call(Operation.java:189)
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:273)
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137)
at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)
at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
Caused by: org.apache.seatunnel.engine.server.exception.TaskGroupContextNotFoundException: task group TaskGroupLocation{jobId=702786240566525958, pipelineId=1, taskGroupId=1} not found.
at org.apache.seatunnel.engine.server.TaskExecutionService.getExecutionContext(TaskExecutionService.java:168)
at org.apache.seatunnel.engine.server.checkpoint.operation.NotifyTaskRestoreOperation.lambda$run$0(NotifyTaskRestoreOperation.java:90)
at org.apache.seatunnel.common.utils.RetryUtils.retryWithException(RetryUtils.java:48)
... 9 more
at org.apache.seatunnel.engine.client.job.ClientJobProxy.waitForJobComplete(ClientJobProxy.java:119)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:174)
... 2 more
2023-04-24 15:46:39,839 INFO org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - run shutdown hook because get close signal
```
### Flink or Spark Version
no use flink or spark
### Java or Scala Version
jdk8
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 commented on issue #4656: [Bug] [connector-jdbc] Specify offline mode to execute discovery as a streaming task
Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 commented on issue #4656:
URL: https://github.com/apache/incubator-seatunnel/issues/4656#issuecomment-1521426618
Hi, this is a bug in V2.3.1, we fixed it in https://github.com/apache/incubator-seatunnel/pull/4449
You can use dev branch for test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] github-actions[bot] closed issue #4656: [Bug] [connector-jdbc] Specify offline mode to execute discovery as a streaming task
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #4656: [Bug] [connector-jdbc] Specify offline mode to execute discovery as a streaming task
URL: https://github.com/apache/seatunnel/issues/4656
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] github-actions[bot] commented on issue #4656: [Bug] [connector-jdbc] Specify offline mode to execute discovery as a streaming task
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4656:
URL: https://github.com/apache/seatunnel/issues/4656#issuecomment-1575859564
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zengqinchris commented on issue #4656: [Bug] [connector-jdbc] Specify offline mode to execute discovery as a streaming task
Posted by "zengqinchris (via GitHub)" <gi...@apache.org>.
zengqinchris commented on issue #4656:
URL: https://github.com/apache/incubator-seatunnel/issues/4656#issuecomment-1522627472
Thank you for your support and assistance. The bug in dev code testing is no longer present
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zengqinchris commented on issue #4656: [Bug] [connector-jdbc] Specify offline mode to execute discovery as a streaming task
Posted by "zengqinchris (via GitHub)" <gi...@apache.org>.
zengqinchris commented on issue #4656:
URL: https://github.com/apache/incubator-seatunnel/issues/4656#issuecomment-1521404248
If there are more than two seatunnel cluster nodes, this problem is easy to occur. If there are only a single seatunnel node, it will occur less frequently
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zengqinchris commented on issue #4656: [Bug] [connector-jdbc] Specify offline mode to execute discovery as a streaming task
Posted by "zengqinchris (via GitHub)" <gi...@apache.org>.
zengqinchris commented on issue #4656:
URL: https://github.com/apache/incubator-seatunnel/issues/4656#issuecomment-1522628570
The bug in dev code fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] github-actions[bot] commented on issue #4656: [Bug] [connector-jdbc] Specify offline mode to execute discovery as a streaming task
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #4656:
URL: https://github.com/apache/seatunnel/issues/4656#issuecomment-1565101404
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zengqinchris commented on issue #4656: [Bug] [connector-jdbc] Specify offline mode to execute discovery as a streaming task
Posted by "zengqinchris (via GitHub)" <gi...@apache.org>.
zengqinchris commented on issue #4656:
URL: https://github.com/apache/incubator-seatunnel/issues/4656#issuecomment-1522628973
> Thank you for your support
Thank you for your support ，fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org