You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "ccczhouxin (via GitHub)" <gi...@apache.org> on 2024/07/08 03:24:05 UTC
[I] The seatunnel cluster has some error [seatunnel]
ccczhouxin opened a new issue, #7127:
URL: https://github.com/apache/seatunnel/issues/7127
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
Sending data to kafka is an error,the cluster has two nodes,one has offline
### SeaTunnel Version
2.3.5
### SeaTunnel Config
```conf
hazelcast:
cluster-name: seatunnel-xugu
network:
rest-api:
enabled: true
endpoint-groups:
CLUSTER_WRITE:
enabled: true
DATA:
enabled: true
join:
tcp-ip:
enabled: true
member-list:
- a:5801
- b:5801
port:
auto-increment: true
port: 5801
properties:
hazelcast.invocation.max.retry.count: 20
hazelcast.tcp.join.port.try.count: 30
hazelcast.logging.type: log4j2
hazelcast.operation.generic.thread.count: 50
hazelcast-client:
cluster-name: seatunnel-xugu
properties:
hazelcast.logging.type: log4j2
connection-strategy:
connection-retry:
cluster-connect-timeout-millis: 3000
network:
cluster-members:
- a:5801
- b:5801
```
### Running Command
```shell
./bin/seatunnel.sh --config config/a
```
### Error Exception
```log
2024-07-07 07:56:36,690 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel-xugu] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2024-07-07 07:56:36,734 WARN [.c.i.s.ClientInvocationService] [hz.client_1.internal-4] - hz.client_1 [seatunnel-xugu] [5.1] Future.complete(Object) on completed future. Request: ClientMessage{connection=null, length=119, operation=Client.Authentication, isRetryable=true, correlationId=19899, messageType=100, isEvent=false, isfragmented=false}, current value: ExceptionalResult{cause=java.lang.InterruptedException}, offered value: ExceptionalResult{cause=com.hazelcast.client.HazelcastClientNotActiveException: Client is shutting down}
java.lang.Exception: null
at com.hazelcast.spi.impl.AbstractInvocationFuture.warnIfSuspiciousDoubleCompletion(AbstractInvocationFuture.java:1253) ~[seatunnel-starter.jar:2.3.6-SNAPSHOT]
at com.hazelcast.spi.impl.AbstractInvocationFuture.complete0(AbstractInvocationFuture.java:1230) ~[seatunnel-starter.jar:2.3.6-SNAPSHOT]
at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionallyInternal(AbstractInvocationFuture.java:1223) ~[seatunnel-starter.jar:2.3.6-SNAPSHOT]
at com.hazelcast.spi.impl.AbstractInvocationFuture.completeExceptionally(AbstractInvocationFuture.java:709) ~[seatunnel-starter.jar:2.3.6-SNAPSHOT]
at com.hazelcast.client.impl.spi.impl.ClientInvocation.completeExceptionally(ClientInvocation.java:294) ~[seatunnel-starter.jar:2.3.6-SNAPSHOT]
at com.hazelcast.client.impl.spi.impl.ClientInvocation.notifyExceptionWithOwnedPermission(ClientInvocation.java:316) ~[seatunnel-starter.jar:2.3.6-SNAPSHOT]
at com.hazelcast.client.impl.spi.impl.ClientInvocationServiceImpl.onConnectionClose(ClientInvocationServiceImpl.java:221) ~[seatunnel-starter.jar:2.3.6-SNAPSHOT]
at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.onConnectionClose(TcpClientConnectionManager.java:758) ~[seatunnel-starter.jar:2.3.6-SNAPSHOT]
at com.hazelcast.client.impl.connection.tcp.TcpClientConnection.close(TcpClientConnection.java:191) ~[seatunnel-starter.jar:2.3.6-SNAPSHOT]
at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.authenticateOnCluster(TcpClientConnectionManager.java:885) ~[seatunnel-starter.jar:2.3.6-SNAPSHOT]
at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.getOrConnectToMember(TcpClientConnectionManager.java:627) ~[seatunnel-starter.jar:2.3.6-SNAPSHOT]
at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager$ConnectToAllClusterMembersTask.lambda$run$0(TcpClientConnectionManager.java:1182) ~[seatunnel-starter.jar:2.3.6-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_221]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_221]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_221]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_221]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_221]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_221]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_221]
at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76) [seatunnel-starter.jar:2.3.6-SNAPSHOT]
at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102) [seatunnel-starter.jar:2.3.6-SNAPSHOT]
2024-07-07 07:56:36,737 WARN [.c.i.c.ClientConnectionManager] [hz.client_1.internal-4] - hz.client_1 [seatunnel-xugu] [5.1] Could not connect to member eafe4bcc-2b7e-47be-9ef9-dea9fed61f42, reason com.hazelcast.core.HazelcastException: java.lang.InterruptedException
2024-07-07 07:56:36,739 INFO [.c.i.c.ClientConnectionManager] [main] - hz.client_1 [seatunnel-xugu] [5.1] Removed connection to endpoint: [localhost]:5801:0db5a450-6149-480f-ba92-345b8eb1bca9, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/localhost:28092->/localhost:5801}, remoteAddress=[localhost]:5801, lastReadTime=2024-07-07 07:56:36.676, lastWriteTime=2024-07-07 07:56:36.238, closedTime=2024-07-07 07:56:36.737, connected server version=5.1}
2024-07-07 07:56:36,739 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel-xugu] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2024-07-07 07:56:36,743 INFO [c.h.c.LifecycleService ] [main] - hz.client_1 [seatunnel-xugu] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2024-07-07 07:56:36,743 INFO [s.c.s.s.c.ClientExecuteCommand] [main] - Closed SeaTunnel client......
2024-07-07 07:56:36,743 INFO [s.c.s.s.c.ClientExecuteCommand] [main] - Closed metrics executor service ......
2024-07-07 07:56:36,743 ERROR [o.a.s.c.s.SeaTunnel ] [main] -
===============================================================================
2024-07-07 07:56:36,743 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Fatal Error,
2024-07-07 07:56:36,743 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Please submit bug report in https://github.com/apache/seatunnel/issues
2024-07-07 07:56:36,744 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Reason:SeaTunnel job executed failed
2024-07-07 07:56:36,746 ERROR [o.a.s.c.s.SeaTunnel ] [main] - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:209)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: org.apache.seatunnel.engine.common.exception.JobException: The taskGroup(TaskGroupLocation{jobId=1464550058967051, pipelineId=1, taskGroupId=1}) deployed node([slave]:5801) offline
at org.apache.seatunnel.engine.server.CoordinatorService.lambda$makeTasksFailed$13(CoordinatorService.java:729)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.apache.seatunnel.engine.server.CoordinatorService.makeTasksFailed(CoordinatorService.java:714)
at org.apache.seatunnel.engine.server.CoordinatorService.lambda$null$11(CoordinatorService.java:704)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.apache.seatunnel.engine.server.CoordinatorService.lambda$failedTaskOnMemberRemoved$12(CoordinatorService.java:702)
at java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1597)
at org.apache.seatunnel.engine.server.CoordinatorService.failedTaskOnMemberRemoved(CoordinatorService.java:697)
at org.apache.seatunnel.engine.server.CoordinatorService.memberRemoved(CoordinatorService.java:740)
at org.apache.seatunnel.engine.server.SeaTunnelServer.memberRemoved(SeaTunnelServer.java:162)
at com.hazelcast.internal.cluster.impl.MembershipManager.lambda$sendMembershipEventNotifications$2(MembershipManager.java:835)
at com.hazelcast.internal.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:217)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76)
at com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:201)
... 2 more
```
### Zeta or Flink or Spark Version
_No response_
### Java or Scala Version
1.8
### Screenshots
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The seatunnel cluster has some error [seatunnel]
Posted by "ccczhouxin (via GitHub)" <gi...@apache.org>.
ccczhouxin commented on issue #7127:
URL: https://github.com/apache/seatunnel/issues/7127#issuecomment-2216105163
thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The seatunnel cluster has some error [seatunnel]
Posted by "emmanuelsosa (via GitHub)" <gi...@apache.org>.
emmanuelsosa commented on issue #7127:
URL: https://github.com/apache/seatunnel/issues/7127#issuecomment-2215112215
To ensure fault tolerance, your cluster must consist of at least three members. In hazelcast clusters that involve leader elections, you need maintain an odd number of nodes (and not just one) to facilitate an undisputed leader election.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The seatunnel cluster has some error [seatunnel]
Posted by "ccczhouxin (via GitHub)" <gi...@apache.org>.
ccczhouxin closed issue #7127: The seatunnel cluster has some error
URL: https://github.com/apache/seatunnel/issues/7127
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org