You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "kwonder0926 (via GitHub)" <gi...@apache.org> on 2025/01/21 03:24:48 UTC
[I] [Bug] [Engine] Frequent "TaskGroupLocation: TaskGroupLocation xxx already exists" error during task execution, causing task failure. [seatunnel]
kwonder0926 opened a new issue, #8566:
URL: https://github.com/apache/seatunnel/issues/8566
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
This issue occurs on both versions 2.3.8 and 2.3.9.
By observing the logs, I found that when I execute a task, my task starts executing normally for a short period, and then it suddenly disconnects, and I don't know the reason for this, leading to a timeout exception. The worker nodes then reinitialize the cluster, and the master node resubmits the task to the worker nodes. At this point, an exception is thrown, indicating that the task already exists, resulting in task failure.
I will attach the key parts of my logs below for further investigation.
### SeaTunnel Version
2.3.8、2.3.9
### SeaTunnel Config
```conf
{
"env" : {
"parallelism" : 2,
"job.mode" : "STREAMING",
"checkpoint.interval" : 2000
},
"source" : [
{
"hosts" : "10.xxx.xxx.xx:27017,10.xxx.xxx.xxx:27017,10.xxx.xxx.xxx:27017",
"database" : [
"letterMsg"
],
"collection" : [
"letterMsg.2024-05-send_detail",
"letterMsg.2024-04-send_detail"
],
"username" : "******",
"password" : "******",
"connection.options" : "replicaSet=rep1&connectTimeoutMS=300000",
"tables_configs" : [
{
"schema" : {
"table" : "letterMsg.2024-05-send_detail",
"fields" : {
"_id" : "STRING",
"order_id" : "STRING",
"phone_num" : "String",
"message_context" : "STRING",
"channel_no" : "STRING",
"send_status" : "STRING",
"send_time" : "TIMESTAMP",
"back_status" : "STRING",
"create_time" : "TIMESTAMP",
"urgent_lv" : "STRING",
"sign" : "STRING",
"sign_id" : "INT",
"query_report_flag" : "INT",
"isp_id" : "INT",
"user_id" : "BIGINT",
"enterprise_id" : "BIGINT",
"mongo_submit_id" : "STRING",
"link_id" : "STRING",
"batch_id" : "STRING",
"push_submit_time" : "TIMESTAMP",
"back_time" : "TIMESTAMP",
"number_id" : "BIGINT",
"receive_report_time" : "TIMESTAMP",
"report_msg_id" : "STRING",
"push_report_time" : "STRING",
"seq_id" : "STRING",
"seq_order_id" : "STRING"
}
}
},
{
"schema" : {
"table" : "letterMsg.2024-04-send_detail",
"fields" : {
"_id" : "STRING",
"order_id" : "STRING",
"phone_num" : "String",
"message_context" : "STRING",
"channel_no" : "STRING",
"send_status" : "STRING",
"send_time" : "TIMESTAMP",
"back_status" : "STRING",
"create_time" : "TIMESTAMP",
"urgent_lv" : "STRING",
"sign" : "STRING",
"sign_id" : "INT",
"query_report_flag" : "INT",
"isp_id" : "INT",
"user_id" : "BIGINT",
"enterprise_id" : "BIGINT",
"mongo_submit_id" : "STRING",
"link_id" : "STRING",
"batch_id" : "STRING",
"push_submit_time" : "TIMESTAMP",
"back_time" : "TIMESTAMP",
"number_id" : "BIGINT",
"receive_report_time" : "TIMESTAMP",
"report_msg_id" : "STRING",
"push_report_time" : "STRING",
"seq_id" : "STRING",
"seq_order_id" : "STRING"
}
}
}
],
"plugin_name" : "MongoDB-CDC"
}
],
"sink" : [
{
"host" : "10.xxx.xxx.xx:8123",
"database" : "xlsms_test",
"table" : "sms_send_detail",
"username" : "******",
"password" : "******",
"primary_key" : "_id",
"support_upsert" : true,
"allow_experimental_lightweight_delete" : true,
"plugin_name" : "Clickhouse"
}
]
}
```
### Running Command
```shell
bin/seatunnel.sh --config ./config/streaming-mongocdc-clickhouse.conf -n syncSendDetailJob
```
### Error Exception
```log
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: org.apache.seatunnel.engine.common.exception.TaskGroupDeployException: java.lang.RuntimeException: TaskGroupLocation: TaskGroupLocation{jobId=933914021243912194, pipelineId=1, taskGroupId=1} already exists
```
### Zeta or Flink or Spark Version
use SeaTunnel Engine In Separated Cluster Mode
### Java or Scala Version
java 1.8.0_171
### Screenshots


### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Engine] Frequent "TaskGroupLocation: TaskGroupLocation xxx already exists" error during task execution, causing task failure. [seatunnel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #8566: [Bug] [Engine] Frequent "TaskGroupLocation: TaskGroupLocation xxx already exists" error during task execution, causing task failure.
URL: https://github.com/apache/seatunnel/issues/8566
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Engine] Frequent "TaskGroupLocation: TaskGroupLocation xxx already exists" error during task execution, causing task failure. [seatunnel]
Posted by "kwonder0926 (via GitHub)" <gi...@apache.org>.
kwonder0926 commented on issue #8566:
URL: https://github.com/apache/seatunnel/issues/8566#issuecomment-2603751842
I saw a statement online that if a table to be transferred is too large, the checkpoint timeout needs to be set longer. But what value should I set it to? I would like to ask if there is any reference for this value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Engine] Frequent "TaskGroupLocation: TaskGroupLocation xxx already exists" error during task execution, causing task failure. [seatunnel]
Posted by "lphbdfj (via GitHub)" <gi...@apache.org>.
lphbdfj commented on issue #8566:
URL: https://github.com/apache/seatunnel/issues/8566#issuecomment-2612828800
I have also encountered this issue and have not yet found a solution. I hope the official team can provide a resolution as soon as possible. -> [Bug] [Engine] Frequent "TaskGroupLocation: TaskGroupLocation xxx already exists" error during task execution, causing task failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Engine] Frequent "TaskGroupLocation: TaskGroupLocation xxx already exists" error during task execution, causing task failure. [seatunnel]
Posted by "kwonder0926 (via GitHub)" <gi...@apache.org>.
kwonder0926 commented on issue #8566:
URL: https://github.com/apache/seatunnel/issues/8566#issuecomment-2756896096
> Hello, may I ask if you have solved this issue? I often encounter this problem now, and there is no solution, with no response from the official support.
并没有解决,但是当把Sink换到Doris后好像没有遇到这个问题了
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Engine] Frequent "TaskGroupLocation: TaskGroupLocation xxx already exists" error during task execution, causing task failure. [seatunnel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #8566:
URL: https://github.com/apache/seatunnel/issues/8566#issuecomment-2693003683
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Engine] Frequent "TaskGroupLocation: TaskGroupLocation xxx already exists" error during task execution, causing task failure. [seatunnel]
Posted by "JanSay (via GitHub)" <gi...@apache.org>.
JanSay commented on issue #8566:
URL: https://github.com/apache/seatunnel/issues/8566#issuecomment-2756872170
Hello, may I ask if you have solved this issue? I often encounter this problem now, and there is no solution, with no response from the official support.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Engine] Frequent "TaskGroupLocation: TaskGroupLocation xxx already exists" error during task execution, causing task failure. [seatunnel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #8566:
URL: https://github.com/apache/seatunnel/issues/8566#issuecomment-2677219187
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Engine] Frequent "TaskGroupLocation: TaskGroupLocation xxx already exists" error during task execution, causing task failure. [seatunnel]
Posted by "jw-itq (via GitHub)" <gi...@apache.org>.
jw-itq commented on issue #8566:
URL: https://github.com/apache/seatunnel/issues/8566#issuecomment-2717859697
> I saw a statement online that if a table to be transferred is too large, the checkpoint timeout needs to be set longer. But what value should I set it to? I would like to ask if there is any reference for this value?
try checkpoint.timeout=300000
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org