You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/10/27 06:44:38 UTC
[GitHub] [dolphinscheduler] Sky-Gu opened a new issue, #12565: [Bug] [Module Name] Flink on yarn. When a flink task is cancelled, the yarn task cannot be cancelled
Sky-Gu opened a new issue, #12565:
URL: https://github.com/apache/dolphinscheduler/issues/12565
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
When a flink task is cancelled, the task process on linux is killed, but the task on yarn cannot be cancelled.
### What you expected to happen
yarn task synchronization is cancelled
### How to reproduce
1. create a flink task, program type: java, deployment mode: cluster, version: >= 1.10
2. cancel flink task
3. worker node This task will be killed
4. the yarn task is running properly
The log when a task is cancelled is as follows:
[INFO] 2022-10-27 13:44:12.510 +0800 org.apache.dolphinscheduler.server.log.LoggerRequestProcessor:[78] - [WorkflowInstance-0][TaskInstance-0] - received command : Command [type=ROLL_VIEW_LOG_REQUEST, opaque=354, bodyLen=123]
[INFO] 2022-10-27 13:44:12.579 +0800 org.apache.dolphinscheduler.server.log.LoggerRequestProcessor:[78] - [WorkflowInstance-0][TaskInstance-0] - received command : Command [type=ROLL_VIEW_LOG_REQUEST, opaque=356, bodyLen=124]
[INFO] 2022-10-27 13:44:37.616 +0800 org.apache.dolphinscheduler.server.worker.processor.TaskKillProcessor:[90] - [WorkflowInstance-0][TaskInstance-0] - task kill command : TaskKillRequestCommand(taskInstanceId=218110)
[INFO] 2022-10-27 13:44:37.866 +0800 org.apache.dolphinscheduler.server.worker.processor.TaskKillProcessor:[195] - [WorkflowInstance-0][TaskInstance-0] - process id:27261, cmd:sudo -u root kill -9 27266 27271
[INFO] 2022-10-27 13:44:37.895 +0800 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.flink.FlinkTask:[375] - [WorkflowInstance-41639][TaskInstance-218110] - find app id: application_1648525148902_325630
[INFO] 2022-10-27 13:44:37.899 +0800 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.flink.FlinkTask:[205] - [WorkflowInstance-41639][TaskInstance-218110] - process has exited, execute path:/tmp/dolphinscheduler/exec/process/9/274_21/41639/218110, processId:27261 ,exitStatusCode:137 ,processWaitForStatus:true ,processExitValue:137
[INFO] 2022-10-27 13:44:37.900 +0800 org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[220] - [WorkflowInstance-41639][TaskInstance-218110] - task instance id : 218110,task final status : KILL
[INFO] 2022-10-27 13:44:37.902 +0800 org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[247] - [WorkflowInstance-41639][TaskInstance-218110] - develop mode is: false
[INFO] 2022-10-27 13:44:37.905 +0800 org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[265] - [WorkflowInstance-41639][TaskInstance-218110] - exec local path: /tmp/dolphinscheduler/exec/process/9/274_21/41639/218110 cleared.
[INFO] 2022-10-27 13:44:37.906 +0800 org.apache.dolphinscheduler.server.worker.processor.TaskKillProcessor:[216] - [WorkflowInstance-0][TaskInstance-0] - log host : 172.16.10.205 , logPath : /opt/soft/dolphinscheduler3.0.0/worker-server/logs/20221027/274_21-41639-218110.log , port : 1234
[INFO] 2022-10-27 13:44:37.906 +0800 org.apache.dolphinscheduler.service.log.LogClientService:[117] - [WorkflowInstance-0][TaskInstance-0] - view log path /opt/soft/dolphinscheduler3.0.0/worker-server/logs/20221027/274_21-41639-218110.log
[INFO] 2022-10-27 13:44:37.908 +0800 org.apache.dolphinscheduler.server.worker.processor.TaskKillProcessor:[86] - [WorkflowInstance-0][TaskInstance-0] - find app id: application_1648525148902_325630
[ERROR] 2022-10-27 13:44:38.156 +0800 org.apache.dolphinscheduler.common.utils.HttpUtils:[162] - [WorkflowInstance-0][TaskInstance-0] - http get:400 response status code is not 200!
[ERROR] 2022-10-27 13:44:38.171 +0800 org.apache.dolphinscheduler.common.utils.HttpUtils:[162] - [WorkflowInstance-0][TaskInstance-0] - http get:404 response status code is not 200!
[INFO] 2022-10-27 13:44:38.173 +0800 org.apache.dolphinscheduler.remote.NettyRemotingClient:[389] - [WorkflowInstance-0][TaskInstance-0] - netty client closed
[INFO] 2022-10-27 13:44:38.174 +0800 org.apache.dolphinscheduler.service.log.LogClientService:[74] - [WorkflowInstance-0][TaskInstance-0] - logger client closed
[INFO] 2022-10-27 13:44:38.181 +0800 org.apache.dolphinscheduler.server.worker.processor.TaskKillProcessor:[121] - [WorkflowInstance-0][TaskInstance-0] - remove REMOTE_CHANNELS, task instance id:218110
[INFO] 2022-10-27 13:44:38.190 +0800 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.flink.FlinkTask:[57] - [WorkflowInstance-0][TaskInstance-0] - FINALIZE_SESSION
### Anything else
_No response_
### Version
3.0.x
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] closed issue #12565: [Bug] [Module Name] Flink on yarn. When a flink task is cancelled, the yarn task cannot be cancelled
Posted by github-actions.
github-actions[bot] closed issue #12565: [Bug] [Module Name] Flink on yarn. When a flink task is cancelled, the yarn task cannot be cancelled
URL: https://github.com/apache/dolphinscheduler/issues/12565
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] Sky-Gu commented on issue #12565: [Bug] [Module Name] Flink on yarn. When a flink task is cancelled, the yarn task cannot be cancelled
Posted by GitBox <gi...@apache.org>.
Sky-Gu commented on issue #12565:
URL: https://github.com/apache/dolphinscheduler/issues/12565#issuecomment-1293158621
HttpUtils.get() does not support the http status code: 307 redirection operation. As a result, it cannot be used properly in Aliyun EMR.
The EMR service performs secondary directional operations on the HA ip address
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #12565: [Bug] [Module Name] Flink on yarn. When a flink task is cancelled, the yarn task cannot be cancelled
Posted by github-actions.
github-actions[bot] commented on issue #12565:
URL: https://github.com/apache/dolphinscheduler/issues/12565#issuecomment-1399096578
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #12565: [Bug] [Module Name] Flink on yarn. When a flink task is cancelled, the yarn task cannot be cancelled
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #12565:
URL: https://github.com/apache/dolphinscheduler/issues/12565#issuecomment-1293082431
Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #12565: [Bug] [Module Name] Flink on yarn. When a flink task is cancelled, the yarn task cannot be cancelled
Posted by github-actions.
github-actions[bot] commented on issue #12565:
URL: https://github.com/apache/dolphinscheduler/issues/12565#issuecomment-1407816736
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org