You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/09/13 06:18:52 UTC
[GitHub] [dolphinscheduler] jokerhzf opened a new issue, #11903: [Bug] [Woker Sever] org.apache.dolphinscheduler.plugin.task.api.ProcessUtils:[66] - kill task failed
jokerhzf opened a new issue, #11903:
URL: https://github.com/apache/dolphinscheduler/issues/11903
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
version:2.0.6 release
```
[INFO] 2022-09-09 10:00:51.577 org.apache.dolphinscheduler.server.worker.WorkerServer:[239] - ready to kill all cache job, job size:18
[INFO] 2022-09-09 10:00:51.610 org.apache.dolphinscheduler.plugin.task.api.ProcessUtils:[62] - process id:26156, cmd:sudo -u xxx kill -9 26156 26175 3 26181 3 26266 3 26267 3 26268
[ERROR] 2022-09-09 10:00:51.629 org.apache.dolphinscheduler.plugin.task.api.ProcessUtils:[66] - kill task failed
org.apache.dolphinscheduler.plugin.task.api.AbstractShell$ExitCodeException: kill: sending signal to 3 failed: Operation not permitted
kill: sending signal to 3 failed: Operation not permitted
kill: sending signal to 26266 failed: No such process
kill: sending signal to 3 failed: Operation not permitted
kill: sending signal to 26267 failed: No such process
kill: sending signal to 3 failed: Operation not permitted
at org.apache.dolphinscheduler.plugin.task.api.AbstractShell.runCommand(AbstractShell.java:210)
at org.apache.dolphinscheduler.plugin.task.api.AbstractShell.run(AbstractShell.java:124)
at org.apache.dolphinscheduler.plugin.task.api.ShellExecutor.execute(ShellExecutor.java:130)
at org.apache.dolphinscheduler.plugin.task.api.ShellExecutor.execCommand(ShellExecutor.java:105)
at org.apache.dolphinscheduler.plugin.task.api.ShellExecutor.execCommand(ShellExecutor.java:87)
at org.apache.dolphinscheduler.plugin.task.util.OSUtils.exeShell(OSUtils.java:95)
at org.apache.dolphinscheduler.plugin.task.util.OSUtils.exeCmd(OSUtils.java:84)
at org.apache.dolphinscheduler.plugin.task.api.ProcessUtils.kill(ProcessUtils.java:64)
at org.apache.dolphinscheduler.server.worker.WorkerServer.killAllRunningTasks(WorkerServer.java:247)
at org.apache.dolphinscheduler.server.worker.WorkerServer.close(WorkerServer.java:211)
at org.apache.dolphinscheduler.server.worker.WorkerServer.stop(WorkerServer.java:231)
at org.apache.dolphinscheduler.server.worker.registry.WorkerRegistryClient.handleConnectionState(WorkerRegistryClient.java:158)
at org.apache.dolphinscheduler.plugin.registry.zookeeper.ZookeeperConnectionStateListener.stateChanged(ZookeeperConnectionStateListener.java:43)
at org.apache.curator.framework.state.ConnectionStateManager.lambda$processEvents$0(ConnectionStateManager.java:280)
at org.apache.curator.framework.listen.MappingListenerManager.lambda$forEach$0(MappingListenerManager.java:93)
at org.apache.curator.framework.listen.MappingListenerManager.forEach(MappingListenerManager.java:90)
at org.apache.curator.framework.listen.StandardListenerManager.forEach(StandardListenerManager.java:89)
at org.apache.curator.framework.state.ConnectionStateManager.processEvents(ConnectionStateManager.java:280)
at org.apache.curator.framework.state.ConnectionStateManager.access$000(ConnectionStateManager.java:44)
at org.apache.curator.framework.state.ConnectionStateManager$1.call(ConnectionStateManager.java:133)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
### What you expected to happen
When the worker is killing the task, the process ID is incorrectly identified due to the regular expression.
killAllRunningTasks() uses `org.apache.dolphinscheduler.plugin.task.api.ProcessUtils.kill(taskRequest)`, but this kill method uses `private static final Pattern WINDOWSATTERN = Pattern.compile("(\\d+)");` regular expressions to match.But this regular will also recognize python3 as a process ID.
e.g.
[xxx@ip-xx-xxx-xx-xx opt]$ pstree -p 19710
python3(19710)─┬─{python3}(19731)
├─{python3}(19732)
└─{python3}(19733)
**regex：**(\d+)
**match result：**
3
19710
3
19731
3
19732
3
19733
### How to reproduce
Configure python3 shell script and trigger killAllRunningTasks()
### Anything else
_No response_
### Version
2.0.6
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] JinyLeeChina closed issue #11903: [Bug] [Worker Sever] org.apache.dolphinscheduler.plugin.task.api.ProcessUtils:[66] - kill task failed
Posted by GitBox <gi...@apache.org>.
JinyLeeChina closed issue #11903: [Bug] [Worker Sever] org.apache.dolphinscheduler.plugin.task.api.ProcessUtils:[66] - kill task failed
URL: https://github.com/apache/dolphinscheduler/issues/11903
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #11903: [Bug] [Worker Sever] org.apache.dolphinscheduler.plugin.task.api.ProcessUtils:[66] - kill task failed
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #11903:
URL: https://github.com/apache/dolphinscheduler/issues/11903#issuecomment-1309580921
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] JinyLeeChina commented on issue #11903: [Bug] [Worker Sever] org.apache.dolphinscheduler.plugin.task.api.ProcessUtils:[66] - kill task failed
Posted by GitBox <gi...@apache.org>.
JinyLeeChina commented on issue #11903:
URL: https://github.com/apache/dolphinscheduler/issues/11903#issuecomment-1247737708
@jokerhzf Has PR been submitted for this issue ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] davidzollo commented on issue #11903: [Bug] [Worker Sever] org.apache.dolphinscheduler.plugin.task.api.ProcessUtils:[66] - kill task failed
Posted by GitBox <gi...@apache.org>.
davidzollo commented on issue #11903:
URL: https://github.com/apache/dolphinscheduler/issues/11903#issuecomment-1248049142
> @jokerhzf Has PR been submitted for this issue ?
this PR https://github.com/apache/dolphinscheduler/pull/11920 can't solve the problem I think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on issue #11903: [Bug] [Worker Sever] org.apache.dolphinscheduler.plugin.task.api.ProcessUtils:[66] - kill task failed
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on issue #11903:
URL: https://github.com/apache/dolphinscheduler/issues/11903#issuecomment-1245152262
> I think workerServer should use ProcessUtils from `org.apache.dolphinscheduler.server.utils` instead of `org.apache.dolphinscheduler.plugin.task.api`
Hi @jokerhzf can you try to process it unified? There should not be two copies of the same code, increasing maintenance costs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #11903: [Bug] [Woker Sever] org.apache.dolphinscheduler.plugin.task.api.ProcessUtils:[66] - kill task failed
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #11903:
URL: https://github.com/apache/dolphinscheduler/issues/11903#issuecomment-1244955269
Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] jokerhzf commented on issue #11903: [Bug] [Woker Sever] org.apache.dolphinscheduler.plugin.task.api.ProcessUtils:[66] - kill task failed
Posted by GitBox <gi...@apache.org>.
jokerhzf commented on issue #11903:
URL: https://github.com/apache/dolphinscheduler/issues/11903#issuecomment-1245030063
I think workerServer should use ProcessUtils from `org.apache.dolphinscheduler.server.utils` instead of `org.apache.dolphinscheduler.plugin.task.api`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] davidzollo commented on issue #11903: [Bug] [Worker Sever] org.apache.dolphinscheduler.plugin.task.api.ProcessUtils:[66] - kill task failed
Posted by GitBox <gi...@apache.org>.
davidzollo commented on issue #11903:
URL: https://github.com/apache/dolphinscheduler/issues/11903#issuecomment-1248048334
>
this PR can't solve the problem I think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org