You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/02/13 06:49:14 UTC
[GitHub] [incubator-dolphinscheduler] gabrywu opened a new issue #1947:
[BUG] ProcessUtils.kill blocked
gabrywu opened a new issue #1947: [BUG] ProcessUtils.kill blocked
URL: https://github.com/apache/incubator-dolphinscheduler/issues/1947
ProcessUtils.kill method always invoke killYarnJob ,no matter what's the task type, why ?
```
public static void kill(TaskInstance taskInstance) {
try {
int processId = taskInstance.getPid();
if(processId == 0 ){
logger.error("process kill failed, process id :{}, task id:{}",
processId, taskInstance.getId());
return ;
}
String cmd = String.format("kill -9 %s", getPidsStr(processId));
logger.info("process id:{}, cmd:{}", processId, cmd);
OSUtils.exeCmd(cmd);
// find log and kill yarn job
killYarnJob(taskInstance);
} catch (Exception e) {
logger.error("kill failed : " + e.getMessage(), e);
}
}
```
In our product environment，the killYarnJob blocked, so all the process instance cannot be killed.
The jstack log:
"Worker-Kill-Thread-Executor" #43 daemon prio=5 os_prio=0 tid=0x00007f0ee5972800 nid=0x15084 waiting on condition [0x00007f0e2c6af000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000004379b7a90> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at io.grpc.stub.ClientCalls$ThreadlessExecutor.waitAndDrain(ClientCalls.java:623)
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:122)
at cn.escheduler.rpc.LogViewServiceGrpc$LogViewServiceBlockingStub.viewLog(LogViewServiceGrpc.java:321)
at cn.escheduler.server.rpc.LogClient.viewLog(LogClient.java:97)
at cn.escheduler.server.utils.ProcessUtils.killYarnJob(ProcessUtils.java:303)
at cn.escheduler.server.utils.ProcessUtils.kill(ProcessUtils.java:270)
at cn.escheduler.server.worker.WorkerServer$3.run(WorkerServer.java:282)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] gabrywu closed issue #1947: [BUG]
ProcessUtils.kill blocked
Posted by GitBox <gi...@apache.org>.
gabrywu closed issue #1947: [BUG] ProcessUtils.kill blocked
URL: https://github.com/apache/incubator-dolphinscheduler/issues/1947
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-dolphinscheduler] gabrywu commented on issue #1947:
[BUG] ProcessUtils.kill blocked
Posted by GitBox <gi...@apache.org>.
gabrywu commented on issue #1947: [BUG] ProcessUtils.kill blocked
URL: https://github.com/apache/incubator-dolphinscheduler/issues/1947#issuecomment-590060875
LogClient has refactored , so close this issue
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services