You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by zj...@apache.org on 2019/03/14 02:02:29 UTC
[zeppelin] branch master updated: ZEPPELIN-4062. Don't wait ipython
kernel if python process failed
This is an automated email from the ASF dual-hosted git repository.
zjffdu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 65a824e ZEPPELIN-4062. Don't wait ipython kernel if python process failed
65a824e is described below
commit 65a824e9315649a4d349c7717f6b990d76239c9b
Author: Jeff Zhang <zj...@apache.org>
AuthorDate: Wed Mar 13 10:31:00 2019 +0800
ZEPPELIN-4062. Don't wait ipython kernel if python process failed
### What is this PR for?
This PR will exist wait ipython process loop if the python process is failed. This can save time to exit the waiting loop earlier, otherwise it would wait until timeout (30 seconds by default)
### What type of PR is it?
[Improvement]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://jira.apache.org/jira/browse/ZEPPELIN-4062
### How should this be tested?
* Manually tested.
Before this PR
```
INFO [2019-03-13 10:06:30,070] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313) - Wait for IPython Kernel to be started
INFO [2019-03-13 10:06:30,175] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313) - Wait for IPython Kernel to be started
INFO [2019-03-13 10:06:30,281] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313) - Wait for IPython Kernel to be started
INFO [2019-03-13 10:06:30,386] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313) - Wait for IPython Kernel to be started
INFO [2019-03-13 10:06:30,489] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313) - Wait for IPython Kernel to be started
INFO [2019-03-13 10:06:30,595] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313) - Wait for IPython Kernel to be started
INFO [2019-03-13 10:06:30,700] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313) - Wait for IPython Kernel to be started
INFO [2019-03-13 10:06:30,805] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313) - Wait for IPython Kernel to be started
WARN [2019-03-13 10:06:30,810] ({Exec Default Executor} IPythonInterpreter.java[onProcessFailed]:419) - Exception happens in Python Process
org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
at java.lang.Thread.run(Thread.java:748)
INFO [2019-03-13 10:06:30,910] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:313) - Wait for IPython Kernel to be started
INFO [2019-03-13 10:06:31,015] ({FIFOScheduler-Worker-1}
```
After this PR
```
INFO [2019-03-13 10:28:12,805] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314) - Wait for IPython Kernel to be started
INFO [2019-03-13 10:28:12,907] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314) - Wait for IPython Kernel to be started
INFO [2019-03-13 10:28:13,011] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314) - Wait for IPython Kernel to be started
INFO [2019-03-13 10:28:13,116] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314) - Wait for IPython Kernel to be started
INFO [2019-03-13 10:28:13,220] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314) - Wait for IPython Kernel to be started
INFO [2019-03-13 10:28:13,325] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314) - Wait for IPython Kernel to be started
WARN [2019-03-13 10:28:13,347] ({Exec Default Executor} IPythonInterpreter.java[onProcessFailed]:423) - Exception happens in Python Process
org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
at java.lang.Thread.run(Thread.java:748)
INFO [2019-03-13 10:28:13,430] ({FIFOScheduler-Worker-1} IPythonInterpreter.java[launchIPythonKernel]:314) - Wait for IPython Kernel to be started
WARN [2019-03-13 10:28:13,431] ({FIFOScheduler-Worker-1} PythonInterpreter.java[open]:98) - Fail to open IPythonInterpreter
java.lang.RuntimeException: Fail to open IPythonInterpreter
at org.apache.zeppelin.python.IPythonInterpreter.open(IPythonInterpreter.java:152)
at org.apache.zeppelin.python.PythonInterpreter.open(PythonInterpreter.java:93)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:598)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:507)
at org.apache.zeppelin.scheduler.Job.run(Job.java:172)
at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:121)
at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:39)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Fail to launch IPython Kernel as the python process is failed
at org.apache.zeppelin.python.IPythonInterpreter.launchIPythonKernel(IPythonInterpreter.java:323)
at org.apache.zeppelin.python.IPythonInterpreter.open(IPythonInterpreter.java:149)
... 10 more
INFO [2019-03-13 10:28:13,431] ({FIFOScheduler-Worker-1} PythonInterpreter.java[open]:104) - IPython is not available, use the native PythonInterpreter
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zj...@apache.org>
Closes #3332 from zjffdu/ZEPPELIN-4062 and squashes the following commits:
130c2a82a [Jeff Zhang] ZEPPELIN-4062. Don't wait ipython kernel if python process failed
---
.../main/java/org/apache/zeppelin/python/IPythonInterpreter.java | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/python/src/main/java/org/apache/zeppelin/python/IPythonInterpreter.java b/python/src/main/java/org/apache/zeppelin/python/IPythonInterpreter.java
index f5c19be..9e23d04 100644
--- a/python/src/main/java/org/apache/zeppelin/python/IPythonInterpreter.java
+++ b/python/src/main/java/org/apache/zeppelin/python/IPythonInterpreter.java
@@ -82,6 +82,7 @@ public class IPythonInterpreter extends Interpreter implements ExecuteResultHand
private boolean useBuiltinPy4j = true;
private boolean usePy4JAuth = true;
private String secret;
+ private volatile boolean pythonProcessFailed = false;
private InterpreterOutputStream interpreterOutput = new InterpreterOutputStream(LOGGER);
@@ -293,7 +294,7 @@ public class IPythonInterpreter extends Interpreter implements ExecuteResultHand
// wait until IPython kernel is started or timeout
long startTime = System.currentTimeMillis();
- while (true) {
+ while (!pythonProcessFailed) {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
@@ -318,6 +319,9 @@ public class IPythonInterpreter extends Interpreter implements ExecuteResultHand
+ " seconds");
}
}
+ if (pythonProcessFailed) {
+ throw new IOException("Fail to launch IPython Kernel as the python process is failed");
+ }
}
protected Map<String, String> setupIPythonEnv() throws IOException {
@@ -417,6 +421,7 @@ public class IPythonInterpreter extends Interpreter implements ExecuteResultHand
@Override
public void onProcessFailed(ExecuteException e) {
LOGGER.warn("Exception happens in Python Process", e);
+ pythonProcessFailed = true;
}
static class ProcessLogOutputStream extends LogOutputStream {