zeppelin git commit: [ZEPPELIN-1941] Fix cron job with release resource option dead lock

[jo...@apache.org]

Repository: zeppelin
Updated Branches:
  refs/heads/master 982cc0d17 -> 215599cb3


[ZEPPELIN-1941] Fix cron job with release resource option dead lock

### What is this PR for?
There is a deadlock in concurrent cron job execution with release resource option.
`Scenario`:
Two notebook run with cron job that release resource after job finished.

In `Notebook.CronJob.execute()` method:

`T1. note.runAll(); // locked paragraphs(lock) and wait to interpreterSettings(lock)`

`T2. notebook.getInterpreterFactory().restart() //locked(interpreterSettings) and wait for paragraphs(lock) during jobAbort.`

This will trigger a deadlock that cause zeppelin hang.

### What type of PR is it?
[Bug Fix ]

### Todos
* [x] - Fix this by avoid acquire lock in job abort method.

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1941

### How should this be tested?
Outline the steps to test the PR here.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?

Author: victor.sheng <vi...@qiniu.com>

Closes #1891 from OopsOutOfMemory/fix_dead_lock_cronjob and squashes the following commits:

517fdfa [victor.sheng] fix cron job with release resource option dead lock


Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/215599cb
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/215599cb
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/215599cb

Branch: refs/heads/master
Commit: 215599cb39420a3564f8fcc7ac64a8da748aa526
Parents: 982cc0d
Author: victor.sheng <vi...@qiniu.com>
Authored: Thu Jan 12 14:54:26 2017 +0800
Committer: Jongyoul Lee <jo...@apache.org>
Committed: Tue Jan 17 02:23:49 2017 +0900

----------------------------------------------------------------------
 .../org/apache/zeppelin/notebook/Paragraph.java | 30 +++++++++++++++++++-
 1 file changed, 29 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/zeppelin/blob/215599cb/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Paragraph.java
----------------------------------------------------------------------
diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Paragraph.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Paragraph.java
index a69c0e7..27a7071 100644
--- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Paragraph.java
+++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Paragraph.java
@@ -454,7 +454,7 @@ public class Paragraph extends Job implements Serializable, Cloneable {
     if (job != null) {
       job.setStatus(Status.ABORT);
     } else {
-      repl.cancel(getInterpreterContext(null));
+      repl.cancel(getInterpreterContextWithoutRunner(null));
     }
     return true;
   }
@@ -498,6 +498,34 @@ public class Paragraph extends Job implements Serializable, Cloneable {
     }));
   }
 
+  private InterpreterContext getInterpreterContextWithoutRunner(InterpreterOutput output) {
+    AngularObjectRegistry registry = null;
+    ResourcePool resourcePool = null;
+
+    if (!factory.getInterpreterSettings(note.getId()).isEmpty()) {
+      InterpreterSetting intpGroup = factory.getInterpreterSettings(note.getId()).get(0);
+      registry = intpGroup.getInterpreterGroup(getUser(), note.getId()).getAngularObjectRegistry();
+      resourcePool = intpGroup.getInterpreterGroup(getUser(), note.getId()).getResourcePool();
+    }
+
+    List<InterpreterContextRunner> runners = new LinkedList<>();
+
+    final Paragraph self = this;
+
+    Credentials credentials = note.getCredentials();
+    if (authenticationInfo != null) {
+      UserCredentials userCredentials =
+              credentials.getUserCredentials(authenticationInfo.getUser());
+      authenticationInfo.setUserCredentials(userCredentials);
+    }
+
+    InterpreterContext interpreterContext =
+            new InterpreterContext(note.getId(), getId(), getRequiredReplName(), this.getTitle(),
+            this.getText(), this.getAuthenticationInfo(), this.getConfig(), this.settings, registry,
+            resourcePool, runners, output);
+    return interpreterContext;
+  }
+
   private InterpreterContext getInterpreterContext(InterpreterOutput output) {
     AngularObjectRegistry registry = null;
     ResourcePool resourcePool = null;