You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by jongyoul <gi...@git.apache.org> on 2017/03/15 14:24:37 UTC
[GitHub] zeppelin pull request #2140: [HOTFIX][ZEPPELIN-2037][ZEPPELIN-1832] "Restart...
GitHub user jongyoul opened a pull request:
https://github.com/apache/zeppelin/pull/2140
[HOTFIX][ZEPPELIN-2037][ZEPPELIN-1832] "Restart" button does not work
### What is this PR for?
Fixing restarting interpreters work correctly. All restart buttons runs restarting only user's interpreter instance including "scoped" and "isolated". If you shutdown the server, Zeppelin terminates all interpreters' processes
### What type of PR is it?
[Bug Fix | Hot Fix]
### Todos
* [ ] - Make "Restart" button work properly
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2037
* https://issues.apache.org/jira/browse/ZEPPELIN-1832
### How should this be tested?
1. Enable shiro
1. Login with "admin"
1. Set "Per user" to "scoped"
1. Run "sc.version" in note1 with "admin"
1. Login with "user1"
1. Run "sc.version" in note1 with "user1"
1. Click the "restart" button in note1 page with "admin"
1. Check the process with 'ps aux | grep RemoteInterpreterServer'. Will find one process
1. Click the "restart" button in note1 page with "user1"
1. Check the process with 'ps aux | grep RemoteInterpreterServer'. Won't find any process
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jongyoul/zeppelin ZEPPELIN-2037
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2140.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2140
----
commit 49dbf65864351b142305a3845899e5f5bbab853e
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2017-03-12T13:55:26Z
WIP
Added unit test for all modes
Fixed dereference bug
commit bb4019d96c2cce90f7c8b10db4aac9a02c3b3cff
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2017-03-15T13:43:48Z
Changed the logic of closing interpreter
Changed closing logic of lazyinterpreter to synchronous execution to guarantee the order of execution
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #2140: [HOTFIX][ZEPPELIN-2037][ZEPPELIN-1832] "Restart...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/2140
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2140: [HOTFIX][ZEPPELIN-2037][ZEPPELIN-1832] "Restart" butto...
Posted by cloverhearts <gi...@git.apache.org>.
Github user cloverhearts commented on the issue:
https://github.com/apache/zeppelin/pull/2140
@jongyoul Tested.
Spark and Markdown and Misc interpreter (without shiro)
```
restart interpreter on web ui ----> restart feature well work.
zeppelin shutdown (daemon stop) ---> well working, I can't reproduce with zombie process. :)
```
Thank you for good fix :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2140: [HOTFIX][ZEPPELIN-2037][ZEPPELIN-1832] "Restart" butto...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/2140
@cloverhearts Thanks for testing it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2140: [HOTFIX][ZEPPELIN-2037][ZEPPELIN-1832] "Restart" butto...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/2140
@jongyoul Great work.
I tested this PR and found some problem. We need some followup here.
#### 1. Can not use interpreter after restart in `Per User scoped mode`,
After restart interpreter, user the same interpreter again, then get error
```
ERROR [2017-03-16 08:07:55,589] ({pool-2-thread-14} Job.java[run]:186) - Job failed
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.TApplicationException: Internal error processing interpret
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:361)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:92)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:407)
at org.apache.zeppelin.scheduler.Job.run(Job.java:181)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:329)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.TApplicationException: Internal error processing interpret
at org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_interpret(RemoteInterpreterService.java:274)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.interpret(RemoteInterpreterService.java:258)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:333)
... 11 more
```
#### 2. Interpreter is closed, but process is not terminated in `Per User scoped mode`,
Login with two different user in two browser.
Let's say each user uses the same interpreter, python.
Then there're one `RemoteInterpreterServer` process and two `python` process created.
Once both user restart interpreter, two `python` process are terminated.
However `RemoteInterpreterServer` process still remains even though reference count becomes 0.
#### 3. Per Note scoped mode
Restart button does not restart interpreter.
#### 4. Per User Isolated mode
Restart button in Interpreter binding page does not work
Restart button in interpreter setting page result the same problem 1) 2)
#### 5. Per Note Isolated mode
Restart button in interpreter binding page works as expected
But restart button in interpreter setting page does not restart interpreter. which expected restart all interpreters.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2140: [HOTFIX][ZEPPELIN-2037][ZEPPELIN-1832] "Restart" butto...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/2140
My travis seems to get network error. I'll merge it into master and branch-0.7
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---