You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by zjffdu <gi...@git.apache.org> on 2018/03/06 09:21:53 UTC
[GitHub] zeppelin pull request #2839: ZEPPELIN-2515. After 100 minutes R process quit...
GitHub user zjffdu opened a pull request:
https://github.com/apache/zeppelin/pull/2839
ZEPPELIN-2515. After 100 minutes R process quits silently and spark.r interpreter becomes unresponsive
### What is this PR for?
This PR would 2 features:
1. Make timeout of sparkr backend configurable.
2. Detect R backend dead and display proper message to frontend.
### What type of PR is it?
[Bug Fix]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2515
### How should this be tested?
* unit test is added
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-2515
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2839.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2839
----
commit 62888c51289d5c1817c5889d24fecdadf1777bd2
Author: Jeff Zhang <zj...@...>
Date: 2018-03-06T07:02:19Z
ZEPPELIN-2515. After 100 minutes R process quits silently and spark.r interpreter becomes unresponsive
----
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by MeethuM <gi...@git.apache.org>.
Github user MeethuM commented on the issue:
https://github.com/apache/zeppelin/pull/2839
maven-3.3.3 and openjdk version "1.8.0_141"
---
[GitHub] zeppelin pull request #2839: ZEPPELIN-2515. After 100 minutes R process quit...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/2839
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2839
@MeethuM Could you try maven 3.3.9 which is what I use ?
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2839
It is integer type, so you can set it as large as java int can be. e.g. 31536000 which is one year (365*24*60*60)
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2839
Could you try `mvn clean package -DskipTests` ?
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by MeethuM <gi...@git.apache.org>.
Github user MeethuM commented on the issue:
https://github.com/apache/zeppelin/pull/2839
Yes i tried mvn clean package -DskipTests.
I could see the same error in travis log as well @ https://travis-ci.org/zjffdu/zeppelin/jobs/349699753
> [ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark2-shims: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by MeethuM <gi...@git.apache.org>.
Github user MeethuM commented on the issue:
https://github.com/apache/zeppelin/pull/2839
I was able to build it successfully and test in my local environment was successful. I am trying to test it again in a cluster environment to make sure everything is working fine. Could you please wait for one more day?
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2839
@MeethuM What JDK & maven version do are you use ?
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2839
@MeethuM There's no limitation on that.
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by MeethuM <gi...@git.apache.org>.
Github user MeethuM commented on the issue:
https://github.com/apache/zeppelin/pull/2839
I tried to build with the command mvn clean package -DskipTests -Pspark-2.0 -Phadoop-2.4 -Pyarn -Ppyspark -Psparkr -Pr -Pscala-2.11 . But its failing with
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check (checkstyle-fail-build) on project zeppelin: Unable to parse configuration of mojo org.apache.maven.plugins:maven-checkstyle-plugin:2.17:check for parameter sourceDirectories: Cannot assign configuration entry 'sourceDirectories' with value '/home/meethu/git/zeppelin-test-3/src/main/java,/home/meethu/git/zeppelin-test-3/src/main/scala' of type java.lang.String to property of type java.util.List -> [Help 1]
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by MeethuM <gi...@git.apache.org>.
Github user MeethuM commented on the issue:
https://github.com/apache/zeppelin/pull/2839
@zjffdu its working. spark.r.backendConnectionTimeout value is taken into consideration for the r interpreter to quit. Is there any maximum limit for this parameter? I dont want my R interpreter to quit at any point of time.
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2839
@MeethuM Have you tried `mvn clean package -DskipTests` ? It is weird that you hit this kind of issue as travis can build it for quite a long time.
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by MeethuM <gi...@git.apache.org>.
Github user MeethuM commented on the issue:
https://github.com/apache/zeppelin/pull/2839
Got the following error while building
> [ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark1-shims: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
Then I tried the command mvn package -DskipTests -DrecompileMode=all and got this error
> [ERROR] Failed to execute goal on project spark1-shims: Could not resolve dependencies for project org.apache.zeppelin:spark1-shims:jar:0.9.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.zeppelin:spark-shims:jar:0.9.0-SNAPSHOT, org.apache.zeppelin:zeppelin-interpreter:jar:0.9.0-SNAPSHOT: Could not find artifact org.apache.zeppelin:spark-shims:jar:0.9.0-SNAPSHOT in apache.snapshots (http://repository.apache.org/snapshots) -> [Help 1]
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2839
@MeethuM I will merge it if no more comments, as I believe the unit test ensure the bug if fixed.
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2839
@MeethuM Usually this kind of error is due to corrupted jar. For travis build, I can fix it via clean the cache. For your local build, I think you can fix it via deleting the scala jar in your local maven repository.
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by MeethuM <gi...@git.apache.org>.
Github user MeethuM commented on the issue:
https://github.com/apache/zeppelin/pull/2839
Tried mvn clean package -DskipTests, but same error repeated.
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by MeethuM <gi...@git.apache.org>.
Github user MeethuM commented on the issue:
https://github.com/apache/zeppelin/pull/2839
With maven 3.3.9 its building..no error so far..Once build is completed, I will verify and confirm.
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by MeethuM <gi...@git.apache.org>.
Github user MeethuM commented on the issue:
https://github.com/apache/zeppelin/pull/2839
@zjffdu When I gave blank value for the parameter, it didnt work..What will be the best value if we want the interpreter to live forever.
---
[GitHub] zeppelin issue #2839: ZEPPELIN-2515. After 100 minutes R process quits silen...
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/2839
Sure, no problem. And thanks for the careful verifying.
---