You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by rerngvit <gi...@git.apache.org> on 2016/06/23 22:18:51 UTC

[GitHub] zeppelin pull request #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and...

GitHub user rerngvit opened a pull request:

    https://github.com/apache/zeppelin/pull/1077

    [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

    ### What is this PR for?
    This PR applies the new interpreter registration mechanism to KnitR and RRepl.
    
    ### What type of PR is it?
    Improvement
    
    ### Todos
    - Move interpreter registration properties from static block to interpreter-setting.json
    
    ### What is the Jira issue?
    https://issues.apache.org/jira/browse/ZEPPELIN-921
    
    ### How should this be tested?
    1. apply patch
    2. rm -r interpreter/r
    3. rm conf/interpreter.json
    4. mvn clean package -DskipTests -Pspark-1.6 -Psparkr
    5. bin/zeppelin-daemon.sh start
    6. run some paragraph with simple R queries
    
    ### Questions:
    * Does the licenses files need update? No
    * Is there breaking changes for older versions? No
    * Does this needs documentation? No

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/rerngvit/incubator-zeppelin ZEPPELIN-921

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/zeppelin/pull/1077.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1077
    
----
commit 460ec8dfc548ab71f94ddae83991bfe191602a2f
Author: rerngvit <re...@kth.se>
Date:   2016-06-10T06:45:01Z

    [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl
    
    This PR applies the new interpreter registration mechanism to KnitR and RRepl.
    
    Improvement
    
    - Move interpreter registration properties from static block to interpreter-setting.json
    
    https://issues.apache.org/jira/browse/ZEPPELIN-921
    
    1. apply patch
    2. rm -r interpreter/r
    3. rm conf/interpreter.json
    4. mvn clean package -DskipTests -Pspark-1.6 -Psparkr
    5. bin/zeppelin-daemon.sh start
    6. run some paragraph with simple R queries
    
    Questions:
    
    Does the licenses files need update? No
    Is there breaking changes for older versions? No
    Does this needs documentation? No

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:

    https://github.com/apache/zeppelin/pull/1077
  
    @rerngvit How about stopping Zeppelin, removing conf/interpreter.json, starting Zeppelin and testing it again?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:

    https://github.com/apache/zeppelin/pull/1077
  
    @rerngvit I'm not good at R, if you, however, built with `-Pr` and got an error while creating interpreter, AFAIK, you should have a R installed by default. Could you please check this first?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

Posted by elbamos <gi...@git.apache.org>.
Github user elbamos commented on the issue:

    https://github.com/apache/zeppelin/pull/1077
  
    I'm sorry I have not had a chance to finish addressing this. One issue is that a PR to interpreter.sh removed the rinterpreter jar from the classpath. It needs to be put back in. That will cause knitr to work. The second issue is that the other spark interpreter is hardcoded into the configuration files I believe related to livy - so it ignores the registration for rrepl as spark.r. 
    
    > On Jul 5, 2016, at 5:38 PM, Rerngvit Yanggratoke <no...@github.com> wrote:
    > 
    > I think I found the root cause of the problem. The key issue is that currently the build configuration packages "zeppelin-zrinterpreter-0.6.0-SNAPSHOT.jar" under "interpreter/spark" instead of under "interpreter/r" (as similar to other interpreter like "flink" or "sh"). Then, when the registration process is done at "InterpreterFactory.java" method "registerInterpreterFromResource" takes the "interpreter-setting.json" from "zeppelin-spark-0.6.0-SNAPSHOT.jar". I will try change this configuration and see whether it addressed the issue.
    > 
    > \u2014
    > You are receiving this because you were mentioned.
    > Reply to this email directly, view it on GitHub, or mute the thread.
    > 



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:

    https://github.com/apache/zeppelin/pull/1077
  
    @rerngvit Could you please try to build with `-Pr` after rebase it from current master?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

Posted by rerngvit <gi...@git.apache.org>.
Github user rerngvit commented on the issue:

    https://github.com/apache/zeppelin/pull/1077
  
    @jongyoul regarding the description text, according to the [build documentation](https://github.com/apache/zeppelin#build), both options should enable R support. Please let me know, if I miss something. 
    
    Note that I also tried building with "-Pr" instead of "-Psparkr". However, even on fresh master branch (i.e., not this patch at all) will result in a runtime error when trying to run an R paragraph ("org.apache.thrift.TApplicationException: Internal error processing createInterpreter"). I think this also should be resolved, but it is unrelated to this PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

Posted by elbamos <gi...@git.apache.org>.
Github user elbamos commented on the issue:

    https://github.com/apache/zeppelin/pull/1077
  
    @rerngvit Thanks for looking at this!  
    
    There were conflicts introduced with the addition of the livy interpreter (or around that time) because sparkr was hard-coded into some of the configuration files.  I've been waiting for all of that code to stabilize before attempting a fix.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

Posted by rerngvit <gi...@git.apache.org>.
Github user rerngvit commented on the issue:

    https://github.com/apache/zeppelin/pull/1077
  
    @jongyoul Thanks. However, I tried that before and still observed the same behaviour. It seems to be related to something else: i.e., the fact interpreter-setting.json for r interpreter is not registered from InterpreterFactory.java for some reason. Any other suggestions are welcome.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

Posted by rerngvit <gi...@git.apache.org>.
Github user rerngvit commented on the issue:

    https://github.com/apache/zeppelin/pull/1077
  
    I have not got any replies on this. Let me close the issue.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and...

Posted by rerngvit <gi...@git.apache.org>.
Github user rerngvit closed the pull request at:

    https://github.com/apache/zeppelin/pull/1077


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

Posted by rerngvit <gi...@git.apache.org>.
Github user rerngvit commented on the issue:

    https://github.com/apache/zeppelin/pull/1077
  
    @elbamos Thanks for your information. I found that moving "zeppelin-zrinterpreter-0.6.0-SNAPSHOT.jar" to under "interpreter/r" will resolve the issue and enable the registration process for Knitr and RRepl to take place. However, it would result in a different runtime error when trying to run an R paragraph. I think this issue should be waited until that dependant problem is resolved.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

Posted by rerngvit <gi...@git.apache.org>.
Github user rerngvit commented on the issue:

    https://github.com/apache/zeppelin/pull/1077
  
    @jongyoul I already installed R on my machine. I showed an excerpt of R command from my shell terminal below. As I mentioned above, trying to build with -Psparkr is working fine, which indicates that a problem is not the R installation (but rather likely a building issue, which is not related to this PR).
    
    $ R --version
    R version 3.3.0 (2016-05-03) -- "Supposedly Educational"
    Copyright (C) 2016 The R Foundation for Statistical Computing
    Platform: x86_64-apple-darwin15.5.0 (64-bit)
    
    @elbamos Thanks for letting me know. That might be the reason for the problem.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:

    https://github.com/apache/zeppelin/pull/1077
  
    @rerngvit In a description, I think `-Psparkr` is not related to this module. we should use `-Pr`, right?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

Posted by rerngvit <gi...@git.apache.org>.
Github user rerngvit commented on the issue:

    https://github.com/apache/zeppelin/pull/1077
  
    @jongyoul I have tried to rebuild with -Pr after rebase to master. I did not get the error "org.apache.thrift.TApplicationException: Internal error processing createInterpreter" anymore. Instead, I got the error "Prefix not found" when I try to run an R paragraph at runtime. This error does not occur on the master branch. This means that the problem is related to how the dynamic registration is done (this PR). Do you have an idea of what likely the root cause of the problem? Thanks.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1077: [ZEPPELIN-921] Apply new mechanism to KnitR and RRepl

Posted by rerngvit <gi...@git.apache.org>.
Github user rerngvit commented on the issue:

    https://github.com/apache/zeppelin/pull/1077
  
    I think I found the root cause of the problem. The key issue is that currently the build configuration packages "zeppelin-zrinterpreter-0.6.0-SNAPSHOT.jar" under "interpreter/spark" instead of under "interpreter/r" (as similar to other interpreter like "flink" or "sh"). Then, when the registration process is done at "InterpreterFactory.java" method "registerInterpreterFromResource" takes the "interpreter-setting.json" from "zeppelin-spark-0.6.0-SNAPSHOT.jar". I will try change this configuration and see whether it addressed the issue.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---