You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by Leemoonsoo <gi...@git.apache.org> on 2016/11/16 04:24:31 UTC
[GitHub] zeppelin pull request #1645: [ZEPPELIN-1671] Conda interpreter
GitHub user Leemoonsoo opened a pull request:
https://github.com/apache/zeppelin/pull/1645
[ZEPPELIN-1671] Conda interpreter
### What is this PR for?
Conda interpreter that manages conda environment for PythonInterpreter
### What type of PR is it?
Feature
### Todos
* [x] - Basic impl
* [ ] - update doc
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1671
### How should this be tested?
Recreate(or create new) your python interpreter setting in gui.
List all conda env
```
%python.conda
```
Activate env
```
%python.conda activate [name]
```
Deactivate env
```
%python.conda deactivate
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Leemoonsoo/zeppelin conda
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1645.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1645
----
commit 394cf8c00641d251fe023d63cda31a4f872cf3dd
Author: Lee moon soo <mo...@apache.org>
Date: 2016-11-16T04:18:08Z
Conda interpreter implementation
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1645: [ZEPPELIN-1671] Conda interpreter
Posted by edgarson <gi...@git.apache.org>.
Github user edgarson commented on the issue:
https://github.com/apache/zeppelin/pull/1645
The same error occurs when User Impersonation is enabled for Python.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1645: [ZEPPELIN-1671] Conda interpreter
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/1645
Made some improvement on output formatting with basic documentation and unittest.
Please review this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1645: [ZEPPELIN-1671] Conda interpreter
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/1645
Merge if there're no more discussions.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1645: [ZEPPELIN-1671] Conda interpreter
Posted by agoodm <gi...@git.apache.org>.
Github user agoodm commented on the issue:
https://github.com/apache/zeppelin/pull/1645
From what I can tell, the main intent of this PR is to support switching between environments on the fly, which is in of itself a very useful feature. Will this also support creating new environments and installing conda packages into the current environment from within Zeppelin?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1645: [ZEPPELIN-1671] Conda interpreter
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/1645
Is 'conda' command in your PATH?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1645: [ZEPPELIN-1671] Conda interpreter
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo closed the pull request at:
https://github.com/apache/zeppelin/pull/1645
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1645: [WIP] [ZEPPELIN-1671] Conda interpreter
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/1645
@tae-jun @agoodm Thanks for the interest.
Yes, the intent is allow user easily switch between environments.
Supporting more actions like creating new env, installing packages will not be very difficult and make super sense to have. but i'd like to leave them as future work.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1645: [ZEPPELIN-1671] Conda interpreter
Posted by Leemoonsoo <gi...@git.apache.org>.
GitHub user Leemoonsoo reopened a pull request:
https://github.com/apache/zeppelin/pull/1645
[ZEPPELIN-1671] Conda interpreter
### What is this PR for?
Conda interpreter that manages conda environment for PythonInterpreter
### What type of PR is it?
Feature
### Todos
* [x] - Basic impl
* [x] - update doc
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1671
### How should this be tested?
Recreate(or create new) your python interpreter setting in gui.
List all conda env
```
%python.conda
```
Activate env
```
%python.conda activate [name]
```
Deactivate env
```
%python.conda deactivate
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Leemoonsoo/zeppelin conda
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1645.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1645
----
commit 394cf8c00641d251fe023d63cda31a4f872cf3dd
Author: Lee moon soo <mo...@apache.org>
Date: 2016-11-16T04:18:08Z
Conda interpreter implementation
commit 1223796931b38af1c2351fdda91ad31b4dcf814c
Author: Lee moon soo <mo...@apache.org>
Date: 2016-11-16T04:28:21Z
Remove unnecessary log
commit 6b9525f303154c5eda65f12256b9bfc13d7c4c58
Author: Lee moon soo <mo...@apache.org>
Date: 2016-11-17T18:40:13Z
Fix unittest
commit 171cbeb003f59c82af90713807faafea281829b1
Author: Lee moon soo <mo...@apache.org>
Date: 2016-11-18T03:55:47Z
make sure single char interpreter name can be parsed
commit 9ae553b55797c3545b04858928ec2715cbe81d43
Author: Lee moon soo <mo...@apache.org>
Date: 2016-11-18T03:56:17Z
Format output and add usage command
commit b889443008aa91f786a6b6c1dec974dccd51595a
Author: Lee moon soo <mo...@apache.org>
Date: 2016-11-18T03:59:31Z
add usage template
commit d979c6af9aed1383844cd306f4f6fd5490ba1dcb
Author: Lee moon soo <mo...@apache.org>
Date: 2016-11-18T05:40:38Z
Add unittest
commit 4842b0ae2431d77d0980af628a91dd3f71254b22
Author: Lee moon soo <mo...@apache.org>
Date: 2016-11-18T05:59:12Z
Add usage in doc
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1645: [ZEPPELIN-1671] Conda interpreter
Posted by nEosAg007 <gi...@git.apache.org>.
Github user nEosAg007 commented on the issue:
https://github.com/apache/zeppelin/pull/1645
I have installed Miniconda and it is in my $PATH as shown below:
[root@ip-xx-xxx-xx-xxx ~]# echo $PATH
/root/miniconda2/bin:/opt/maven/bin:/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
On 24 June 2017 at 03:42, Lee moon soo <no...@github.com> wrote:
> Is 'conda' command in your PATH?
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/zeppelin/pull/1645#issuecomment-310784287>, or mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/AMT5MHBHh-nOHy_hodPScMw-ygmdP0oGks5sHDhNgaJpZM4KzWi0>
> .
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1645: [ZEPPELIN-1671] Conda interpreter
Posted by pauldj54 <gi...@git.apache.org>.
Github user pauldj54 commented on the issue:
https://github.com/apache/zeppelin/pull/1645
Hi everyone,
I got the same error as @nEosAg007 . I added conda and all anaconda3 libraries to my path (in my case /opt/anaconda3/bin) but this makes python3 the default python in my environment. That makes Zeppelin start crashing with the error
```
apache.zeppelin.interpreter.InterpreterException: File "/usr/bin/hdp-select", line 205
print "ERROR: Invalid package - " + name
^
SyntaxError: Missing parentheses in call to 'print'
ls: cannot access /usr/hdp//hadoop/lib: No such file or directory
Exception in thread "main" java.lang.IllegalStateException: hdp.version is not set while running Spark under HDP, please set through HDP_VERSION in spark-env.sh or add a java-opts file in conf with -Dhdp.version=xxx
```
My platform is a HDP 2.5 with Zeppelin 0.7.3 and this is my zeppelin.sh configuration for Python
```
export PYTHONPATH="${SPARK_HOME}/python:${SPARK_HOME}/python/lib/py4j-0.8.2.1-src.zip"
export SPARK_YARN_USER_ENV="PYTHONPATH=${PYTHONPATH}"
export PYSPARK_DRIVER_PYTHON="/var/opt/teradata/anaconda3/envs/py35/bin/ipython"
export PYSPARK_PYTHON="/var/opt/teradata/anaconda3/envs/py35/bin/python"
export PYLIB="/var/opt/teradata/anaconda3/envs/py35/lib"
```
Is there an optimal way to configure Zeppelin to make it fully functional?
Kind Regards, Paul
---
[GitHub] zeppelin issue #1645: [ZEPPELIN-1671] Conda interpreter
Posted by nEosAg007 <gi...@git.apache.org>.
Github user nEosAg007 commented on the issue:
https://github.com/apache/zeppelin/pull/1645
This is not working with latest 0.7.2, it gives the same error as earlier, as shown below:
But the https://issues.apache.org/jira/browse/ZEPPELIN-1671 shows fixed in 0.7.0
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1645: [ZEPPELIN-1671] Conda interpreter
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/1645
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1645: [ZEPPELIN-1671] Conda interpreter
Posted by tae-jun <gi...@git.apache.org>.
Github user tae-jun commented on the issue:
https://github.com/apache/zeppelin/pull/1645
@Leemoonsoo Awesome! I will test it after updating the doc.
By the way, can I list installed packages of conda?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---