You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by mo...@apache.org on 2017/01/13 21:36:46 UTC
zeppelin git commit: [ZEPPELIN-1917] Improve python.conda interpreter
Repository: zeppelin
Updated Branches:
refs/heads/master 2b0e2a41c -> 034fdc673
[ZEPPELIN-1917] Improve python.conda interpreter
### What is this PR for?
Add missing commands to the `python.conda` interpreter
- `conda info`
- `conda list`
- `conda create`
- `conda install`
- `conda uninstall (alias of remove)`
- `conda env *`
#### Implementation Detail
The reason I modified `PythonProcess` is due to NPE
```java
// https://github.com/apache/zeppelin/blob/master/python/src/main/java/org/apache/zeppelin/python/PythonProcess.java#L107-L118
public String sendAndGetResult(String cmd) throws IOException {
writer.println(cmd);
writer.println();
writer.println("\"" + STATEMENT_END + "\"");
StringBuilder output = new StringBuilder();
String line = null;
// NPE when line is null
while (!(line = reader.readLine()).contains(STATEMENT_END)) {
logger.debug("Read line from python shell : " + line);
output.append(line + "\n");
}
return output.toString();
}
```
```
java.lang.NullPointerException
at org.apache.zeppelin.python.PythonProcess.sendAndGetResult(PythonProcess.java:113)
at org.apache.zeppelin.python.PythonInterpreter.sendCommandToPython(PythonInterpreter.java:250)
at org.apache.zeppelin.python.PythonInterpreter.bootStrapInterpreter(PythonInterpreter.java:272)
at org.apache.zeppelin.python.PythonInterpreter.open(PythonInterpreter.java:100)
at org.apache.zeppelin.python.PythonCondaInterpreter.restartPythonProcess(PythonCondaInterpreter.java:139)
at org.apache.zeppelin.python.PythonCondaInterpreter.interpret(PythonCondaInterpreter.java:88)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:494)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
```
### What type of PR is it?
[Improvement | Refactoring]
### Todos
* [x] - info
* [x] - list
* [x] - create
* [x] - install
* [x] - uninstall (= remove)
* [x] - env *
### What is the Jira issue?
[ZEPPELIN-1917](https://issues.apache.org/jira/browse/ZEPPELIN-1917)
### How should this be tested?
1. Install [miniconda](http://conda.pydata.org/miniconda.html)
2. Make sure that your python interpreter can use `conda` (check the Interpreter Binding page)
3. Remove `test` conda env since we will create in the following section
```sh
$ conda env remove --yes --name test
```
4. Run these commands with `%python.conda`
```
%python.conda info
%python.conda env list
%python.conda create --name test
# you should be able to see `test` in the list
%python.conda env list
%python.conda activate pymysql
%python.conda install pymysql
# you should be able to import
%python
import pymysql.cursors
%python.conda uninstall pymysql
%python.conda deactivate pymysql
# you should be able to see `No module named pymysql.cursor` since we deactivated
%python
import pymysql.cursors
```
### Screenshots (if appropriate)
![conda-screenshot](https://cloud.githubusercontent.com/assets/4968473/21747565/98c0e366-d5ad-11e6-8000-e293996089fa.gif)
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - NO
Author: 1ambda <1a...@gmail.com>
Closes #1868 from 1ambda/ZEPPELIN-1917/improve-conda-interpreter and squashes the following commits:
3ba171a [1ambda] fix: Wrap output style
292ed6d [1ambda] refactor: Throw exception in runCommand
2d4aa7d [1ambda] test: Add some tests
49a4a11 [1ambda] feat: Supports other env commands
6eb7e92 [1ambda] fix: NPE in PythonProcess
9c5dd86 [1ambda] refactor: Activate, Deactivate
f955889 [1ambda] fix: minor
935cb89 [1ambda] refactor: Abstract commands
b1c4c9f [1ambda] feat: Add conda remove (uninstall)
e539c42 [1ambda] feat: Add conda install
4f58fa2 [1ambda] feat: Add conda create
7da132d [1ambda] docs: Add missing conda list description
929ca8a [1ambda] feat: Make conda output beautiful
0c6ebb4 [1ambda] feat: Add list conda command
017c76f [1ambda] refactor: Import InterpreterResult.{Code, Type} to short codes
b8a5154 [1ambda] refactor: Simplify exception flow so private funcs don't need care exceptions
64d4bef [1ambda] style: Rename some funcs
afc456d [1ambda] refactor: Add private to member vars
f36fc74 [1ambda] feat: Add info command
2eb9bf5 [1ambda] style: Remove useless newlines
bd2564e [1ambda] refactor: PythonCondaInterpreter.interpret
f0d69bc [1ambda] fix: Use specific command for env list in conda
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/034fdc67
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/034fdc67
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/034fdc67
Branch: refs/heads/master
Commit: 034fdc6735e075c89f727bb6bc6fddbc89b639c4
Parents: 2b0e2a4
Author: 1ambda <1a...@gmail.com>
Authored: Wed Jan 11 07:56:35 2017 +0900
Committer: Lee moon soo <mo...@apache.org>
Committed: Fri Jan 13 13:36:37 2017 -0800
----------------------------------------------------------------------
.../zeppelin/python/PythonCondaInterpreter.java | 319 ++++++++++++++-----
.../apache/zeppelin/python/PythonProcess.java | 5 +-
.../resources/output_templates/conda_usage.html | 25 +-
.../python/PythonCondaInterpreterTest.java | 63 ++--
4 files changed, 307 insertions(+), 105 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/034fdc67/python/src/main/java/org/apache/zeppelin/python/PythonCondaInterpreter.java
----------------------------------------------------------------------
diff --git a/python/src/main/java/org/apache/zeppelin/python/PythonCondaInterpreter.java b/python/src/main/java/org/apache/zeppelin/python/PythonCondaInterpreter.java
index 304e1f0..455d786 100644
--- a/python/src/main/java/org/apache/zeppelin/python/PythonCondaInterpreter.java
+++ b/python/src/main/java/org/apache/zeppelin/python/PythonCondaInterpreter.java
@@ -16,14 +16,16 @@
*/
package org.apache.zeppelin.python;
+import org.apache.commons.lang.StringUtils;
import org.apache.zeppelin.interpreter.*;
+import org.apache.zeppelin.interpreter.InterpreterResult.Code;
+import org.apache.zeppelin.interpreter.InterpreterResult.Type;
import org.apache.zeppelin.scheduler.Scheduler;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
-import java.util.HashMap;
-import java.util.Properties;
+import java.util.*;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
@@ -36,11 +38,17 @@ public class PythonCondaInterpreter extends Interpreter {
public static final String CONDA_PYTHON_PATH = "/bin/python";
public static final String DEFAULT_ZEPPELIN_PYTHON = "python";
- Pattern condaEnvListPattern = Pattern.compile("([^\\s]*)[\\s*]*\\s(.*)");
- Pattern listPattern = Pattern.compile("env\\s*list\\s?");
- Pattern activatePattern = Pattern.compile("activate\\s*(.*)");
- Pattern deactivatePattern = Pattern.compile("deactivate");
- Pattern helpPattern = Pattern.compile("help");
+ public static final Pattern PATTERN_OUTPUT_ENV_LIST = Pattern.compile("([^\\s]*)[\\s*]*\\s(.*)");
+ public static final Pattern PATTERN_COMMAND_ENV_LIST = Pattern.compile("env\\s*list\\s?");
+ public static final Pattern PATTERN_COMMAND_ENV = Pattern.compile("env\\s*(.*)");
+ public static final Pattern PATTERN_COMMAND_LIST = Pattern.compile("list");
+ public static final Pattern PATTERN_COMMAND_CREATE = Pattern.compile("create\\s*(.*)");
+ public static final Pattern PATTERN_COMMAND_ACTIVATE = Pattern.compile("activate\\s*(.*)");
+ public static final Pattern PATTERN_COMMAND_DEACTIVATE = Pattern.compile("deactivate");
+ public static final Pattern PATTERN_COMMAND_INSTALL = Pattern.compile("install\\s*(.*)");
+ public static final Pattern PATTERN_COMMAND_UNINSTALL = Pattern.compile("uninstall\\s*(.*)");
+ public static final Pattern PATTERN_COMMAND_HELP = Pattern.compile("help");
+ public static final Pattern PATTERN_COMMAND_INFO = Pattern.compile("info");
public PythonCondaInterpreter(Properties property) {
super(property);
@@ -59,33 +67,53 @@ public class PythonCondaInterpreter extends Interpreter {
@Override
public InterpreterResult interpret(String st, InterpreterContext context) {
InterpreterOutput out = context.out;
+ Matcher activateMatcher = PATTERN_COMMAND_ACTIVATE.matcher(st);
+ Matcher createMatcher = PATTERN_COMMAND_CREATE.matcher(st);
+ Matcher installMatcher = PATTERN_COMMAND_INSTALL.matcher(st);
+ Matcher uninstallMatcher = PATTERN_COMMAND_UNINSTALL.matcher(st);
+ Matcher envMatcher = PATTERN_COMMAND_ENV.matcher(st);
- Matcher listMatcher = listPattern.matcher(st);
- Matcher activateMatcher = activatePattern.matcher(st);
- Matcher deactivateMatcher = deactivatePattern.matcher(st);
- Matcher helpMatcher = helpPattern.matcher(st);
-
- if (st == null || st.isEmpty() || listMatcher.matches()) {
- listEnv(out, getCondaEnvs());
- return new InterpreterResult(InterpreterResult.Code.SUCCESS);
- } else if (activateMatcher.matches()) {
- String envName = activateMatcher.group(1);
- changePythonEnvironment(envName);
- restartPythonProcess();
- return new InterpreterResult(InterpreterResult.Code.SUCCESS, "\"" + envName + "\" activated");
- } else if (deactivateMatcher.matches()) {
- changePythonEnvironment(null);
- restartPythonProcess();
- return new InterpreterResult(InterpreterResult.Code.SUCCESS, "Deactivated");
- } else if (helpMatcher.matches()) {
- printUsage(out);
- return new InterpreterResult(InterpreterResult.Code.SUCCESS);
- } else {
- return new InterpreterResult(InterpreterResult.Code.ERROR, "Not supported command: " + st);
+ try {
+ if (PATTERN_COMMAND_ENV_LIST.matcher(st).matches()) {
+ String result = runCondaEnvList();
+ return new InterpreterResult(Code.SUCCESS, Type.HTML, result);
+ } else if (envMatcher.matches()) {
+ // `envMatcher` should be used after `listEnvMatcher`
+ String result = runCondaEnv(getRestArgsFromMatcher(envMatcher));
+ return new InterpreterResult(Code.SUCCESS, Type.HTML, result);
+ } else if (PATTERN_COMMAND_LIST.matcher(st).matches()) {
+ String result = runCondaList();
+ return new InterpreterResult(Code.SUCCESS, Type.HTML, result);
+ } else if (createMatcher.matches()) {
+ String result = runCondaCreate(getRestArgsFromMatcher(createMatcher));
+ return new InterpreterResult(Code.SUCCESS, Type.HTML, result);
+ } else if (activateMatcher.matches()) {
+ String envName = activateMatcher.group(1).trim();
+ return runCondaActivate(envName);
+ } else if (PATTERN_COMMAND_DEACTIVATE.matcher(st).matches()) {
+ return runCondaDeactivate();
+ } else if (installMatcher.matches()) {
+ String result = runCondaInstall(getRestArgsFromMatcher(installMatcher));
+ return new InterpreterResult(Code.SUCCESS, Type.HTML, result);
+ } else if (uninstallMatcher.matches()) {
+ String result = runCondaUninstall(getRestArgsFromMatcher(uninstallMatcher));
+ return new InterpreterResult(Code.SUCCESS, Type.HTML, result);
+ } else if (st == null || PATTERN_COMMAND_HELP.matcher(st).matches()) {
+ runCondaHelp(out);
+ return new InterpreterResult(Code.SUCCESS);
+ } else if (PATTERN_COMMAND_INFO.matcher(st).matches()) {
+ String result = runCondaInfo();
+ return new InterpreterResult(Code.SUCCESS, Type.HTML, result);
+ } else {
+ return new InterpreterResult(Code.ERROR, "Not supported command: " + st);
+ }
+ } catch (RuntimeException | IOException | InterruptedException e) {
+ throw new InterpreterException(e);
}
}
- private void changePythonEnvironment(String envName) {
+ private void changePythonEnvironment(String envName)
+ throws IOException, InterruptedException {
PythonInterpreter python = getPythonInterpreter();
String binPath = null;
if (envName == null) {
@@ -94,7 +122,7 @@ public class PythonCondaInterpreter extends Interpreter {
binPath = DEFAULT_ZEPPELIN_PYTHON;
}
} else {
- HashMap<String, String> envList = getCondaEnvs();
+ Map<String, String> envList = getCondaEnvs();
for (String name : envList.keySet()) {
if (envName.equals(name)) {
binPath = envList.get(name) + CONDA_PYTHON_PATH;
@@ -114,7 +142,8 @@ public class PythonCondaInterpreter extends Interpreter {
protected PythonInterpreter getPythonInterpreter() {
LazyOpenInterpreter lazy = null;
PythonInterpreter python = null;
- Interpreter p = getInterpreterInTheSameSessionByClassName(PythonInterpreter.class.getName());
+ Interpreter p =
+ getInterpreterInTheSameSessionByClassName(PythonInterpreter.class.getName());
while (p instanceof WrappedInterpreter) {
if (p instanceof LazyOpenInterpreter) {
@@ -130,59 +159,75 @@ public class PythonCondaInterpreter extends Interpreter {
return python;
}
- private HashMap getCondaEnvs() {
- HashMap envList = null;
+ public static String runCondaCommandForTextOutput(String title, List<String> commands)
+ throws IOException, InterruptedException {
- StringBuilder sb = createStringBuilder();
- try {
- int exit = runCommand(sb, "conda", "env", "list");
- if (exit == 0) {
- envList = new HashMap();
- String[] lines = sb.toString().split("\n");
- for (String s : lines) {
- if (s == null || s.isEmpty() || s.startsWith("#")) {
- continue;
- }
- Matcher match = condaEnvListPattern.matcher(s);
-
- if (!match.matches()) {
- continue;
- }
- envList.put(match.group(1), match.group(2));
- }
- }
- } catch (IOException | InterruptedException e) {
- throw new InterpreterException(e);
- }
+ String result = runCommand(commands);
+ return wrapCondaBasicOutputStyle(title, result);
+ }
+
+ private String runCondaCommandForTableOutput(String title, List<String> commands)
+ throws IOException, InterruptedException {
+
+ StringBuilder sb = new StringBuilder();
+ String result = runCommand(commands);
+
+ // use table output for pretty output
+ Map<String, String> envPerName = parseCondaCommonStdout(result);
+ return wrapCondaTableOutputStyle(title, envPerName);
+ }
+
+ protected Map<String, String> getCondaEnvs()
+ throws IOException, InterruptedException {
+ String result = runCommand("conda", "env", "list");
+ Map<String, String> envList = parseCondaCommonStdout(result);
return envList;
}
- private void listEnv(InterpreterOutput out, HashMap<String, String> envList) {
- try {
- out.setType(InterpreterResult.Type.HTML);
- out.write("<h4>Conda environments</h4>\n");
- // start table
- out.write("<div style=\"display:table\">\n");
+ private String runCondaEnvList() throws IOException, InterruptedException {
+ return wrapCondaTableOutputStyle("Environment List", getCondaEnvs());
+ }
- for (String name : envList.keySet()) {
- String path = envList.get(name);
+ private String runCondaEnv(List<String> restArgs)
+ throws IOException, InterruptedException {
- out.write(String.format("<div style=\"display:table-row\">" +
- "<div style=\"display:table-cell;width:150px\">%s</div>" +
- "<div style=\"display:table-cell;\">%s</div>" +
- "</div>\n",
- name, path));
- }
- // end table
- out.write("</div><br />\n");
- out.write("<small><code>%python.conda help</code> for the usage</small>\n");
- } catch (IOException e) {
- throw new InterpreterException(e);
+ restArgs.add(0, "conda");
+ restArgs.add(1, "env");
+ restArgs.add(3, "--yes"); // --yes should be inserted after command
+
+ return runCondaCommandForTextOutput(null, restArgs);
+ }
+
+ private InterpreterResult runCondaActivate(String envName)
+ throws IOException, InterruptedException {
+
+ if (null == envName || envName.isEmpty()) {
+ return new InterpreterResult(Code.ERROR, "Env name should be specified");
}
+
+ changePythonEnvironment(envName);
+ restartPythonProcess();
+
+ return new InterpreterResult(Code.SUCCESS, "'" + envName + "' is activated");
}
+ private InterpreterResult runCondaDeactivate()
+ throws IOException, InterruptedException {
- private void printUsage(InterpreterOutput out) {
+ changePythonEnvironment(null);
+ restartPythonProcess();
+ return new InterpreterResult(Code.SUCCESS, "Deactivated");
+ }
+
+ private String runCondaList() throws IOException, InterruptedException {
+ List<String> commands = new ArrayList<String>();
+ commands.add("conda");
+ commands.add("list");
+
+ return runCondaCommandForTableOutput("Installed Package List", commands);
+ }
+
+ private void runCondaHelp(InterpreterOutput out) {
try {
out.setType(InterpreterResult.Type.HTML);
out.writeResource("output_templates/conda_usage.html");
@@ -191,6 +236,98 @@ public class PythonCondaInterpreter extends Interpreter {
}
}
+ private String runCondaInfo() throws IOException, InterruptedException {
+ List<String> commands = new ArrayList<String>();
+ commands.add("conda");
+ commands.add("info");
+
+ return runCondaCommandForTextOutput("Conda Information", commands);
+ }
+
+ private String runCondaCreate(List<String> restArgs)
+ throws IOException, InterruptedException {
+ restArgs.add(0, "conda");
+ restArgs.add(1, "create");
+ restArgs.add(2, "--yes");
+
+ return runCondaCommandForTextOutput("Environment Creation", restArgs);
+ }
+
+ private String runCondaInstall(List<String> restArgs)
+ throws IOException, InterruptedException {
+
+ restArgs.add(0, "conda");
+ restArgs.add(1, "install");
+ restArgs.add(2, "--yes");
+
+ return runCondaCommandForTextOutput("Package Installation", restArgs);
+ }
+
+ private String runCondaUninstall(List<String> restArgs)
+ throws IOException, InterruptedException {
+
+ restArgs.add(0, "conda");
+ restArgs.add(1, "uninstall");
+ restArgs.add(2, "--yes");
+
+ return runCondaCommandForTextOutput("Package Uninstallation", restArgs);
+ }
+
+ public static String wrapCondaBasicOutputStyle(String title, String content) {
+ StringBuilder sb = new StringBuilder();
+ if (null != title && !title.isEmpty()) {
+ sb.append("<h4>").append(title).append("</h4>\n")
+ .append("</div><br />\n");
+ }
+ sb.append("<div style=\"white-space:pre-wrap;\">\n")
+ .append(content)
+ .append("</div>");
+
+ return sb.toString();
+ }
+
+ public static String wrapCondaTableOutputStyle(String title, Map<String, String> kv) {
+ StringBuilder sb = new StringBuilder();
+
+ if (null != title && !title.isEmpty()) {
+ sb.append("<h4>").append(title).append("</h4>\n");
+ }
+
+ sb.append("<div style=\"display:table;white-space:pre-wrap;\">\n");
+ for (String name : kv.keySet()) {
+ String path = kv.get(name);
+
+ sb.append(String.format("<div style=\"display:table-row\">" +
+ "<div style=\"display:table-cell;width:150px\">%s</div>" +
+ "<div style=\"display:table-cell;\">%s</div>" +
+ "</div>\n",
+ name, path));
+ }
+ sb.append("</div>\n");
+
+ return sb.toString();
+ }
+
+ public static Map<String, String> parseCondaCommonStdout(String out)
+ throws IOException, InterruptedException {
+
+ Map<String, String> kv = new LinkedHashMap<String, String>();
+ String[] lines = out.split("\n");
+ for (String s : lines) {
+ if (s == null || s.isEmpty() || s.startsWith("#")) {
+ continue;
+ }
+ Matcher match = PATTERN_OUTPUT_ENV_LIST.matcher(s);
+
+ if (!match.matches()) {
+ continue;
+ }
+ kv.put(match.group(1), match.group(2));
+ }
+
+ return kv;
+ }
+
@Override
public void cancel(InterpreterContext context) {
@@ -206,7 +343,6 @@ public class PythonCondaInterpreter extends Interpreter {
return 0;
}
-
/**
* Use python interpreter's scheduler.
* To make sure %python.conda paragraph and %python paragraph runs sequentially
@@ -221,9 +357,12 @@ public class PythonCondaInterpreter extends Interpreter {
}
}
- protected int runCommand(StringBuilder sb, String ... command)
+ public static String runCommand(List<String> commands)
throws IOException, InterruptedException {
- ProcessBuilder builder = new ProcessBuilder(command);
+
+ StringBuilder sb = new StringBuilder();
+
+ ProcessBuilder builder = new ProcessBuilder(commands);
builder.redirectErrorStream(true);
Process process = builder.start();
InputStream stdout = process.getInputStream();
@@ -234,10 +373,28 @@ public class PythonCondaInterpreter extends Interpreter {
sb.append("\n");
}
int r = process.waitFor(); // Let the process finish.
- return r;
+
+ if (r != 0) {
+ throw new RuntimeException("Failed to execute `" +
+ StringUtils.join(commands, " ") + "` exited with " + r);
+ }
+
+ return sb.toString();
+ }
+
+ public static String runCommand(String ... command)
+ throws IOException, InterruptedException {
+
+ List<String> list = new ArrayList<>(command.length);
+ for (String arg : command) {
+ list.add(arg);
+ }
+
+ return runCommand(list);
}
- protected StringBuilder createStringBuilder() {
- return new StringBuilder();
+ public static List<String> getRestArgsFromMatcher(Matcher m) {
+ // Arrays.asList just returns fixed-size, so we should use ctor instead of
+ return new ArrayList<>(Arrays.asList(m.group(1).split(" ")));
}
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/034fdc67/python/src/main/java/org/apache/zeppelin/python/PythonProcess.java
----------------------------------------------------------------------
diff --git a/python/src/main/java/org/apache/zeppelin/python/PythonProcess.java b/python/src/main/java/org/apache/zeppelin/python/PythonProcess.java
index ef9bd2d..578ffeb 100644
--- a/python/src/main/java/org/apache/zeppelin/python/PythonProcess.java
+++ b/python/src/main/java/org/apache/zeppelin/python/PythonProcess.java
@@ -110,10 +110,13 @@ public class PythonProcess {
writer.println("\"" + STATEMENT_END + "\"");
StringBuilder output = new StringBuilder();
String line = null;
- while (!(line = reader.readLine()).contains(STATEMENT_END)) {
+
+ while ((line = reader.readLine()) != null &&
+ !line.contains(STATEMENT_END)) {
logger.debug("Read line from python shell : " + line);
output.append(line + "\n");
}
+
return output.toString();
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/034fdc67/python/src/main/resources/output_templates/conda_usage.html
----------------------------------------------------------------------
diff --git a/python/src/main/resources/output_templates/conda_usage.html b/python/src/main/resources/output_templates/conda_usage.html
index 79191b8..e1146fc 100644
--- a/python/src/main/resources/output_templates/conda_usage.html
+++ b/python/src/main/resources/output_templates/conda_usage.html
@@ -13,6 +13,18 @@ limitations under the License.
-->
<h4>Usage</h4>
<div>
+ Get the Conda Infomation
+ <pre>%python.conda info</pre>
+</div>
+<div>
+ List the Conda environments
+ <pre>%python.conda env list</pre>
+</div>
+<div>
+ Create a conda enviornment
+ <pre>%python.conda create --name [ENV NAME]</pre>
+</div>
+<div>
Activate an environment (python interpreter will be restarted)
<pre>%python.conda activate [ENV NAME]</pre>
</div>
@@ -21,7 +33,14 @@ limitations under the License.
<pre>%python.conda deactivate</pre>
</div>
<div>
- List the Conda environments
- <pre>%python.conda</pre>
+ Get installed package list inside the current environment
+ <pre>%python.conda list</pre>
+</div>
+<div>
+ Install Package
+ <pre>%python.conda install [PACKAGE NAME]</pre>
+</div>
+<div>
+ Uninstall Package
+ <pre>%python.conda uninstall [PACKAGE NAME]</pre>
</div>
-
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/034fdc67/python/src/test/java/org/apache/zeppelin/python/PythonCondaInterpreterTest.java
----------------------------------------------------------------------
diff --git a/python/src/test/java/org/apache/zeppelin/python/PythonCondaInterpreterTest.java b/python/src/test/java/org/apache/zeppelin/python/PythonCondaInterpreterTest.java
index b654d2e..c6d2a84 100644
--- a/python/src/test/java/org/apache/zeppelin/python/PythonCondaInterpreterTest.java
+++ b/python/src/test/java/org/apache/zeppelin/python/PythonCondaInterpreterTest.java
@@ -23,13 +23,11 @@ import org.junit.Before;
import org.junit.Test;
import java.io.IOException;
-import java.util.Arrays;
-import java.util.HashMap;
-import java.util.Properties;
+import java.util.*;
+import java.util.regex.Matcher;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
-import static org.mockito.Matchers.*;
import static org.mockito.Mockito.*;
public class PythonCondaInterpreterTest {
@@ -49,35 +47,32 @@ public class PythonCondaInterpreterTest {
doReturn(python).when(conda).getPythonInterpreter();
}
- private void setCondaEnvs() throws IOException, InterruptedException {
- StringBuilder sb = new StringBuilder();
- sb.append("#comment\n\nenv1 * /path1\nenv2\t/path2\n");
-
- doReturn(sb).when(conda).createStringBuilder();
- doReturn(0).when(conda)
- .runCommand(any(StringBuilder.class), anyString(), anyString(), anyString());
+ private void setMockCondaEnvList() throws IOException, InterruptedException {
+ Map<String, String> envList = new LinkedHashMap<String, String>();
+ envList.put("env1", "/path1");
+ envList.put("env2", "/path2");
+ doReturn(envList).when(conda).getCondaEnvs();
}
@Test
public void testListEnv() throws IOException, InterruptedException {
- setCondaEnvs();
+ setMockCondaEnvList();
// list available env
InterpreterContext context = getInterpreterContext();
- InterpreterResult result = conda.interpret("", context);
+ InterpreterResult result = conda.interpret("env list", context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
- context.out.flush();
- String out = new String(context.out.toByteArray());
- assertTrue(out.contains(">env1<"));
- assertTrue(out.contains(">/path1<"));
- assertTrue(out.contains(">env2<"));
- assertTrue(out.contains(">/path2<"));
+ assertTrue(result.toString().contains(">env1<"));
+ assertTrue(result.toString().contains("/path1<"));
+ assertTrue(result.toString().contains(">env2<"));
+ assertTrue(result.toString().contains("/path2<"));
}
@Test
public void testActivateEnv() throws IOException, InterruptedException {
- setCondaEnvs();
+ setMockCondaEnvList();
+
InterpreterContext context = getInterpreterContext();
conda.interpret("activate env1", context);
verify(python, times(1)).open();
@@ -94,6 +89,34 @@ public class PythonCondaInterpreterTest {
verify(python).setPythonCommand("python");
}
+ @Test
+ public void testParseCondaCommonStdout()
+ throws IOException, InterruptedException {
+
+ StringBuilder sb = new StringBuilder()
+ .append("# comment1\n")
+ .append("# comment2\n")
+ .append("env1 /location1\n")
+ .append("env2 /location2\n");
+
+ Map<String, String> locationPerEnv =
+ PythonCondaInterpreter.parseCondaCommonStdout(sb.toString());
+
+ assertEquals("/location1", locationPerEnv.get("env1"));
+ assertEquals("/location2", locationPerEnv.get("env2"));
+ }
+
+ @Test
+ public void testGetRestArgsFromMatcher() {
+ Matcher m =
+ PythonCondaInterpreter.PATTERN_COMMAND_ENV.matcher("env remove --name test --yes");
+ m.matches();
+
+ List<String> restArgs = PythonCondaInterpreter.getRestArgsFromMatcher(m);
+ List<String> expected = Arrays.asList(new String[]{"remove", "--name", "test", "--yes"});
+ assertEquals(expected, restArgs);
+ }
+
private InterpreterContext getInterpreterContext() {
return new InterpreterContext(
"noteId",