You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zeppelin.apache.org by jo...@apache.org on 2018/07/19 23:01:23 UTC
zeppelin git commit: [zeppelin-3625] [branch-0.8] Pandasql
interpreter fails to query over python dataframe
Repository: zeppelin
Updated Branches:
refs/heads/branch-0.8 b682ce16b -> e133507da
[zeppelin-3625] [branch-0.8] Pandasql interpreter fails to query over python dataframe
### What is this PR for?
fix pandasql query over dataframe for `branch-0.8`. the reason in this branch seems to be different from `master`. first of all there's NPE with python context being used from pandas without being initialised. after fixing it there's another problem with `_displayhook`.
### What type of PR is it?
[Bug Fix]
### Todos
* [x] - fix npe
* [x] - fix `_displayhook`
* [x] - test
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3625
### How should this be tested?
explained in the issue
### Screenshots (if appropriate)
original NPE different from `master`:
<img width="1232" alt="screen shot 2018-07-15 at 7 26 39 pm" src="https://user-images.githubusercontent.com/1642088/42735603-f01172a0-8891-11e8-87a5-8b359209b889.png">
after fixing npe, issue with `_displayhook`:
<img width="1271" alt="screen shot 2018-07-15 at 7 32 54 pm" src="https://user-images.githubusercontent.com/1642088/42735612-1bc13a34-8892-11e8-9379-af56c46a2ecb.png">
and interpreter logs show
<img width="884" alt="screen shot 2018-07-15 at 7 33 47 pm" src="https://user-images.githubusercontent.com/1642088/42735615-3563e108-8892-11e8-979a-73d9fcf28c0a.png">
at this point not sure why can't execute `__zeppelin__._displayhook()` in ipython kernel and not sure what's the purpose of this command on pandasql->interpret. removing this command resolves issue as below. opinions on this are appreciated in case there're better options
<img width="1270" alt="screen shot 2018-07-15 at 7 37 01 pm" src="https://user-images.githubusercontent.com/1642088/42735636-d83db566-8892-11e8-92fc-383957cbcee1.png">
### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?
Author: Khalid Huseynov <kh...@gmail.com>
Closes #3073 from khalidhuseynov/fix-0.8/ZEPPELIN-3625 and squashes the following commits:
d48418790 [Khalid Huseynov] enable all python tests
bce568971 [Khalid Huseynov] remove _displayhook from pandasql interpret
af11abd1b [Khalid Huseynov] fix npe
Project: http://git-wip-us.apache.org/repos/asf/zeppelin/repo
Commit: http://git-wip-us.apache.org/repos/asf/zeppelin/commit/e133507d
Tree: http://git-wip-us.apache.org/repos/asf/zeppelin/tree/e133507d
Diff: http://git-wip-us.apache.org/repos/asf/zeppelin/diff/e133507d
Branch: refs/heads/branch-0.8
Commit: e133507dac4e16bdb749e822e0d00ab7fba17782
Parents: b682ce1
Author: Khalid Huseynov <kh...@gmail.com>
Authored: Thu Jul 19 12:12:24 2018 +0900
Committer: Jongyoul Lee <jo...@apache.org>
Committed: Fri Jul 20 08:01:16 2018 +0900
----------------------------------------------------------------------
python/pom.xml | 5 -----
.../main/java/org/apache/zeppelin/python/PythonInterpreter.java | 2 +-
.../org/apache/zeppelin/python/PythonInterpreterPandasSql.java | 2 +-
3 files changed, 2 insertions(+), 7 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e133507d/python/pom.xml
----------------------------------------------------------------------
diff --git a/python/pom.xml b/python/pom.xml
index 468a7bc..2e6912c 100644
--- a/python/pom.xml
+++ b/python/pom.xml
@@ -35,11 +35,6 @@
<properties>
<interpreter.name>python</interpreter.name>
<python.py4j.version>0.9.2</python.py4j.version>
- <python.test.exclude>
- **/PythonInterpreterWithPythonInstalledTest.java,
- **/PythonInterpreterPandasSqlTest.java,
- **/PythonInterpreterMatplotlibTest.java
- </python.test.exclude>
<pypi.repo.url>https://pypi.python.org/packages</pypi.repo.url>
<python.py4j.repo.folder>/64/5c/01e13b68e8caafece40d549f232c9b5677ad1016071a48d04cc3895acaa3</python.py4j.repo.folder>
<grpc.version>1.4.0</grpc.version>
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e133507d/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
----------------------------------------------------------------------
diff --git a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
index fab2ed9..cac743d 100644
--- a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
+++ b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreter.java
@@ -554,7 +554,7 @@ public class PythonInterpreter extends Interpreter implements ExecuteResultHandl
}
try {
- interpret(bootstrapCode, context);
+ interpret(bootstrapCode, InterpreterContext.get());
} catch (InterpreterException e) {
throw new IOException(e);
}
http://git-wip-us.apache.org/repos/asf/zeppelin/blob/e133507d/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java
----------------------------------------------------------------------
diff --git a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java
index 54984c3..ea05ce7 100644
--- a/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java
+++ b/python/src/main/java/org/apache/zeppelin/python/PythonInterpreterPandasSql.java
@@ -90,7 +90,7 @@ public class PythonInterpreterPandasSql extends Interpreter {
Interpreter python = getPythonInterpreter();
return python.interpret(
- "__zeppelin__.show(pysqldf('" + st + "'))\n__zeppelin__._displayhook()", context);
+ "__zeppelin__.show(pysqldf('" + st + "'))", context);
}
@Override