You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by astroshim <gi...@git.apache.org> on 2016/09/27 16:08:42 UTC
[GitHub] zeppelin pull request #1465: [ZEPPELIN-1500] pyspark completion doesn't work...
GitHub user astroshim opened a pull request:
https://github.com/apache/zeppelin/pull/1465
[ZEPPELIN-1500] pyspark completion doesn't work.
### What is this PR for?
@fanlu reported bug on https://github.com/apache/zeppelin/pull/530.
This PR fixes bug of checking condition in PySparkInterpreter.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1500
### How should this be tested?
Outline the steps to test the PR here.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/astroshim/zeppelin ZEPPELIN-1500
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1465.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1465
----
commit ee2a29b95b0c1e5a3968fe0f722b57084d33910a
Author: astroshim <hs...@nflabs.com>
Date: 2016-09-27T16:05:04Z
fix bug of condition check.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1465: [ZEPPELIN-1500] pyspark completion doesn't work.
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim commented on the issue:
https://github.com/apache/zeppelin/pull/1465
re-trigger CI
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1465: [ZEPPELIN-1500] pyspark completion doesn't work.
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1465
why is `pythonScriptInitialized` check removed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1465: [ZEPPELIN-1500] pyspark completion doesn't work...
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim closed the pull request at:
https://github.com/apache/zeppelin/pull/1465
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1465: [ZEPPELIN-1500] pyspark completion doesn't work.
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim commented on the issue:
https://github.com/apache/zeppelin/pull/1465
please review this PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1465: [ZEPPELIN-1500] pyspark completion doesn't work.
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim commented on the issue:
https://github.com/apache/zeppelin/pull/1465
@felixcheung you mean test case?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1465: [ZEPPELIN-1500] pyspark completion doesn't work.
Posted by fanlu <gi...@git.apache.org>.
Github user fanlu commented on the issue:
https://github.com/apache/zeppelin/pull/1465
@astroshim Another problem has emerged
ERROR [2016-09-28 23:29:20,018] ({pool-1-thread-2} TThreadPoolServer.java[run]:296) - Error occurred during processing of message.
java.lang.NullPointerException
at org.apache.zeppelin.spark.PySparkInterpreter.completion(PySparkInterpreter.java:453)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.completion(LazyOpenInterpreter.java:122)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.completion(RemoteInterpreterServer.java:469)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$completion.getResult(RemoteInterpreterService.java:1742)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$completion.getResult(RemoteInterpreterService.java:1727)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1465: [ZEPPELIN-1500] pyspark completion doesn't work.
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1465
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1465: [ZEPPELIN-1500] pyspark completion doesn't work...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/1465
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1465: [ZEPPELIN-1500] pyspark completion doesn't work.
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1465
do we have tests for completion?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1465: [ZEPPELIN-1500] pyspark completion doesn't work.
Posted by fanlu <gi...@git.apache.org>.
Github user fanlu commented on the issue:
https://github.com/apache/zeppelin/pull/1465
I use git apply this patch
but the problem is still there
ERROR [2016-09-28 23:23:53,892] ({qtp659748578-108} NotebookServer.java[onMessage]:261) - Can't handle message
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.TApplicationException: completion failed: unknown result
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.completion(RemoteInterpreter.java:453)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.completion(LazyOpenInterpreter.java:122)
at org.apache.zeppelin.notebook.Paragraph.completion(Paragraph.java:242)
at org.apache.zeppelin.notebook.Note.completion(Note.java:520)
at org.apache.zeppelin.socket.NotebookServer.completion(NotebookServer.java:797)
at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:217)
at org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(NotebookSocket.java:56)
at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextMessage(JettyListenerEventDriver.java:128)
at org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69)
at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:65)
at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextFrame(JettyListenerEventDriver.java:122)
at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:161)
at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:309)
at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:214)
at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:220)
at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:258)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:632)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:480)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.TApplicationException: completion failed: unknown result
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_completion(RemoteInterpreterService.java:362)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.completion(RemoteInterpreterService.java:342)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.completion(RemoteInterpreter.java:449)
... 21 more
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1465: [ZEPPELIN-1500] pyspark completion doesn't work...
Posted by astroshim <gi...@git.apache.org>.
GitHub user astroshim reopened a pull request:
https://github.com/apache/zeppelin/pull/1465
[ZEPPELIN-1500] pyspark completion doesn't work.
### What is this PR for?
@fanlu reported bug on https://github.com/apache/zeppelin/pull/530.
This PR fixes bug of checking condition in PySparkInterpreter.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1500
### How should this be tested?
Outline the steps to test the PR here.
### Questions:
- Does the licenses files need update? no
- Is there breaking changes for older versions? no
- Does this needs documentation? no
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/astroshim/zeppelin ZEPPELIN-1500
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1465.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1465
----
commit ee2a29b95b0c1e5a3968fe0f722b57084d33910a
Author: astroshim <hs...@nflabs.com>
Date: 2016-09-27T16:05:04Z
fix bug of condition check.
commit 60da1c3cfe15075ed8e2bb0609e2a322fed8bb54
Author: astroshim <hs...@nflabs.com>
Date: 2016-09-27T16:18:47Z
fix condition
commit 49282b7bfee6f70aaa56199de4dee91fc82579c1
Author: astroshim <hs...@nflabs.com>
Date: 2016-09-30T06:50:13Z
remove NullPointException
commit 7e93968ffd00cbd6f0d6241249f27dc11455abbb
Author: astroshim <hs...@nflabs.com>
Date: 2016-11-08T09:24:41Z
Merge branch 'ZEPPELIN-1500' of https://github.com/astroshim/zeppelin into ZEPPELIN-1500
commit cb4b0fbbf0b98e5a9525d0a2ab26f395bfee6efd
Author: astroshim <hs...@nflabs.com>
Date: 2016-11-08T09:29:45Z
add testcase
commit e50606ae6932bd289c620e8ecd769983b01101e2
Author: astroshim <hs...@nflabs.com>
Date: 2016-11-08T09:30:36Z
Merge branch 'master' into ZEPPELIN-1500
commit d442c12659661652f3cbbcb14e3ddec4ab3c137b
Author: astroshim <hs...@nflabs.com>
Date: 2016-11-08T13:05:58Z
fix to wait completion data
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1465: [ZEPPELIN-1500] pyspark completion doesn't work...
Posted by astroshim <gi...@git.apache.org>.
GitHub user astroshim reopened a pull request:
https://github.com/apache/zeppelin/pull/1465
[ZEPPELIN-1500] pyspark completion doesn't work.
### What is this PR for?
@fanlu reported bug on https://github.com/apache/zeppelin/pull/530.
This PR fixes bug of checking condition in PySparkInterpreter.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1500
### How should this be tested?
Outline the steps to test the PR here.
### Questions:
- Does the licenses files need update? no
- Is there breaking changes for older versions? no
- Does this needs documentation? no
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/astroshim/zeppelin ZEPPELIN-1500
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1465.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1465
----
commit ee2a29b95b0c1e5a3968fe0f722b57084d33910a
Author: astroshim <hs...@nflabs.com>
Date: 2016-09-27T16:05:04Z
fix bug of condition check.
commit 60da1c3cfe15075ed8e2bb0609e2a322fed8bb54
Author: astroshim <hs...@nflabs.com>
Date: 2016-09-27T16:18:47Z
fix condition
commit 49282b7bfee6f70aaa56199de4dee91fc82579c1
Author: astroshim <hs...@nflabs.com>
Date: 2016-09-30T06:50:13Z
remove NullPointException
commit 7e93968ffd00cbd6f0d6241249f27dc11455abbb
Author: astroshim <hs...@nflabs.com>
Date: 2016-11-08T09:24:41Z
Merge branch 'ZEPPELIN-1500' of https://github.com/astroshim/zeppelin into ZEPPELIN-1500
commit cb4b0fbbf0b98e5a9525d0a2ab26f395bfee6efd
Author: astroshim <hs...@nflabs.com>
Date: 2016-11-08T09:29:45Z
add testcase
commit e50606ae6932bd289c620e8ecd769983b01101e2
Author: astroshim <hs...@nflabs.com>
Date: 2016-11-08T09:30:36Z
Merge branch 'master' into ZEPPELIN-1500
commit d442c12659661652f3cbbcb14e3ddec4ab3c137b
Author: astroshim <hs...@nflabs.com>
Date: 2016-11-08T13:05:58Z
fix to wait completion data
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1465: [ZEPPELIN-1500] pyspark completion doesn't work.
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim commented on the issue:
https://github.com/apache/zeppelin/pull/1465
@felixcheung Thank you for quick review.
It's because we should get result message even if `pythonScriptInitialized` is `true` and I thought `pythonScriptInitialized` on the `completion()` increases the complexity of code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1465: [ZEPPELIN-1500] pyspark completion doesn't work...
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim closed the pull request at:
https://github.com/apache/zeppelin/pull/1465
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1465: [ZEPPELIN-1500] pyspark completion doesn't work...
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim closed the pull request at:
https://github.com/apache/zeppelin/pull/1465
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1465: [ZEPPELIN-1500] pyspark completion doesn't work.
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim commented on the issue:
https://github.com/apache/zeppelin/pull/1465
rebuild CI
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1465: [ZEPPELIN-1500] pyspark completion doesn't work.
Posted by astroshim <gi...@git.apache.org>.
Github user astroshim commented on the issue:
https://github.com/apache/zeppelin/pull/1465
@fanlu That error seems like occurs before launching pyspark interpreter.
Could you please try completion after launching pyspark interpreter if works or not like following?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1465: [ZEPPELIN-1500] pyspark completion doesn't work.
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/1465
LGTM and there're no further discussions
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1465: [ZEPPELIN-1500] pyspark completion doesn't work.
Posted by fanlu <gi...@git.apache.org>.
Github user fanlu commented on the issue:
https://github.com/apache/zeppelin/pull/1465
@astroshim It worked, thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1465: [ZEPPELIN-1500] pyspark completion doesn't work...
Posted by astroshim <gi...@git.apache.org>.
GitHub user astroshim reopened a pull request:
https://github.com/apache/zeppelin/pull/1465
[ZEPPELIN-1500] pyspark completion doesn't work.
### What is this PR for?
@fanlu reported bug on https://github.com/apache/zeppelin/pull/530.
This PR fixes bug of checking condition in PySparkInterpreter.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1500
### How should this be tested?
Outline the steps to test the PR here.
### Questions:
- Does the licenses files need update? no
- Is there breaking changes for older versions? no
- Does this needs documentation? no
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/astroshim/zeppelin ZEPPELIN-1500
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1465.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1465
----
commit ee2a29b95b0c1e5a3968fe0f722b57084d33910a
Author: astroshim <hs...@nflabs.com>
Date: 2016-09-27T16:05:04Z
fix bug of condition check.
commit 60da1c3cfe15075ed8e2bb0609e2a322fed8bb54
Author: astroshim <hs...@nflabs.com>
Date: 2016-09-27T16:18:47Z
fix condition
commit 49282b7bfee6f70aaa56199de4dee91fc82579c1
Author: astroshim <hs...@nflabs.com>
Date: 2016-09-30T06:50:13Z
remove NullPointException
commit 7e93968ffd00cbd6f0d6241249f27dc11455abbb
Author: astroshim <hs...@nflabs.com>
Date: 2016-11-08T09:24:41Z
Merge branch 'ZEPPELIN-1500' of https://github.com/astroshim/zeppelin into ZEPPELIN-1500
commit cb4b0fbbf0b98e5a9525d0a2ab26f395bfee6efd
Author: astroshim <hs...@nflabs.com>
Date: 2016-11-08T09:29:45Z
add testcase
commit e50606ae6932bd289c620e8ecd769983b01101e2
Author: astroshim <hs...@nflabs.com>
Date: 2016-11-08T09:30:36Z
Merge branch 'master' into ZEPPELIN-1500
commit d442c12659661652f3cbbcb14e3ddec4ab3c137b
Author: astroshim <hs...@nflabs.com>
Date: 2016-11-08T13:05:58Z
fix to wait completion data
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---