You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by zjffdu <gi...@git.apache.org> on 2018/06/21 09:29:59 UTC
[GitHub] zeppelin pull request #3039: ZEPPELIN-3348. Enable paragraph level propertie...
GitHub user zjffdu opened a pull request:
https://github.com/apache/zeppelin/pull/3039
ZEPPELIN-3348. Enable paragraph level properties
### What is this PR for?
This PR is trying to introduce paragraph level properties so that user can customize the interpreter behavior in paragraph level. User can specify the paragraph level properties after the repl name (e.g. `%spark(pool=p1, imageWidth=200)`)
The jdbc interpreter is another example, but we use other kind of hack method for that before. This PR also update jdbc interpreter to adopt the paragraph level properties.
### What type of PR is it?
[Feature]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3348
### How should this be tested?
* CI pass
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-3348
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/3039.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3039
----
commit cc54a1b1b3a6fa9a8385d650840b4789b4c98053
Author: Jeff Zhang <zj...@...>
Date: 2018-06-21T06:04:40Z
ZEPPELIN-3348. Enable paragraph level properties
----
---
[GitHub] zeppelin issue #3039: ZEPPELIN-3348. Enable paragraph level properties
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/3039
Specifically in this PR, LGTM though
---
[GitHub] zeppelin issue #3039: ZEPPELIN-3348. Enable paragraph level properties
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/3039
Thanks @mebelousov for the feedback. We could do that in a followup PR to improve user friendliness. Because currently I can not imagine in what kind scenario user would add lots of parameters. I would be happy to see the real use case for that.
---
[GitHub] zeppelin issue #3039: ZEPPELIN-3348. Enable paragraph level properties
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/3039
In my guess, you might want to change some spark properties with executing Spark paragraph. In this case, we might need to restart SparkInterpreter to adopt some properties like `spark.executor.memory` and so on.
---
[GitHub] zeppelin issue #3039: ZEPPELIN-3348. Enable paragraph level properties
Posted by mebelousov <gi...@git.apache.org>.
Github user mebelousov commented on the issue:
https://github.com/apache/zeppelin/pull/3039
@zjffdu great powerful feature!
When the parameters will be a lot, it will be not easy to find and write the parameter name. I wish it would be more user-friendly with drop-down panel.
Also parameters must be explicitly displayed for easy debug as in your case.
Does make sense to add some user functionality?
---
[GitHub] zeppelin issue #3039: ZEPPELIN-3348. Enable paragraph level properties
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/3039
Sorry @jongyoul I don't understand that. Why do you think it would affect restarting interpreters ? Do you have any more detail examples ?
---
[GitHub] zeppelin pull request #3039: ZEPPELIN-3348. Enable paragraph level propertie...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/3039
---
[GitHub] zeppelin issue #3039: ZEPPELIN-3348. Enable paragraph level properties
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/3039
Thanks for review, will merge if no more comments
---
[GitHub] zeppelin issue #3039: ZEPPELIN-3348. Enable paragraph level properties
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/3039
@Leemoonsoo @prabhjyotsingh @felixcheung @jongyoul Could you help review it ? Thanks
---
[GitHub] zeppelin issue #3039: ZEPPELIN-3348. Enable paragraph level properties
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/3039
@zjffdu Thanks for explaining. I thought it could change all of the properties by this settings.
---
[GitHub] zeppelin issue #3039: ZEPPELIN-3348. Enable paragraph level properties
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/3039
Hi, basically, parsing properties with every execution may not affect much in performance on interpreter side. But It might affect the behaviours of restarting interpreters. e. g. every restarting while running spark Interpreter with different paragraphs. Does it make sense as well? Otherwise, do you have any further steps to handle this kind of behaviours?
---
[GitHub] zeppelin issue #3039: ZEPPELIN-3348. Enable paragraph level properties
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/3039
Noļ¼this is not this ticket's purpose. If user want to change interpreter level properties, they need inline conf interpreter. This ticket is to allow customize paragraph level behavior, e.g. pool of spark job, image size of current paragraph. So user can not specify `spark.executor.memory` as paragraph level properties, because it is interpreter level property.
---