You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by Savalek <gi...@git.apache.org> on 2018/06/15 10:55:56 UTC
[GitHub] zeppelin pull request #3024: [ZEPPELIN-3545] save all tables to ResourcePool
GitHub user Savalek opened a pull request:
https://github.com/apache/zeppelin/pull/3024
[ZEPPELIN-3545] save all tables to ResourcePool
### What is this PR for?
Now if paragraph's output contains more than one table in ResourcePool saves only last table.
It would be desirable that in ResoursePool stores all tables.
### What type of PR is it?
Improvement
### What is the Jira issue?
[ZEPPELIN-3545](https://issues.apache.org/jira/browse/ZEPPELIN-3545)
### Screenshots
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/TinkoffCreditSystems/zeppelin ZEPPELIN-3545
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/3024.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3024
----
commit 2209e8d3943da5bea3ecbb5ebd3cd6751e04a3b9
Author: Savalek <de...@...>
Date: 2018-06-09T09:49:32Z
[ZEPPELIN-3545] save all tables to ResourcePool
----
---
[GitHub] zeppelin issue #3024: [ZEPPELIN-3545] save all tables to ResourcePool
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/3024
@mebelousov There're many options for how to specify which result to be stored into resource pool.
e.g.
```
%spark(saveToResourcePool=1,2,4)
```
Or
```
%spark(1.saveToResourcePool=true, 2.saveToResourcePool=true, 4.saveToResourcePool=true)
```
We can discuss more about which is the best approach, the key point here is to allow user to customize it via paragraph levle properties.
---
[GitHub] zeppelin issue #3024: [ZEPPELIN-3545] save all tables to ResourcePool
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/3024
Thanks @Savalek for this contribution, but I think putting all tables into ResourcePool doesn't make sense. As it would occupy lots of memory. I plan to introduce paragraph level properties (ZEPPELIN-3348), so that user can control whether to put the interpreter result into ResourcePool.
---
[GitHub] zeppelin pull request #3024: [ZEPPELIN-3545] save all tables to ResourcePool
Posted by Savalek <gi...@git.apache.org>.
Github user Savalek closed the pull request at:
https://github.com/apache/zeppelin/pull/3024
---
[GitHub] zeppelin issue #3024: [ZEPPELIN-3545] save all tables to ResourcePool
Posted by mebelousov <gi...@git.apache.org>.
Github user mebelousov commented on the issue:
https://github.com/apache/zeppelin/pull/3024
@zjffdu thank you about ResourcePool impoving.
Share please your vision how it would be. For example, a paragraph has 5 table results. How user will define which of them would be added into ResourcePool?
---
[GitHub] zeppelin issue #3024: [ZEPPELIN-3545] save all tables to ResourcePool
Posted by mebelousov <gi...@git.apache.org>.
Github user mebelousov commented on the issue:
https://github.com/apache/zeppelin/pull/3024
@zjffdu
I support adding only selected table results to Resource Pool.
As paragraph can have multiple results than I propose to add result level properties.
---
[GitHub] zeppelin issue #3024: [ZEPPELIN-3545] save all tables to ResourcePool
Posted by zjffdu <gi...@git.apache.org>.
Github user zjffdu commented on the issue:
https://github.com/apache/zeppelin/pull/3024
I plan to introduce one paragraph property to indicate whether the result should be put into ResourcePool (Because I think most of time people don't want to save it into ResourcePool, so it doesn't make sense to save it into ResourcePool by default). The following is what I imagine.
```
%spark(saveToResourcePool=true)
...
spark code
...
```
Regarding your scenario of multiple tables, I am not sure the exact scenario, But at least we could introduce more fine grained properties to control that. It would be better to share your real scenario, so that we can see which approach is better.
---