You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by jongyoul <gi...@git.apache.org> on 2016/06/13 14:24:33 UTC
[GitHub] zeppelin pull request #1005: ZEPPELIN-995 Change scheduler for JDBC interpre...
GitHub user jongyoul opened a pull request:
https://github.com/apache/zeppelin/pull/1005
ZEPPELIN-995 Change scheduler for JDBC interpreter to use concurrent execution
### What is this PR for?
Changed scheduler from FIFO to Parallels in JdbcInterpreter. This is a default behaviour of HiveInterpreter. When we merge all JDBC-like interpreter into JDBC, we need to change default behaviour of JdbcInterpreter.
### What type of PR is it?
[Feature]
### Todos
* [x] - Changed scheduler
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-995
### How should this be tested?
You can run multiple queries simultaneously.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jongyoul/zeppelin ZEPPELIN-995
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1005.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1005
----
commit 3bda98859b0c23df760edca1c03a991f7d167c48
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2016-06-13T14:18:24Z
Changed scheduler from FIFO to Parallels in JdbcInterpreter
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1005: ZEPPELIN-995 Change scheduler for JDBC interpre...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul closed the pull request at:
https://github.com/apache/zeppelin/pull/1005
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1005: ZEPPELIN-995 Change scheduler for JDBC interpre...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/1005
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1005: ZEPPELIN-995 Change scheduler for JDBC interpreter to ...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/1005
@Leemoonsoo I also agree to makes this configurable. I'll follow up this with another PR. I also think we need to replace using getScheduler to another way.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1005: ZEPPELIN-995 Change scheduler for JDBC interpreter to ...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/1005
@Leemoonsoo I've totally understood your idea and agree with you. I'll patch for it and push it again
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1005: ZEPPELIN-995 Change scheduler for JDBC interpreter to ...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/1005
How about make it configurable and set parallel scheduler by default?
Some user might want to run query in parallel, but some might want to avoid run queries in parallel.
Scheduler of SparkSql interpreter can be configurable through 'zeppelin.spark.concurrentSQL'.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1005: ZEPPELIN-995 Change scheduler for JDBC interpreter to ...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/1005
Merging it into master and branch-0.6
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1005: ZEPPELIN-995 Change scheduler for JDBC interpreter to ...
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the issue:
https://github.com/apache/zeppelin/pull/1005
Looks great, thank you for prompt update!
@jongyoul is there a reason to make it configurable from another PR instead of this one? Just curious
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1005: ZEPPELIN-995 Change scheduler for JDBC interpreter to ...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/1005
@jongyoul Right, it's not dynamically configurable. Interpreter need to be restarted to reconfigure.
I think it's possibly related to ZEPPELIN-999 and it's long term plan. Let's say a user want to use jdbc interpreter for hive and mysql. Currently,
A. user can create single interpreter setting and create two configuration for both hive and mysql connection. And select connection via %jdbc(hive) or %jdbc(mysql).
B. or user can create two interpreter setting and each interpreter setting have configuration for hive and mysql connection respectively. In this case, select jdbc connection limited and annoying, because a notebook can not use two or more same type of interpreter settings at the same time. So user have to constantly bind/unbind interpreter setting to switch hive <-> mysql.
If our long term plan is generalize ZEPPELIN-999 and allow user use interpreter setting alias for interpreter selection, like %hive, %mysql, then all disadvantages of approach **B** will be eliminated.
Managing only single jdbc connection in single interpreter setting, with generalized ZEPPELIN-999 will give advantages, like leverage [interpreter authorization](https://issues.apache.org/jira/browse/ZEPPELIN-945). And in this case, i was thinking simple returning appropriate scheduler from getScheduler() by configuration would be enough.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1005: ZEPPELIN-995 Change scheduler for JDBC interpreter to ...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/1005
I was thinking just creating multiple interpreter settings if user want to use different scheduler. Then just returning appropriate scheduler from getScheduler() by configuration would be enough. Isn't it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1005: ZEPPELIN-995 Change scheduler for JDBC interpreter to ...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/1005
@Leemoonsoo I've missed something. I've known it was not configurable dynamically. Let me check.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1005: ZEPPELIN-995 Change scheduler for JDBC interpre...
Posted by jongyoul <gi...@git.apache.org>.
GitHub user jongyoul reopened a pull request:
https://github.com/apache/zeppelin/pull/1005
ZEPPELIN-995 Change scheduler for JDBC interpreter to use concurrent execution
### What is this PR for?
Changed scheduler from FIFO to Parallels in JdbcInterpreter. This is a default behaviour of HiveInterpreter. When we merge all JDBC-like interpreter into JDBC, we need to change default behaviour of JdbcInterpreter.
### What type of PR is it?
[Feature]
### Todos
* [x] - Changed scheduler
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-995
### How should this be tested?
You can run multiple queries simultaneously.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jongyoul/zeppelin ZEPPELIN-995
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1005.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1005
----
commit 3bda98859b0c23df760edca1c03a991f7d167c48
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2016-06-13T14:18:24Z
Changed scheduler from FIFO to Parallels in JdbcInterpreter
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1005: ZEPPELIN-995 Change scheduler for JDBC interpreter to ...
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the issue:
https://github.com/apache/zeppelin/pull/1005
@jongyoul Thank you for taking care of this. I agree this should be ParallelScheduler. \U0001f44d
LGTM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---