You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by GitBox <gi...@apache.org> on 2020/04/07 07:46:30 UTC
[GitHub] [zeppelin] zjffdu opened a new pull request #3717: [ZEPPELIN-4717].
Support savepoint for insert statement and non-sql paragraph
zjffdu opened a new pull request #3717: [ZEPPELIN-4717]. Support savepoint for insert statement and non-sql paragraph
URL: https://github.com/apache/zeppelin/pull/3717
### What is this PR for?
A few sentences describing the overall goals of the pull request's commits.
First time? Check out the contributing guide - https://zeppelin.apache.org/contribution/contributions.html
### What type of PR is it?
[Bug Fix | Improvement | Feature | Documentation | Hot Fix | Refactoring]
### Todos
* [ ] - Task
### What is the Jira issue?
* Open an issue on Jira https://issues.apache.org/jira/browse/ZEPPELIN/
* Put link here, and add [ZEPPELIN-*Jira number*] in PR title, eg. [ZEPPELIN-533]
### How should this be tested?
* First time? Setup Travis CI as described on https://zeppelin.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed behavior
* Outline any manual steps to test the PR here.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] zjffdu commented on issue #3717: [ZEPPELIN-4717].
Support savepoint for insert statement and non-sql paragraph
Posted by GitBox <gi...@apache.org>.
zjffdu commented on issue #3717: [ZEPPELIN-4717]. Support savepoint for insert statement and non-sql paragraph
URL: https://github.com/apache/zeppelin/pull/3717#issuecomment-615978173
CI is passed now, https://travis-ci.org/github/zjffdu/zeppelin/builds/676534808
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] alexott commented on a change in pull request #3717:
[ZEPPELIN-4717]. Support savepoint for insert statement and non-sql paragraph
Posted by GitBox <gi...@apache.org>.
alexott commented on a change in pull request #3717: [ZEPPELIN-4717]. Support savepoint for insert statement and non-sql paragraph
URL: https://github.com/apache/zeppelin/pull/3717#discussion_r410867134
##########
File path: flink/src/main/java/org/apache/zeppelin/flink/FlinkSqlInterrpeter.java
##########
@@ -504,24 +499,20 @@ public void callSet(String key, String value, InterpreterContext context) throws
throw new IOException(key + " is not a valid table/sql config, please check link: " +
"https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/table/config.html");
}
- currentConfigOptions.put(key, value);
+ paragraphTableConfigMap.get(context.getParagraphId()).put(key, value);
}
- private void callInsertInto(String sql,
+ public void callInsertInto(String sql,
InterpreterContext context) throws IOException {
if (!isBatch()) {
context.getLocalProperties().put("flink.streaming.insert_into", "true");
}
try {
lock.lock();
- if (context.getLocalProperties().containsKey("parallelism")) {
- this.tbenv.getConfig().getConfiguration()
- .set(ExecutionConfigOptions.TABLE_EXEC_RESOURCE_DEFAULT_PARALLELISM,
- Integer.parseInt(context.getLocalProperties().get("parallelism")));
- }
// set table config from set statement until now.
- for (Map.Entry<String, String> entry : currentConfigOptions.entrySet()) {
+ Map<String, String> paragraphTableConfig = paragraphTableConfigMap.get(context.getParagraphId());
Review comment:
same here
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] alexott commented on a change in pull request #3717:
[ZEPPELIN-4717]. Support savepoint for insert statement and non-sql paragraph
Posted by GitBox <gi...@apache.org>.
alexott commented on a change in pull request #3717: [ZEPPELIN-4717]. Support savepoint for insert statement and non-sql paragraph
URL: https://github.com/apache/zeppelin/pull/3717#discussion_r410867087
##########
File path: flink/src/main/java/org/apache/zeppelin/flink/FlinkSqlInterrpeter.java
##########
@@ -466,34 +478,17 @@ private void callExplain(String sql, InterpreterContext context) throws IOExcept
public void callSelect(String sql, InterpreterContext context) throws IOException {
try {
lock.lock();
- // set parallelism from paragraph local property
- if (context.getLocalProperties().containsKey("parallelism")) {
- this.tbenv.getConfig().getConfiguration()
- .set(ExecutionConfigOptions.TABLE_EXEC_RESOURCE_DEFAULT_PARALLELISM,
- Integer.parseInt(context.getLocalProperties().get("parallelism")));
- }
-
// set table config from set statement until now.
- for (Map.Entry<String, String> entry : currentConfigOptions.entrySet()) {
+ Map<String, String> paragraphTableConfig = paragraphTableConfigMap.get(context.getParagraphId());
Review comment:
Above, we used following construct:
```
Map<String, String> tableConfig = paragraphTableConfigMap.getOrDefault(
context.getParagraphId(), new HashMap<>());
```
can we get `null` here? Maybe instead of having direct access `paragraphTableConfigMap`, we can implement helper function that will guarantee that we always get Map?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services