You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/05/08 13:15:23 UTC
[GitHub] [flink] godfreyhe opened a new pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
godfreyhe opened a new pull request #12042:
URL: https://github.com/apache/flink/pull/12042
## What is the purpose of the change
*Introduce TableEnvironment#createStatementSet api to support accept DML statements and Tables. The planner can optimize all added statements and Tables in StatementSet together and then submit as one job.*
## Brief change log
- *add TableEnvironment#createStatementSet method*
- *add StatementSet interface*
- *add related classes and methods in flink-python*
## Verifying this change
This change added tests and can be verified as follows:
- *Extended TableEnvironmentITCase to verify StatementSet*
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (**yes** / no)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (**yes** / no)
- If yes, how is the feature documented? (not applicable / docs / **JavaDocs** / not documented)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4e7e6ed50a36257a3e12cf9166d2d53742a148de Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835)
* 95af2c710bc1453b1930b46dbed83708f08fcc7a Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dianfu commented on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
dianfu commented on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-626085432
@godfreyhe The changes of python part LGTM.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KurtYoung commented on a change in pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
KurtYoung commented on a change in pull request #12042:
URL: https://github.com/apache/flink/pull/12042#discussion_r422442948
##########
File path: flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java
##########
@@ -654,13 +655,39 @@ public TableResult executeSql(String statement) {
return executeOperation(operations.get(0));
}
+ @Override
+ public StatementSet createStatementSet() {
+ return new StatementSetImpl(this);
+ }
+
@Override
public TableResult executeInternal(List<ModifyOperation> operations) {
- if (operations.size() != 1) {
- throw new TableException("Only one ModifyOperation is supported now.");
- }
+ List<Transformation<?>> transformations = translate(operations);
+ String jobName = extractJobName(operations);
+ Pipeline pipeline = execEnv.createPipeline(transformations, tableConfig, jobName);
+ try {
+ JobClient jobClient = execEnv.executeAsync(pipeline);
+ TableSchema.Builder builder = TableSchema.builder();
+ Object[] affectedRowCounts = new Long[operations.size()];
+ for (int i = 0; i < operations.size(); ++i) {
+ // if only one operation, field name is 'affected_rowcount', else is 'affected_rowcount_$i'
+ String fieldName = "affected_rowcount";
Review comment:
Yes, that will be long. But if we don't show this, after we display some real row counts in the future, it would be impossible for users to understand the meaning.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KurtYoung commented on a change in pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
KurtYoung commented on a change in pull request #12042:
URL: https://github.com/apache/flink/pull/12042#discussion_r422157743
##########
File path: flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java
##########
@@ -654,13 +655,39 @@ public TableResult executeSql(String statement) {
return executeOperation(operations.get(0));
}
+ @Override
+ public StatementSet createStatementSet() {
+ return new StatementSetImpl(this);
+ }
+
@Override
public TableResult executeInternal(List<ModifyOperation> operations) {
- if (operations.size() != 1) {
- throw new TableException("Only one ModifyOperation is supported now.");
- }
+ List<Transformation<?>> transformations = translate(operations);
+ String jobName = extractJobName(operations);
+ Pipeline pipeline = execEnv.createPipeline(transformations, tableConfig, jobName);
+ try {
+ JobClient jobClient = execEnv.executeAsync(pipeline);
+ TableSchema.Builder builder = TableSchema.builder();
+ Object[] affectedRowCounts = new Long[operations.size()];
+ for (int i = 0; i < operations.size(); ++i) {
+ // if only one operation, field name is 'affected_rowcount', else is 'affected_rowcount_$i'
+ String fieldName = "affected_rowcount";
Review comment:
how about using the target table name as the column name?
##########
File path: flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/StatementSetImpl.java
##########
@@ -0,0 +1,102 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.api.internal;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.table.api.ExplainDetail;
+import org.apache.flink.table.api.StatementSet;
+import org.apache.flink.table.api.Table;
+import org.apache.flink.table.api.TableException;
+import org.apache.flink.table.api.TableResult;
+import org.apache.flink.table.catalog.ObjectIdentifier;
+import org.apache.flink.table.catalog.UnresolvedIdentifier;
+import org.apache.flink.table.operations.CatalogSinkModifyOperation;
+import org.apache.flink.table.operations.ModifyOperation;
+import org.apache.flink.table.operations.Operation;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation for {@link StatementSet}.
+ */
+@Internal
+class StatementSetImpl implements StatementSet {
+ private List<ModifyOperation> operations = new ArrayList<>();
+ private final TableEnvironmentInternal tableEnvironment;
Review comment:
move this before `operations`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] godfreyhe commented on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
godfreyhe commented on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-626276027
@KurtYoung I have updated based on comments. please take an look again if you have time
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4e7e6ed50a36257a3e12cf9166d2d53742a148de Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835)
* 95af2c710bc1453b1930b46dbed83708f08fcc7a UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KurtYoung commented on a change in pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
KurtYoung commented on a change in pull request #12042:
URL: https://github.com/apache/flink/pull/12042#discussion_r422440561
##########
File path: flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java
##########
@@ -654,13 +655,39 @@ public TableResult executeSql(String statement) {
return executeOperation(operations.get(0));
}
+ @Override
+ public StatementSet createStatementSet() {
+ return new StatementSetImpl(this);
+ }
+
@Override
public TableResult executeInternal(List<ModifyOperation> operations) {
- if (operations.size() != 1) {
- throw new TableException("Only one ModifyOperation is supported now.");
- }
+ List<Transformation<?>> transformations = translate(operations);
+ String jobName = extractJobName(operations);
+ Pipeline pipeline = execEnv.createPipeline(transformations, tableConfig, jobName);
+ try {
+ JobClient jobClient = execEnv.executeAsync(pipeline);
+ TableSchema.Builder builder = TableSchema.builder();
+ Object[] affectedRowCounts = new Long[operations.size()];
+ for (int i = 0; i < operations.size(); ++i) {
+ // if only one operation, field name is 'affected_rowcount', else is 'affected_rowcount_$i'
+ String fieldName = "affected_rowcount";
Review comment:
Seems too long to me, I think most users are aware of this bigint is representing the row count inserted into the table
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] godfreyhe commented on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
godfreyhe commented on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625811701
@dianfu @hequn8128 could you take a look at python part ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] godfreyhe commented on a change in pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
godfreyhe commented on a change in pull request #12042:
URL: https://github.com/apache/flink/pull/12042#discussion_r422232606
##########
File path: flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java
##########
@@ -654,13 +655,39 @@ public TableResult executeSql(String statement) {
return executeOperation(operations.get(0));
}
+ @Override
+ public StatementSet createStatementSet() {
+ return new StatementSetImpl(this);
+ }
+
@Override
public TableResult executeInternal(List<ModifyOperation> operations) {
- if (operations.size() != 1) {
- throw new TableException("Only one ModifyOperation is supported now.");
- }
+ List<Transformation<?>> transformations = translate(operations);
+ String jobName = extractJobName(operations);
+ Pipeline pipeline = execEnv.createPipeline(transformations, tableConfig, jobName);
+ try {
+ JobClient jobClient = execEnv.executeAsync(pipeline);
+ TableSchema.Builder builder = TableSchema.builder();
+ Object[] affectedRowCounts = new Long[operations.size()];
+ for (int i = 0; i < operations.size(); ++i) {
+ // if only one operation, field name is 'affected_rowcount', else is 'affected_rowcount_$i'
+ String fieldName = "affected_rowcount";
Review comment:
I think "affected_rowcount" is necessary, how about "affected_rowcount_sink_table_name" ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
}, {
"hash" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880",
"triggerID" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 95af2c710bc1453b1930b46dbed83708f08fcc7a Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840)
* 8b565b90f9b2d13629f95c15095229b6c0273c1b Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4e7e6ed50a36257a3e12cf9166d2d53742a148de UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 95af2c710bc1453b1930b46dbed83708f08fcc7a Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4e7e6ed50a36257a3e12cf9166d2d53742a148de Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835)
* 95af2c710bc1453b1930b46dbed83708f08fcc7a Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] godfreyhe commented on a change in pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
godfreyhe commented on a change in pull request #12042:
URL: https://github.com/apache/flink/pull/12042#discussion_r422464046
##########
File path: flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java
##########
@@ -654,13 +655,39 @@ public TableResult executeSql(String statement) {
return executeOperation(operations.get(0));
}
+ @Override
+ public StatementSet createStatementSet() {
+ return new StatementSetImpl(this);
+ }
+
@Override
public TableResult executeInternal(List<ModifyOperation> operations) {
- if (operations.size() != 1) {
- throw new TableException("Only one ModifyOperation is supported now.");
- }
+ List<Transformation<?>> transformations = translate(operations);
+ String jobName = extractJobName(operations);
+ Pipeline pipeline = execEnv.createPipeline(transformations, tableConfig, jobName);
+ try {
+ JobClient jobClient = execEnv.executeAsync(pipeline);
+ TableSchema.Builder builder = TableSchema.builder();
+ Object[] affectedRowCounts = new Long[operations.size()];
+ for (int i = 0; i < operations.size(); ++i) {
+ // if only one operation, field name is 'affected_rowcount', else is 'affected_rowcount_$i'
+ String fieldName = "affected_rowcount";
Review comment:
yes, if there are many sinks.
I will try to extract sink identifier names as the shortest form (also make sure them are unique) to reduce the length of field name
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
}, {
"hash" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880",
"triggerID" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"triggerType" : "PUSH"
}, {
"hash" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888",
"triggerID" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"triggerType" : "PUSH"
}, {
"hash" : "a3ce5de621cabc2ca7aba0037d1616afa6178b17",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=899",
"triggerID" : "a3ce5de621cabc2ca7aba0037d1616afa6178b17",
"triggerType" : "PUSH"
}, {
"hash" : "b81c9a59867b94c5941f43087f7dd2a359ba07a5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=904",
"triggerID" : "b81c9a59867b94c5941f43087f7dd2a359ba07a5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ce5de621cabc2ca7aba0037d1616afa6178b17 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=899)
* b81c9a59867b94c5941f43087f7dd2a359ba07a5 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=904)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
}, {
"hash" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880",
"triggerID" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"triggerType" : "PUSH"
}, {
"hash" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888",
"triggerID" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 67032e8cb903a95ec384d6e5e55150dc5c4a8144 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
}, {
"hash" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880",
"triggerID" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"triggerType" : "PUSH"
}, {
"hash" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888",
"triggerID" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8b565b90f9b2d13629f95c15095229b6c0273c1b Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880)
* 67032e8cb903a95ec384d6e5e55150dc5c4a8144 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
}, {
"hash" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880",
"triggerID" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"triggerType" : "PUSH"
}, {
"hash" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888",
"triggerID" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"triggerType" : "PUSH"
}, {
"hash" : "a3ce5de621cabc2ca7aba0037d1616afa6178b17",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=899",
"triggerID" : "a3ce5de621cabc2ca7aba0037d1616afa6178b17",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 67032e8cb903a95ec384d6e5e55150dc5c4a8144 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888)
* a3ce5de621cabc2ca7aba0037d1616afa6178b17 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=899)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
}, {
"hash" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880",
"triggerID" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"triggerType" : "PUSH"
}, {
"hash" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888",
"triggerID" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"triggerType" : "PUSH"
}, {
"hash" : "a3ce5de621cabc2ca7aba0037d1616afa6178b17",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a3ce5de621cabc2ca7aba0037d1616afa6178b17",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 67032e8cb903a95ec384d6e5e55150dc5c4a8144 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888)
* a3ce5de621cabc2ca7aba0037d1616afa6178b17 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
}, {
"hash" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880",
"triggerID" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"triggerType" : "PUSH"
}, {
"hash" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888",
"triggerID" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 95af2c710bc1453b1930b46dbed83708f08fcc7a Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840)
* 8b565b90f9b2d13629f95c15095229b6c0273c1b Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880)
* 67032e8cb903a95ec384d6e5e55150dc5c4a8144 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
}, {
"hash" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880",
"triggerID" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"triggerType" : "PUSH"
}, {
"hash" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888",
"triggerID" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"triggerType" : "PUSH"
}, {
"hash" : "a3ce5de621cabc2ca7aba0037d1616afa6178b17",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=899",
"triggerID" : "a3ce5de621cabc2ca7aba0037d1616afa6178b17",
"triggerType" : "PUSH"
}, {
"hash" : "b81c9a59867b94c5941f43087f7dd2a359ba07a5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b81c9a59867b94c5941f43087f7dd2a359ba07a5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ce5de621cabc2ca7aba0037d1616afa6178b17 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=899)
* b81c9a59867b94c5941f43087f7dd2a359ba07a5 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
}, {
"hash" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880",
"triggerID" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"triggerType" : "PUSH"
}, {
"hash" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888",
"triggerID" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"triggerType" : "PUSH"
}, {
"hash" : "a3ce5de621cabc2ca7aba0037d1616afa6178b17",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=899",
"triggerID" : "a3ce5de621cabc2ca7aba0037d1616afa6178b17",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a3ce5de621cabc2ca7aba0037d1616afa6178b17 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=899)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] godfreyhe commented on a change in pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
godfreyhe commented on a change in pull request #12042:
URL: https://github.com/apache/flink/pull/12042#discussion_r422441179
##########
File path: flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java
##########
@@ -654,13 +655,39 @@ public TableResult executeSql(String statement) {
return executeOperation(operations.get(0));
}
+ @Override
+ public StatementSet createStatementSet() {
+ return new StatementSetImpl(this);
+ }
+
@Override
public TableResult executeInternal(List<ModifyOperation> operations) {
- if (operations.size() != 1) {
- throw new TableException("Only one ModifyOperation is supported now.");
- }
+ List<Transformation<?>> transformations = translate(operations);
+ String jobName = extractJobName(operations);
+ Pipeline pipeline = execEnv.createPipeline(transformations, tableConfig, jobName);
+ try {
+ JobClient jobClient = execEnv.executeAsync(pipeline);
+ TableSchema.Builder builder = TableSchema.builder();
+ Object[] affectedRowCounts = new Long[operations.size()];
+ for (int i = 0; i < operations.size(); ++i) {
+ // if only one operation, field name is 'affected_rowcount', else is 'affected_rowcount_$i'
+ String fieldName = "affected_rowcount";
Review comment:
in fact, the table qualified name may be long. in addition, we may need to add catalog name and database name to distinguish between tables with the same name.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
}, {
"hash" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 95af2c710bc1453b1930b46dbed83708f08fcc7a Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840)
* 8b565b90f9b2d13629f95c15095229b6c0273c1b UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 4e7e6ed50a36257a3e12cf9166d2d53742a148de Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] hequn8128 commented on a change in pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
hequn8128 commented on a change in pull request #12042:
URL: https://github.com/apache/flink/pull/12042#discussion_r422453409
##########
File path: flink-python/pyflink/table/statement_set.py
##########
@@ -0,0 +1,93 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+from abc import ABCMeta
+
+from pyflink.util.utils import to_j_explain_detail_arr
+
+__all__ = ['StatementSet']
+
+
+class StatementSet(object):
Review comment:
Add this class in the `pyflink/table/__init__.py` file
##########
File path: flink-python/pyflink/table/statement_set.py
##########
@@ -0,0 +1,93 @@
+################################################################################
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+from abc import ABCMeta
+
+from pyflink.util.utils import to_j_explain_detail_arr
+
+__all__ = ['StatementSet']
+
+
+class StatementSet(object):
+ """
+ A StatementSet accepts DML statements or Tables,
+ the planner can optimize all added statements and Tables together
+ and then submit as one job.
+
+ .. note::
+ The added statements and Tables will be cleared
+ when calling the `execute` method.
+ """
+
+ __metaclass__ = ABCMeta
Review comment:
It seems we don't need this. The class is not an abstract class in the Python side.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
}, {
"hash" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880",
"triggerID" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"triggerType" : "PUSH"
}, {
"hash" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=888",
"triggerID" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"triggerType" : "PUSH"
}, {
"hash" : "a3ce5de621cabc2ca7aba0037d1616afa6178b17",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=899",
"triggerID" : "a3ce5de621cabc2ca7aba0037d1616afa6178b17",
"triggerType" : "PUSH"
}, {
"hash" : "b81c9a59867b94c5941f43087f7dd2a359ba07a5",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=904",
"triggerID" : "b81c9a59867b94c5941f43087f7dd2a359ba07a5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b81c9a59867b94c5941f43087f7dd2a359ba07a5 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=904)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625811143
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 4e7e6ed50a36257a3e12cf9166d2d53742a148de (Fri May 08 13:17:40 UTC 2020)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
* **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-16367).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work.
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] KurtYoung commented on a change in pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
KurtYoung commented on a change in pull request #12042:
URL: https://github.com/apache/flink/pull/12042#discussion_r422486671
##########
File path: flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableEnvironmentImpl.java
##########
@@ -894,14 +907,48 @@ private TableResult buildShowResult(String[] objects) {
.build();
}
- private String extractJobName(Operation operation) {
- String tableName;
- if (operation instanceof CatalogSinkModifyOperation) {
- tableName = ((CatalogSinkModifyOperation) operation).getTableIdentifier().toString();
- } else {
- throw new UnsupportedOperationException("Unsupported operation: " + operation);
+ /**
+ * extract sink identifier names from {@link ModifyOperation}s.
+ *
+ * <p>This method will keep the shortest form of an identifier,
+ * which should be unique between all identifiers. e.g.
+ * cat.db.tbl1, cat.db.tbl2, cat.db1.tbl3, cat.db2.tbl3, cat1.db.tbl4, cat2.db.tbl4
+ * the result is
+ * tbl1, tbl2, db1.tbl3, db2.tbl3, cat1.db.tbl4, cat2.db.tbl4
+ */
+ private List<String> extractSinkIdentifierNames(List<ModifyOperation> operations) {
Review comment:
I would prefer to keep it simple for now, just use the full identify is enough.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #12042: [FLINK-16367] [table] Introduce TableEnvironment#createStatementSet api
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #12042:
URL: https://github.com/apache/flink/pull/12042#issuecomment-625871137
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=835",
"triggerID" : "4e7e6ed50a36257a3e12cf9166d2d53742a148de",
"triggerType" : "PUSH"
}, {
"hash" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840",
"triggerID" : "95af2c710bc1453b1930b46dbed83708f08fcc7a",
"triggerType" : "PUSH"
}, {
"hash" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880",
"triggerID" : "8b565b90f9b2d13629f95c15095229b6c0273c1b",
"triggerType" : "PUSH"
}, {
"hash" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "67032e8cb903a95ec384d6e5e55150dc5c4a8144",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 95af2c710bc1453b1930b46dbed83708f08fcc7a Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=840)
* 8b565b90f9b2d13629f95c15095229b6c0273c1b Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=880)
* 67032e8cb903a95ec384d6e5e55150dc5c4a8144 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org