You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2021/03/08 12:43:44 UTC
[GitHub] [flink] lincoln-lil opened a new pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
lincoln-lil opened a new pull request #15117:
URL: https://github.com/apache/flink/pull/15117
## What is the purpose of the change
Fix incorrect simplification for coalesce call on a groupingsets' result
## Brief change log
Fix nullability inference logic in ExpandUtil
## Verifying this change
Added ut & it cases
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): ( no )
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: ( no )
- The serializers: ( no )
- The runtime per-record code paths (performance sensitive): ( no )
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: ( no )
- The S3 file system connector: ( no )
## Documentation
- Does this pull request introduce a new feature? (no)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
}, {
"hash" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431",
"triggerID" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"triggerType" : "PUSH"
}, {
"hash" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14454",
"triggerID" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"triggerType" : "PUSH"
}, {
"hash" : "79d036eb98a74f28c87ada0e2be3b6947f94a965",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15252",
"triggerID" : "79d036eb98a74f28c87ada0e2be3b6947f94a965",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 79d036eb98a74f28c87ada0e2be3b6947f94a965 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15252)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
}, {
"hash" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431",
"triggerID" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"triggerType" : "PUSH"
}, {
"hash" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14454",
"triggerID" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"triggerType" : "PUSH"
}, {
"hash" : "79d036eb98a74f28c87ada0e2be3b6947f94a965",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15252",
"triggerID" : "79d036eb98a74f28c87ada0e2be3b6947f94a965",
"triggerType" : "PUSH"
}, {
"hash" : "b06d70f7ef08aa1ff5278f864571fb52b7848113",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15326",
"triggerID" : "b06d70f7ef08aa1ff5278f864571fb52b7848113",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 79d036eb98a74f28c87ada0e2be3b6947f94a965 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15252)
* b06d70f7ef08aa1ff5278f864571fb52b7848113 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15326)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
}, {
"hash" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431",
"triggerID" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"triggerType" : "PUSH"
}, {
"hash" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14454",
"triggerID" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a5652d53837d6fdb6b5612d16e6cf793256bedc9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431)
* d4a0523eed71bd99b6514303c484242bbab4f753 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14454)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
}, {
"hash" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431",
"triggerID" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6987ed77d824d2104821394134dca1c3f71d9bc2 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309)
* a5652d53837d6fdb6b5612d16e6cf793256bedc9 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] godfreyhe commented on a change in pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
godfreyhe commented on a change in pull request #15117:
URL: https://github.com/apache/flink/pull/15117#discussion_r594003569
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/plan/nodes/calcite/Expand.scala
##########
@@ -59,10 +60,13 @@ abstract class Expand(
if (projects.size() <= 1) {
return litmus.fail("Expand should output more than one rows, otherwise use Project.")
}
- if (projects.exists(_.size != outputRowType.getFieldCount)) {
+ if (null == outputFieldNames) {
+ return litmus.fail("Expand should have none empty output field names.")
+ }
Review comment:
give the default name if `outputFieldNames` is null
and I think it's better we do not give `outputFieldNames` and derive names in `deriveRowType`
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/plan/nodes/calcite/Expand.scala
##########
@@ -79,7 +83,27 @@ abstract class Expand(
litmus.succeed()
}
- override def deriveRowType(): RelDataType = outputRowType
+ override def deriveRowType(): RelDataType = {
+ val fieldNullableMap = mutable.Map[Int, Boolean]()
+ projects.map {
+ project =>
+ project.zipWithIndex.map {
+ f => {
+ val nullable = fieldNullableMap.get(f._2).getOrElse(f._1.getType.isNullable)
+ fieldNullableMap.put(f._2, nullable || f._1.getType.isNullable)
+ }
+ }
+ }
+ val typeList = ListBuffer[RelDataType]()
+ val typeFactory = cluster.getTypeFactory
+ projects.get(0).zipWithIndex.map {
Review comment:
another case we should consider is Expand can be more generic (not only used for distinct agg, but also for other case), after that the fields in the Expand node may be different type, so we should check whether a column's types in all expanded rows are the same or we should derive the common type for each column.
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/plan/utils/ExpandUtil.scala
##########
@@ -19,16 +19,11 @@
package org.apache.flink.table.planner.plan.utils
import org.apache.flink.table.planner.calcite.FlinkRelBuilder
-import org.apache.flink.table.planner.plan.nodes.calcite.{Expand, LogicalExpand}
Review comment:
nit: reformat imports order
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/plan/metadata/FlinkRelMdDistinctRowCount.scala
##########
@@ -223,7 +223,7 @@ class FlinkRelMdDistinctRowCount private extends MetadataHandler[BuiltInMetadata
val groupKeyOfCurrentProject = new JArrayList[Int]()
groupKeySkipExpandId.foreach { key =>
project.get(key) match {
- case literal: RexLiteral if literal.isNull => // do nothing
+ case literal: RexLiteral if literal.isNull || literal.isAlwaysFalse => // do nothing
Review comment:
there are some corner case ?
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/plan/utils/ExpandUtil.scala
##########
@@ -190,15 +172,12 @@ object ExpandUtil {
def createExpandProjects(
rexBuilder: RexBuilder,
inputType: RelDataType,
- outputType: RelDataType,
Review comment:
the java doc should also be updated
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
}, {
"hash" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431",
"triggerID" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"triggerType" : "PUSH"
}, {
"hash" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14454",
"triggerID" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"triggerType" : "PUSH"
}, {
"hash" : "79d036eb98a74f28c87ada0e2be3b6947f94a965",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15252",
"triggerID" : "79d036eb98a74f28c87ada0e2be3b6947f94a965",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d4a0523eed71bd99b6514303c484242bbab4f753 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14454)
* 79d036eb98a74f28c87ada0e2be3b6947f94a965 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15252)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
}, {
"hash" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431",
"triggerID" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"triggerType" : "PUSH"
}, {
"hash" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14454",
"triggerID" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"triggerType" : "PUSH"
}, {
"hash" : "79d036eb98a74f28c87ada0e2be3b6947f94a965",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15252",
"triggerID" : "79d036eb98a74f28c87ada0e2be3b6947f94a965",
"triggerType" : "PUSH"
}, {
"hash" : "b06d70f7ef08aa1ff5278f864571fb52b7848113",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15326",
"triggerID" : "b06d70f7ef08aa1ff5278f864571fb52b7848113",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* b06d70f7ef08aa1ff5278f864571fb52b7848113 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15326)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
}, {
"hash" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431",
"triggerID" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"triggerType" : "PUSH"
}, {
"hash" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14454",
"triggerID" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"triggerType" : "PUSH"
}, {
"hash" : "79d036eb98a74f28c87ada0e2be3b6947f94a965",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15252",
"triggerID" : "79d036eb98a74f28c87ada0e2be3b6947f94a965",
"triggerType" : "PUSH"
}, {
"hash" : "b06d70f7ef08aa1ff5278f864571fb52b7848113",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "b06d70f7ef08aa1ff5278f864571fb52b7848113",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 79d036eb98a74f28c87ada0e2be3b6947f94a965 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=15252)
* b06d70f7ef08aa1ff5278f864571fb52b7848113 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6987ed77d824d2104821394134dca1c3f71d9bc2 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
}, {
"hash" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6987ed77d824d2104821394134dca1c3f71d9bc2 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309)
* a5652d53837d6fdb6b5612d16e6cf793256bedc9 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6987ed77d824d2104821394134dca1c3f71d9bc2 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6987ed77d824d2104821394134dca1c3f71d9bc2 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] lincoln-lil commented on a change in pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
lincoln-lil commented on a change in pull request #15117:
URL: https://github.com/apache/flink/pull/15117#discussion_r594210887
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/plan/metadata/FlinkRelMdDistinctRowCount.scala
##########
@@ -223,7 +223,7 @@ class FlinkRelMdDistinctRowCount private extends MetadataHandler[BuiltInMetadata
val groupKeyOfCurrentProject = new JArrayList[Int]()
groupKeySkipExpandId.foreach { key =>
project.get(key) match {
- case literal: RexLiteral if literal.isNull => // do nothing
+ case literal: RexLiteral if literal.isNull || literal.isAlwaysFalse => // do nothing
Review comment:
It's unnecessary now, just a patch for the boolean type literal which introduced from the previous offline discussion.
I'll remove it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
}, {
"hash" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431",
"triggerID" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"triggerType" : "PUSH"
}, {
"hash" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14454",
"triggerID" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"triggerType" : "PUSH"
}, {
"hash" : "79d036eb98a74f28c87ada0e2be3b6947f94a965",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "79d036eb98a74f28c87ada0e2be3b6947f94a965",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d4a0523eed71bd99b6514303c484242bbab4f753 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14454)
* 79d036eb98a74f28c87ada0e2be3b6947f94a965 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
}, {
"hash" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431",
"triggerID" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a5652d53837d6fdb6b5612d16e6cf793256bedc9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] lincoln-lil commented on a change in pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
lincoln-lil commented on a change in pull request #15117:
URL: https://github.com/apache/flink/pull/15117#discussion_r594205345
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/plan/nodes/calcite/Expand.scala
##########
@@ -59,10 +60,13 @@ abstract class Expand(
if (projects.size() <= 1) {
return litmus.fail("Expand should output more than one rows, otherwise use Project.")
}
- if (projects.exists(_.size != outputRowType.getFieldCount)) {
+ if (null == outputFieldNames) {
+ return litmus.fail("Expand should have none empty output field names.")
+ }
Review comment:
make sense
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792732201
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 6987ed77d824d2104821394134dca1c3f71d9bc2 (Mon Mar 08 12:46:09 UTC 2021)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
* **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-21655).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work.
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] lincoln-lil commented on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
lincoln-lil commented on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-797188040
The test failure is irrelevant module kafka_gelly testing timeout.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
}, {
"hash" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431",
"triggerID" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"triggerType" : "PUSH"
}, {
"hash" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a5652d53837d6fdb6b5612d16e6cf793256bedc9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431)
* d4a0523eed71bd99b6514303c484242bbab4f753 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] godfreyhe closed pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
godfreyhe closed pull request #15117:
URL: https://github.com/apache/flink/pull/15117
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #15117:
URL: https://github.com/apache/flink/pull/15117#issuecomment-792736135
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14309",
"triggerID" : "6987ed77d824d2104821394134dca1c3f71d9bc2",
"triggerType" : "PUSH"
}, {
"hash" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14431",
"triggerID" : "a5652d53837d6fdb6b5612d16e6cf793256bedc9",
"triggerType" : "PUSH"
}, {
"hash" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14454",
"triggerID" : "d4a0523eed71bd99b6514303c484242bbab4f753",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d4a0523eed71bd99b6514303c484242bbab4f753 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=14454)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] lincoln-lil commented on a change in pull request #15117: [FLINK-21655] [table-planner-blink] Fix incorrect simplification for coalesce call on a groupingsets' result
Posted by GitBox <gi...@apache.org>.
lincoln-lil commented on a change in pull request #15117:
URL: https://github.com/apache/flink/pull/15117#discussion_r594206522
##########
File path: flink-table/flink-table-planner-blink/src/main/scala/org/apache/flink/table/planner/plan/nodes/calcite/Expand.scala
##########
@@ -79,7 +83,27 @@ abstract class Expand(
litmus.succeed()
}
- override def deriveRowType(): RelDataType = outputRowType
+ override def deriveRowType(): RelDataType = {
+ val fieldNullableMap = mutable.Map[Int, Boolean]()
+ projects.map {
+ project =>
+ project.zipWithIndex.map {
+ f => {
+ val nullable = fieldNullableMap.get(f._2).getOrElse(f._1.getType.isNullable)
+ fieldNullableMap.put(f._2, nullable || f._1.getType.isNullable)
+ }
+ }
+ }
+ val typeList = ListBuffer[RelDataType]()
+ val typeFactory = cluster.getTypeFactory
+ projects.get(0).zipWithIndex.map {
Review comment:
good point！I'll try to investigate the two ways.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org