You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "jeyhunkarimov (via GitHub)" <gi...@apache.org> on 2024/03/17 17:00:10 UTC
[PR] [FLINK-34633][table] Support unnesting array constants [flink]
jeyhunkarimov opened a new pull request, #24510:
URL: https://github.com/apache/flink/pull/24510
## What is the purpose of the change
Support unnesting array constants
## Brief change log
- Create a new rule `UncollectToTableFunctionScanRule`
- Tests
## Verifying this change
- `batch/sql/UnnestTest`
- `stream/sql/UnnestTest`
- `stream/sql/LogicalUnnestRuleTest`
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34633][table] Support unnesting array constants [flink]
Posted by "LadyForest (via GitHub)" <gi...@apache.org>.
LadyForest commented on PR #24510:
URL: https://github.com/apache/flink/pull/24510#issuecomment-2044375318
I just noticed that FLINK-34162 has been merged. Would you mind rebasing the master and resolving the conflicts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34633][table] Support unnesting array constants [flink]
Posted by "LadyForest (via GitHub)" <gi...@apache.org>.
LadyForest commented on PR #24510:
URL: https://github.com/apache/flink/pull/24510#issuecomment-2031513708
Hi @jeyhunkarimov, thanks for reaching out and sorry for the late response. I'll take a look ASAP.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34633][table] Support unnesting array constants [flink]
Posted by "LadyForest (via GitHub)" <gi...@apache.org>.
LadyForest merged PR #24510:
URL: https://github.com/apache/flink/pull/24510
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34633][table] Support unnesting array constants [flink]
Posted by "jeyhunkarimov (via GitHub)" <gi...@apache.org>.
jeyhunkarimov commented on code in PR #24510:
URL: https://github.com/apache/flink/pull/24510#discussion_r1560114423
##########
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/utils/UnnestConversionUtil.scala:
##########
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.table.planner.plan.utils
+
+import org.apache.flink.table.functions.BuiltInFunctionDefinitions
+import org.apache.flink.table.planner.calcite.FlinkTypeFactory
+import org.apache.flink.table.planner.functions.bridging.BridgingSqlFunction
+import org.apache.flink.table.planner.utils.ShortcutUtils
+import org.apache.flink.table.runtime.functions.table.UnnestRowsFunction
+import org.apache.flink.table.types.logical.utils.LogicalTypeUtils.toRowType
+
+import com.google.common.collect.ImmutableList
+import org.apache.calcite.plan.{RelOptCluster, RelTraitSet}
+import org.apache.calcite.plan.hep.HepRelVertex
+import org.apache.calcite.rel.RelNode
+import org.apache.calcite.rel.core.Uncollect
+import org.apache.calcite.rel.logical._
+
+import java.util.Collections
+
+/** Unnest conversion utils. */
+trait UnnestConversionUtil {
Review Comment:
After rebasing on top of the java-migrated `LogicalUnnestRule ` ([FLINK-34162](https://issues.apache.org/jira/browse/FLINK-34162)) I removed this class
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34633][table] Support unnesting array constants [flink]
Posted by "jeyhunkarimov (via GitHub)" <gi...@apache.org>.
jeyhunkarimov commented on PR #24510:
URL: https://github.com/apache/flink/pull/24510#issuecomment-2048528560
Thanks for your review @LadyForest. I rebased and addressed your comments. Could you please re-check the PR in your available time? Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34633][table] Support unnesting array constants [flink]
Posted by "jeyhunkarimov (via GitHub)" <gi...@apache.org>.
jeyhunkarimov commented on PR #24510:
URL: https://github.com/apache/flink/pull/24510#issuecomment-2018929843
Hi @LadyForest can you please review this PR? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34633][table] Support unnesting array constants [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #24510:
URL: https://github.com/apache/flink/pull/24510#issuecomment-2002539941
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e239874a6d95df04d4c5919b09fc800b82cacc44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e239874a6d95df04d4c5919b09fc800b82cacc44",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e239874a6d95df04d4c5919b09fc800b82cacc44 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34633][table] Support unnesting array constants [flink]
Posted by "LadyForest (via GitHub)" <gi...@apache.org>.
LadyForest commented on code in PR #24510:
URL: https://github.com/apache/flink/pull/24510#discussion_r1568258000
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/rules/logical/UncollectToTableFunctionScanRule.java:
##########
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.rules.logical;
+
+import org.apache.flink.table.functions.BuiltInFunctionDefinitions;
+import org.apache.flink.table.planner.calcite.FlinkTypeFactory;
+import org.apache.flink.table.planner.functions.bridging.BridgingSqlFunction;
+import org.apache.flink.table.planner.utils.ShortcutUtils;
+import org.apache.flink.table.runtime.functions.table.UnnestRowsFunction;
+import org.apache.flink.table.types.logical.LogicalType;
+
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.plan.hep.HepRelVertex;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Uncollect;
+import org.apache.calcite.rel.logical.LogicalFilter;
+import org.apache.calcite.rel.logical.LogicalProject;
+import org.apache.calcite.rel.logical.LogicalTableFunctionScan;
+import org.apache.calcite.rel.logical.LogicalValues;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.sql.SqlFunction;
+import org.immutables.value.Value;
+
+import java.util.Collections;
+
+import static org.apache.flink.table.types.logical.utils.LogicalTypeUtils.toRowType;
+
+/**
+ * Planner rule that converts [[Uncollect]] values to
+ * [[org.apache.calcite.rel.core.TableFunctionScan]].
+ */
Review Comment:
Nit: replace with java doc link
```suggestion
/**
* Planner rule that converts {@link Uncollect} values to {@link
* org.apache.calcite.rel.core.TableFunctionScan}.
*/
```
##########
flink-table/flink-table-planner/src/test/scala/org/apache/flink/table/planner/plan/batch/sql/UnnestTest.scala:
##########
@@ -19,8 +19,34 @@ package org.apache.flink.table.planner.plan.batch.sql
import org.apache.flink.table.planner.plan.common.UnnestTestBase
import org.apache.flink.table.planner.utils.TableTestUtil
+import org.apache.flink.types.Row
+import org.apache.flink.util.CollectionUtil
+
+import org.assertj.core.api.Assertions.assertThat
+import org.junit.jupiter.api.Test
class UnnestTest extends UnnestTestBase(true) {
override def getTableTestUtil: TableTestUtil = batchTestUtil()
+
+ @Test
+ def testUnnestWithValuesBatch(): Unit = {
+ val src = util.tableEnv.sqlQuery("SELECT * FROM UNNEST(ARRAY[1,2,3])")
+ val rows: java.util.List[Row] = CollectionUtil.iteratorToList(src.execute.collect)
+ assertThat(rows.size()).isEqualTo(3)
+ assertThat(rows.get(0).toString).isEqualTo("+I[1]")
+ assertThat(rows.get(1).toString).isEqualTo("+I[2]")
+ assertThat(rows.get(2).toString).isEqualTo("+I[3]")
+ }
+
+ @Test
+ def testUnnestWithValuesBatch2(): Unit = {
+ val src =
+ util.tableEnv.sqlQuery("SELECT * FROM (VALUES('a')) CROSS JOIN UNNEST(ARRAY[1, 2, 3])")
+ val rows: java.util.List[Row] = CollectionUtil.iteratorToList(src.execute.collect)
+ assertThat(rows.size()).isEqualTo(3)
+ assertThat(rows.get(0).toString).isEqualTo("+I[a, 1]")
+ assertThat(rows.get(1).toString).isEqualTo("+I[a, 2]")
+ assertThat(rows.get(2).toString).isEqualTo("+I[a, 3]")
+ }
Review Comment:
`org.apache.flink.table.planner.plan.batch.sql.UnnestTest` and `org.apache.flink.table.planner.plan.stream.sql.UnnestTest` are mainly focusing on the plan test.
Would you mind moving these tests to `org.apache.flink.table.planner.runtime.batch.sql.UnnestITCase` and `org.apache.flink.table.planner.runtime.stream.sql.UnnestITCase` respectively?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34633][table] Support unnesting array constants [flink]
Posted by "LadyForest (via GitHub)" <gi...@apache.org>.
LadyForest commented on PR #24510:
URL: https://github.com/apache/flink/pull/24510#issuecomment-2058380570
> Thanks for your review @LadyForest. I rebased and addressed your comments. Could you please re-check the PR in your available time? Thanks
Apologies for the delayed response, as I was on call last week. Will take a look ASAP.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34633][table] Support unnesting array constants [flink]
Posted by "jeyhunkarimov (via GitHub)" <gi...@apache.org>.
jeyhunkarimov commented on code in PR #24510:
URL: https://github.com/apache/flink/pull/24510#discussion_r1560115307
##########
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/logical/LogicalUnnestRule.scala:
##########
@@ -73,59 +75,13 @@ class LogicalUnnestRule(operand: RelOptRuleOperand, description: String)
val outer = getRel(correlate.getLeft)
val array = getRel(correlate.getRight)
- def convert(relNode: RelNode): RelNode = {
- relNode match {
- case rs: HepRelVertex =>
- convert(getRel(rs))
-
- case f: LogicalProject =>
- f.copy(f.getTraitSet, ImmutableList.of(convert(getRel(f.getInput))))
-
- case f: LogicalFilter =>
- f.copy(f.getTraitSet, ImmutableList.of(convert(getRel(f.getInput))))
-
- case uc: Uncollect =>
- // convert Uncollect into TableFunctionScan
- val cluster = correlate.getCluster
- val typeFactory = ShortcutUtils.unwrapTypeFactory(cluster)
- val relDataType = uc.getInput.getRowType.getFieldList.get(0).getValue
- val logicalType = FlinkTypeFactory.toLogicalType(relDataType)
-
- val sqlFunction =
- BridgingSqlFunction.of(cluster, BuiltInFunctionDefinitions.INTERNAL_UNNEST_ROWS)
-
- val rexCall = cluster.getRexBuilder.makeCall(
- typeFactory.createFieldTypeFromLogicalType(
- toRowType(UnnestRowsFunction.getUnnestedType(logicalType))),
- sqlFunction,
- getRel(uc.getInput).asInstanceOf[LogicalProject].getProjects
- )
-
- new LogicalTableFunctionScan(
- cluster,
- correlate.getTraitSet,
- Collections.emptyList(),
- rexCall,
- null,
- rexCall.getType,
- null)
- }
- }
-
// convert unnest into table function scan
- val tableFunctionScan = convert(array)
+ val tableFunctionScan = convert(array, correlate.getCluster, correlate.getTraitSet)
Review Comment:
Yes, after the rebase ([FLINK-34162](https://issues.apache.org/jira/browse/FLINK-34162)) it is unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34633][table] Support unnesting array constants [flink]
Posted by "LadyForest (via GitHub)" <gi...@apache.org>.
LadyForest commented on code in PR #24510:
URL: https://github.com/apache/flink/pull/24510#discussion_r1566868181
##########
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/logical/UncollectToTableFunctionScanRule.scala:
##########
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.table.planner.plan.rules.logical
+
+import org.apache.flink.table.functions.BuiltInFunctionDefinitions
+import org.apache.flink.table.planner.calcite.FlinkTypeFactory
+import org.apache.flink.table.planner.functions.bridging.BridgingSqlFunction
+import org.apache.flink.table.planner.utils.ShortcutUtils
+import org.apache.flink.table.runtime.functions.table.UnnestRowsFunction
+import org.apache.flink.table.types.logical.utils.LogicalTypeUtils.toRowType
+
+import org.apache.calcite.plan.{RelOptRule, RelOptRuleCall, RelOptRuleOperand}
+import org.apache.calcite.plan.RelOptRule._
+import org.apache.calcite.plan.hep.HepRelVertex
+import org.apache.calcite.rel.RelNode
+import org.apache.calcite.rel.core.Uncollect
+import org.apache.calcite.rel.logical.{LogicalFilter, LogicalProject, LogicalTableFunctionScan, LogicalValues}
+
+import java.util.Collections
+
+/**
+ * Planner rule that converts [[Uncollect]] values to
+ * [[org.apache.calcite.rel.core.TableFunctionScan]].
+ */
+class UncollectToTableFunctionScanRule(operand: RelOptRuleOperand, description: String)
Review Comment:
Hi @jeyhunkarimov, would you mind migrating the rule to Java since scala-free is a trend (please refer to FLINK-34156 for more details)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34633][table] Support unnesting array constants [flink]
Posted by "LadyForest (via GitHub)" <gi...@apache.org>.
LadyForest commented on code in PR #24510:
URL: https://github.com/apache/flink/pull/24510#discussion_r1554809993
##########
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/logical/LogicalUnnestRule.scala:
##########
@@ -20,12 +20,13 @@ package org.apache.flink.table.planner.plan.rules.logical
import org.apache.flink.table.functions.BuiltInFunctionDefinitions
import org.apache.flink.table.planner.calcite.FlinkTypeFactory
import org.apache.flink.table.planner.functions.bridging.BridgingSqlFunction
+import org.apache.flink.table.planner.plan.utils.UnnestConversionUtil
import org.apache.flink.table.planner.utils.ShortcutUtils
import org.apache.flink.table.runtime.functions.table.UnnestRowsFunction
import org.apache.flink.table.types.logical.utils.LogicalTypeUtils.toRowType
import com.google.common.collect.ImmutableList
-import org.apache.calcite.plan.{RelOptRule, RelOptRuleCall, RelOptRuleOperand}
+import org.apache.calcite.plan.{RelOptCluster, RelOptRule, RelOptRuleCall, RelOptRuleOperand, RelTraitSet}
Review Comment:
Please remove the following unused import since a base trait has been extracted
```scalca
import org.apache.flink.table.functions.BuiltInFunctionDefinitions
import org.apache.flink.table.planner.calcite.FlinkTypeFactory
import org.apache.flink.table.planner.functions.bridging.BridgingSqlFunction
import org.apache.flink.table.planner.utils.ShortcutUtils
import org.apache.flink.table.runtime.functions.table.UnnestRowsFunction
import org.apache.flink.table.types.logical.utils.LogicalTypeUtils.toRowType
import org.apache.calcite.plan.hep.HepRelVertex
import org.apache.calcite.rel.RelNode
import java.util.Collections
import org.apache.calcite.plan.{RelOptCluster, RelTraitSet}
```
##########
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/utils/UnnestConversionUtil.scala:
##########
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.table.planner.plan.utils
+
+import org.apache.flink.table.functions.BuiltInFunctionDefinitions
+import org.apache.flink.table.planner.calcite.FlinkTypeFactory
+import org.apache.flink.table.planner.functions.bridging.BridgingSqlFunction
+import org.apache.flink.table.planner.utils.ShortcutUtils
+import org.apache.flink.table.runtime.functions.table.UnnestRowsFunction
+import org.apache.flink.table.types.logical.utils.LogicalTypeUtils.toRowType
+
+import com.google.common.collect.ImmutableList
+import org.apache.calcite.plan.{RelOptCluster, RelTraitSet}
+import org.apache.calcite.plan.hep.HepRelVertex
+import org.apache.calcite.rel.RelNode
+import org.apache.calcite.rel.core.Uncollect
+import org.apache.calcite.rel.logical._
+
+import java.util.Collections
+
+/** Unnest conversion utils. */
+trait UnnestConversionUtil {
Review Comment:
Nit: Naming a trait with "Util" indicates a more procedural or imperative style rather than a pure functional or object-oriented approach. What about `UnnestConverter`?
##########
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/utils/UnnestConversionUtil.scala:
##########
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.table.planner.plan.utils
+
+import org.apache.flink.table.functions.BuiltInFunctionDefinitions
+import org.apache.flink.table.planner.calcite.FlinkTypeFactory
+import org.apache.flink.table.planner.functions.bridging.BridgingSqlFunction
+import org.apache.flink.table.planner.utils.ShortcutUtils
+import org.apache.flink.table.runtime.functions.table.UnnestRowsFunction
+import org.apache.flink.table.types.logical.utils.LogicalTypeUtils.toRowType
+
+import com.google.common.collect.ImmutableList
+import org.apache.calcite.plan.{RelOptCluster, RelTraitSet}
Review Comment:
Please remove unused import
##########
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/logical/LogicalUnnestRule.scala:
##########
@@ -73,59 +75,13 @@ class LogicalUnnestRule(operand: RelOptRuleOperand, description: String)
val outer = getRel(correlate.getLeft)
val array = getRel(correlate.getRight)
- def convert(relNode: RelNode): RelNode = {
- relNode match {
- case rs: HepRelVertex =>
- convert(getRel(rs))
-
- case f: LogicalProject =>
- f.copy(f.getTraitSet, ImmutableList.of(convert(getRel(f.getInput))))
-
- case f: LogicalFilter =>
- f.copy(f.getTraitSet, ImmutableList.of(convert(getRel(f.getInput))))
-
- case uc: Uncollect =>
- // convert Uncollect into TableFunctionScan
- val cluster = correlate.getCluster
- val typeFactory = ShortcutUtils.unwrapTypeFactory(cluster)
- val relDataType = uc.getInput.getRowType.getFieldList.get(0).getValue
- val logicalType = FlinkTypeFactory.toLogicalType(relDataType)
-
- val sqlFunction =
- BridgingSqlFunction.of(cluster, BuiltInFunctionDefinitions.INTERNAL_UNNEST_ROWS)
-
- val rexCall = cluster.getRexBuilder.makeCall(
- typeFactory.createFieldTypeFromLogicalType(
- toRowType(UnnestRowsFunction.getUnnestedType(logicalType))),
- sqlFunction,
- getRel(uc.getInput).asInstanceOf[LogicalProject].getProjects
- )
-
- new LogicalTableFunctionScan(
- cluster,
- correlate.getTraitSet,
- Collections.emptyList(),
- rexCall,
- null,
- rexCall.getType,
- null)
- }
- }
-
// convert unnest into table function scan
- val tableFunctionScan = convert(array)
+ val tableFunctionScan = convert(array, correlate.getCluster, correlate.getTraitSet)
Review Comment:
Nit: I think we can remain the signature of `convert` unchanged. WDYT?
##########
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/logical/UncollectToTableFunctionScanRule.scala:
##########
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.flink.table.planner.plan.rules.logical
+
+import org.apache.flink.table.functions.BuiltInFunctionDefinitions
+import org.apache.flink.table.planner.calcite.FlinkTypeFactory
+import org.apache.flink.table.planner.functions.bridging.BridgingSqlFunction
+import org.apache.flink.table.planner.plan.utils.UnnestConversionUtil
+import org.apache.flink.table.planner.utils.ShortcutUtils
+import org.apache.flink.table.runtime.functions.table.UnnestRowsFunction
+import org.apache.flink.table.types.logical.utils.LogicalTypeUtils.toRowType
+
+import com.google.common.collect.ImmutableList
+import org.apache.calcite.plan.{RelOptRule, RelOptRuleCall, RelOptRuleOperand}
+import org.apache.calcite.plan.RelOptRule._
+import org.apache.calcite.plan.hep.HepRelVertex
+import org.apache.calcite.rel.RelNode
+import org.apache.calcite.rel.core.Uncollect
+import org.apache.calcite.rel.logical._
+
+import java.util.Collections
+
+/** Planner rule that converts Uncollect values to TableFunctionScan. */
Review Comment:
```suggestion
/**
* Planner rule that converts [[Uncollect]] values to
* [[org.apache.calcite.rel.core.TableFunctionScan]].
*/
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34633][table] Support unnesting array constants [flink]
Posted by "jeyhunkarimov (via GitHub)" <gi...@apache.org>.
jeyhunkarimov commented on PR #24510:
URL: https://github.com/apache/flink/pull/24510#issuecomment-2063179382
Thanks a lot for the review @LadyForest
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org