You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "snuyanzin (via GitHub)" <gi...@apache.org> on 2024/01/18 22:49:43 UTC
[PR] [FLINK-34158][table] Migrate WindowAggregateReduceFunctionsRule to java [flink]
snuyanzin opened a new pull request, #24140:
URL: https://github.com/apache/flink/pull/24140
## What is the purpose of the change
The PR migrates `WindowAggregateReduceFunctionsRule` to java
it doesn't touch `WindowAggregateReduceFunctionsRuleTest` to be sure that java version continues passing it
## Verifying this change
This change is already covered by existing tests
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): ( no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no )
- The runtime per-record code paths (performance sensitive): ( no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: ( no )
- The S3 file system connector: ( no )
## Documentation
- Does this pull request introduce a new feature? ( no)
- If yes, how is the feature documented? (not applicable )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34158][table] Migrate WindowAggregateReduceFunctionsRule to java [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #24140:
URL: https://github.com/apache/flink/pull/24140#issuecomment-1899349577
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "43dc43ac4748dfc387fbac363d0ed17ae87adca2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "43dc43ac4748dfc387fbac363d0ed17ae87adca2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 43dc43ac4748dfc387fbac363d0ed17ae87adca2 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34158][table] Migrate WindowAggregateReduceFunctionsRule to java [flink]
Posted by "JingGe (via GitHub)" <gi...@apache.org>.
JingGe commented on code in PR #24140:
URL: https://github.com/apache/flink/pull/24140#discussion_r1468850983
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/rules/logical/WindowAggregateReduceFunctionsRule.java:
##########
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.rules.logical;
+
+import org.apache.flink.table.planner.plan.nodes.calcite.LogicalWindowAggregate;
+
+import org.apache.calcite.plan.Contexts;
+import org.apache.calcite.rel.core.Aggregate;
+import org.apache.calcite.rel.core.AggregateCall;
+import org.apache.calcite.rel.core.RelFactories;
+import org.apache.calcite.rel.logical.LogicalAggregate;
+import org.apache.calcite.rel.rules.AggregateReduceFunctionsRule;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+import org.apache.calcite.tools.RelBuilderFactory;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Rule to convert complex aggregation functions into simpler ones. Have a look at
+ * [[AggregateReduceFunctionsRule]] for details.
+ */
+public class WindowAggregateReduceFunctionsRule extends AggregateReduceFunctionsRule {
+ private static final RelBuilderFactory LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE =
+ RelBuilder.proto(
+ Contexts.of(
+ RelFactories.DEFAULT_STRUCT,
+ RelBuilder.Config.DEFAULT.withPruneInputOfAggregate(false)));
+
+ public static final WindowAggregateReduceFunctionsRule
+ WINDOW_AGGREGATE_REDUCE_FUNCTIONS_INSTANCE =
+ new WindowAggregateReduceFunctionsRule(
+ Config.DEFAULT
+ .withRelBuilderFactory(LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE)
+ .withOperandSupplier(
+ b ->
+ b.operand(LogicalWindowAggregate.class)
+ .anyInputs())
+ .as(Config.class));
+
+ protected WindowAggregateReduceFunctionsRule(Config config) {
+ super(config);
+ }
+
+ protected void newAggregateRel(
Review Comment:
@ Override ?
##########
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/FlinkBatchRuleSets.scala:
##########
@@ -268,7 +268,7 @@ object FlinkBatchRuleSets {
// reduce aggregate functions like AVG, STDDEV_POP etc.
CoreRules.AGGREGATE_REDUCE_FUNCTIONS,
- WindowAggregateReduceFunctionsRule.INSTANCE,
+ WindowAggregateReduceFunctionsRule.WINDOW_AGGREGATE_REDUCE_FUNCTIONS_INSTANCE,
Review Comment:
Would you like share your thoughts about using the new static instance name? The new name contains redundant information that was already offered by the class name.
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/rules/logical/WindowAggregateReduceFunctionsRule.java:
##########
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.rules.logical;
+
+import org.apache.flink.table.planner.plan.nodes.calcite.LogicalWindowAggregate;
+
+import org.apache.calcite.plan.Contexts;
+import org.apache.calcite.rel.core.Aggregate;
+import org.apache.calcite.rel.core.AggregateCall;
+import org.apache.calcite.rel.core.RelFactories;
+import org.apache.calcite.rel.logical.LogicalAggregate;
+import org.apache.calcite.rel.rules.AggregateReduceFunctionsRule;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+import org.apache.calcite.tools.RelBuilderFactory;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Rule to convert complex aggregation functions into simpler ones. Have a look at
+ * [[AggregateReduceFunctionsRule]] for details.
+ */
+public class WindowAggregateReduceFunctionsRule extends AggregateReduceFunctionsRule {
+ private static final RelBuilderFactory LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE =
+ RelBuilder.proto(
+ Contexts.of(
+ RelFactories.DEFAULT_STRUCT,
+ RelBuilder.Config.DEFAULT.withPruneInputOfAggregate(false)));
+
+ public static final WindowAggregateReduceFunctionsRule
Review Comment:
Does it make sense to move the logic to Config class and follow the instance pattern used by other Rule classes?
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/rules/logical/WindowAggregateReduceFunctionsRule.java:
##########
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.rules.logical;
+
+import org.apache.flink.table.planner.plan.nodes.calcite.LogicalWindowAggregate;
+
+import org.apache.calcite.plan.Contexts;
+import org.apache.calcite.rel.core.Aggregate;
+import org.apache.calcite.rel.core.AggregateCall;
+import org.apache.calcite.rel.core.RelFactories;
+import org.apache.calcite.rel.logical.LogicalAggregate;
+import org.apache.calcite.rel.rules.AggregateReduceFunctionsRule;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+import org.apache.calcite.tools.RelBuilderFactory;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Rule to convert complex aggregation functions into simpler ones. Have a look at
+ * [[AggregateReduceFunctionsRule]] for details.
+ */
+public class WindowAggregateReduceFunctionsRule extends AggregateReduceFunctionsRule {
+ private static final RelBuilderFactory LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE =
+ RelBuilder.proto(
+ Contexts.of(
+ RelFactories.DEFAULT_STRUCT,
+ RelBuilder.Config.DEFAULT.withPruneInputOfAggregate(false)));
+
+ public static final WindowAggregateReduceFunctionsRule
+ WINDOW_AGGREGATE_REDUCE_FUNCTIONS_INSTANCE =
+ new WindowAggregateReduceFunctionsRule(
+ Config.DEFAULT
+ .withRelBuilderFactory(LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE)
+ .withOperandSupplier(
+ b ->
+ b.operand(LogicalWindowAggregate.class)
+ .anyInputs())
+ .as(Config.class));
+
+ protected WindowAggregateReduceFunctionsRule(Config config) {
+ super(config);
+ }
+
+ protected void newAggregateRel(
+ RelBuilder relBuilder, Aggregate oldAgg, List<AggregateCall> newCalls) {
+
+ // create a LogicalAggregate with simpler aggregation functions
+ super.newAggregateRel(relBuilder, oldAgg, newCalls);
+ // pop LogicalAggregate from RelBuilder
+ LogicalAggregate newAgg = (LogicalAggregate) relBuilder.build();
+
+ // create a new LogicalWindowAggregate (based on the new LogicalAggregate) and push it on
+ // the
+ // RelBuilder
+ LogicalWindowAggregate oldWindowAgg = (LogicalWindowAggregate) oldAgg;
+ LogicalWindowAggregate newWindowAgg =
+ LogicalWindowAggregate.create(
+ oldWindowAgg.getWindow(), oldWindowAgg.getNamedProperties(), newAgg);
+ relBuilder.push(newWindowAgg);
+ }
+
+ protected void newCalcRel(RelBuilder relBuilder, RelDataType rowType, List<RexNode> exprs) {
Review Comment:
@ Override ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34158][table] Migrate WindowAggregateReduceFunctionsRule to java [flink]
Posted by "RyanSkraba (via GitHub)" <gi...@apache.org>.
RyanSkraba commented on code in PR #24140:
URL: https://github.com/apache/flink/pull/24140#discussion_r1562627239
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/rules/logical/WindowAggregateReduceFunctionsRule.java:
##########
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.rules.logical;
+
+import org.apache.flink.table.planner.plan.nodes.calcite.LogicalWindowAggregate;
+
+import org.apache.calcite.plan.Contexts;
+import org.apache.calcite.rel.core.Aggregate;
+import org.apache.calcite.rel.core.AggregateCall;
+import org.apache.calcite.rel.core.RelFactories;
+import org.apache.calcite.rel.logical.LogicalAggregate;
+import org.apache.calcite.rel.rules.AggregateReduceFunctionsRule;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+import org.apache.calcite.tools.RelBuilderFactory;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Rule to convert complex aggregation functions into simpler ones. Have a look at
+ * {@link AggregateReduceFunctionsRule} for details.
+ */
+public class WindowAggregateReduceFunctionsRule extends AggregateReduceFunctionsRule {
+ private static final RelBuilderFactory LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE =
+ RelBuilder.proto(
+ Contexts.of(
+ RelFactories.DEFAULT_STRUCT,
+ RelBuilder.Config.DEFAULT.withPruneInputOfAggregate(false)));
+
+ public static final WindowAggregateReduceFunctionsRule INSTANCE =
+ new WindowAggregateReduceFunctionsRule(
+ Config.DEFAULT
+ .withRelBuilderFactory(LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE)
+ .withOperandSupplier(
+ b -> b.operand(LogicalWindowAggregate.class).anyInputs())
+ .as(Config.class));
Review Comment:
Lowest priority: Are the descriptions just for debugging? Would it be useful to have a `.withDescription` here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34158][table] Migrate WindowAggregateReduceFunctionsRule to java [flink]
Posted by "JingGe (via GitHub)" <gi...@apache.org>.
JingGe commented on code in PR #24140:
URL: https://github.com/apache/flink/pull/24140#discussion_r1471343469
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/rules/logical/WindowAggregateReduceFunctionsRule.java:
##########
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.rules.logical;
+
+import org.apache.flink.table.planner.plan.nodes.calcite.LogicalWindowAggregate;
+
+import org.apache.calcite.plan.Contexts;
+import org.apache.calcite.rel.core.Aggregate;
+import org.apache.calcite.rel.core.AggregateCall;
+import org.apache.calcite.rel.core.RelFactories;
+import org.apache.calcite.rel.logical.LogicalAggregate;
+import org.apache.calcite.rel.rules.AggregateReduceFunctionsRule;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+import org.apache.calcite.tools.RelBuilderFactory;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Rule to convert complex aggregation functions into simpler ones. Have a look at
+ * [[AggregateReduceFunctionsRule]] for details.
+ */
+public class WindowAggregateReduceFunctionsRule extends AggregateReduceFunctionsRule {
+ private static final RelBuilderFactory LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE =
+ RelBuilder.proto(
+ Contexts.of(
+ RelFactories.DEFAULT_STRUCT,
+ RelBuilder.Config.DEFAULT.withPruneInputOfAggregate(false)));
+
+ public static final WindowAggregateReduceFunctionsRule
Review Comment:
Just like you did in another PR #24139. There are also some other examples e.g. `FlinkJoinReorderRule.Config`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34158][table] Migrate WindowAggregateReduceFunctionsRule to java [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin commented on code in PR #24140:
URL: https://github.com/apache/flink/pull/24140#discussion_r1470432426
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/rules/logical/WindowAggregateReduceFunctionsRule.java:
##########
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.rules.logical;
+
+import org.apache.flink.table.planner.plan.nodes.calcite.LogicalWindowAggregate;
+
+import org.apache.calcite.plan.Contexts;
+import org.apache.calcite.rel.core.Aggregate;
+import org.apache.calcite.rel.core.AggregateCall;
+import org.apache.calcite.rel.core.RelFactories;
+import org.apache.calcite.rel.logical.LogicalAggregate;
+import org.apache.calcite.rel.rules.AggregateReduceFunctionsRule;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+import org.apache.calcite.tools.RelBuilderFactory;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Rule to convert complex aggregation functions into simpler ones. Have a look at
+ * [[AggregateReduceFunctionsRule]] for details.
+ */
+public class WindowAggregateReduceFunctionsRule extends AggregateReduceFunctionsRule {
+ private static final RelBuilderFactory LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE =
+ RelBuilder.proto(
+ Contexts.of(
+ RelFactories.DEFAULT_STRUCT,
+ RelBuilder.Config.DEFAULT.withPruneInputOfAggregate(false)));
+
+ public static final WindowAggregateReduceFunctionsRule
+ WINDOW_AGGREGATE_REDUCE_FUNCTIONS_INSTANCE =
+ new WindowAggregateReduceFunctionsRule(
+ Config.DEFAULT
+ .withRelBuilderFactory(LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE)
+ .withOperandSupplier(
+ b ->
+ b.operand(LogicalWindowAggregate.class)
+ .anyInputs())
+ .as(Config.class));
+
+ protected WindowAggregateReduceFunctionsRule(Config config) {
+ super(config);
+ }
+
+ protected void newAggregateRel(
+ RelBuilder relBuilder, Aggregate oldAgg, List<AggregateCall> newCalls) {
+
+ // create a LogicalAggregate with simpler aggregation functions
+ super.newAggregateRel(relBuilder, oldAgg, newCalls);
+ // pop LogicalAggregate from RelBuilder
+ LogicalAggregate newAgg = (LogicalAggregate) relBuilder.build();
+
+ // create a new LogicalWindowAggregate (based on the new LogicalAggregate) and push it on
+ // the
+ // RelBuilder
+ LogicalWindowAggregate oldWindowAgg = (LogicalWindowAggregate) oldAgg;
+ LogicalWindowAggregate newWindowAgg =
+ LogicalWindowAggregate.create(
+ oldWindowAgg.getWindow(), oldWindowAgg.getNamedProperties(), newAgg);
+ relBuilder.push(newWindowAgg);
+ }
+
+ protected void newCalcRel(RelBuilder relBuilder, RelDataType rowType, List<RexNode> exprs) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34158][table] Migrate WindowAggregateReduceFunctionsRule to java [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin merged PR #24140:
URL: https://github.com/apache/flink/pull/24140
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34158][table] Migrate WindowAggregateReduceFunctionsRule to java [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin commented on code in PR #24140:
URL: https://github.com/apache/flink/pull/24140#discussion_r1519024028
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/rules/logical/WindowAggregateReduceFunctionsRule.java:
##########
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.rules.logical;
+
+import org.apache.flink.table.planner.plan.nodes.calcite.LogicalWindowAggregate;
+
+import org.apache.calcite.plan.Contexts;
+import org.apache.calcite.rel.core.Aggregate;
+import org.apache.calcite.rel.core.AggregateCall;
+import org.apache.calcite.rel.core.RelFactories;
+import org.apache.calcite.rel.logical.LogicalAggregate;
+import org.apache.calcite.rel.rules.AggregateReduceFunctionsRule;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+import org.apache.calcite.tools.RelBuilderFactory;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Rule to convert complex aggregation functions into simpler ones. Have a look at
+ * [[AggregateReduceFunctionsRule]] for details.
+ */
+public class WindowAggregateReduceFunctionsRule extends AggregateReduceFunctionsRule {
+ private static final RelBuilderFactory LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE =
+ RelBuilder.proto(
+ Contexts.of(
+ RelFactories.DEFAULT_STRUCT,
+ RelBuilder.Config.DEFAULT.withPruneInputOfAggregate(false)));
+
+ public static final WindowAggregateReduceFunctionsRule
Review Comment:
No, unfortunately that's not possible or at least I don't know how to do it.
The problem is that normally the rule should extend from `RelRule` where custom config class is a parameter. However here we extend from `AggregateReduceFunctionsRule` where the config is already defained inside `AggregateReduceFunctionsRule` and we can not change it since the class is inside Calcite
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34158][table] Migrate WindowAggregateReduceFunctionsRule to java [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin commented on code in PR #24140:
URL: https://github.com/apache/flink/pull/24140#discussion_r1470432177
##########
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/FlinkBatchRuleSets.scala:
##########
@@ -268,7 +268,7 @@ object FlinkBatchRuleSets {
// reduce aggregate functions like AVG, STDDEV_POP etc.
CoreRules.AGGREGATE_REDUCE_FUNCTIONS,
- WindowAggregateReduceFunctionsRule.INSTANCE,
+ WindowAggregateReduceFunctionsRule.WINDOW_AGGREGATE_REDUCE_FUNCTIONS_INSTANCE,
Review Comment:
yep, makes sense, thanks
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/rules/logical/WindowAggregateReduceFunctionsRule.java:
##########
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.rules.logical;
+
+import org.apache.flink.table.planner.plan.nodes.calcite.LogicalWindowAggregate;
+
+import org.apache.calcite.plan.Contexts;
+import org.apache.calcite.rel.core.Aggregate;
+import org.apache.calcite.rel.core.AggregateCall;
+import org.apache.calcite.rel.core.RelFactories;
+import org.apache.calcite.rel.logical.LogicalAggregate;
+import org.apache.calcite.rel.rules.AggregateReduceFunctionsRule;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+import org.apache.calcite.tools.RelBuilderFactory;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Rule to convert complex aggregation functions into simpler ones. Have a look at
+ * [[AggregateReduceFunctionsRule]] for details.
+ */
+public class WindowAggregateReduceFunctionsRule extends AggregateReduceFunctionsRule {
+ private static final RelBuilderFactory LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE =
+ RelBuilder.proto(
+ Contexts.of(
+ RelFactories.DEFAULT_STRUCT,
+ RelBuilder.Config.DEFAULT.withPruneInputOfAggregate(false)));
+
+ public static final WindowAggregateReduceFunctionsRule
+ WINDOW_AGGREGATE_REDUCE_FUNCTIONS_INSTANCE =
+ new WindowAggregateReduceFunctionsRule(
+ Config.DEFAULT
+ .withRelBuilderFactory(LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE)
+ .withOperandSupplier(
+ b ->
+ b.operand(LogicalWindowAggregate.class)
+ .anyInputs())
+ .as(Config.class));
+
+ protected WindowAggregateReduceFunctionsRule(Config config) {
+ super(config);
+ }
+
+ protected void newAggregateRel(
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34158][table] Migrate WindowAggregateReduceFunctionsRule to java [flink]
Posted by "JingGe (via GitHub)" <gi...@apache.org>.
JingGe commented on code in PR #24140:
URL: https://github.com/apache/flink/pull/24140#discussion_r1468844018
##########
flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/planner/plan/rules/FlinkBatchRuleSets.scala:
##########
@@ -268,7 +268,7 @@ object FlinkBatchRuleSets {
// reduce aggregate functions like AVG, STDDEV_POP etc.
CoreRules.AGGREGATE_REDUCE_FUNCTIONS,
- WindowAggregateReduceFunctionsRule.INSTANCE,
+ WindowAggregateReduceFunctionsRule.WINDOW_AGGREGATE_REDUCE_FUNCTIONS_INSTANCE,
Review Comment:
Would you like to share your thoughts about using the new static instance name? The new name contains redundant information that was already offered by the class name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-34158][table] Migrate WindowAggregateReduceFunctionsRule to java [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin commented on code in PR #24140:
URL: https://github.com/apache/flink/pull/24140#discussion_r1470433205
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/rules/logical/WindowAggregateReduceFunctionsRule.java:
##########
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.plan.rules.logical;
+
+import org.apache.flink.table.planner.plan.nodes.calcite.LogicalWindowAggregate;
+
+import org.apache.calcite.plan.Contexts;
+import org.apache.calcite.rel.core.Aggregate;
+import org.apache.calcite.rel.core.AggregateCall;
+import org.apache.calcite.rel.core.RelFactories;
+import org.apache.calcite.rel.logical.LogicalAggregate;
+import org.apache.calcite.rel.rules.AggregateReduceFunctionsRule;
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.tools.RelBuilder;
+import org.apache.calcite.tools.RelBuilderFactory;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * Rule to convert complex aggregation functions into simpler ones. Have a look at
+ * [[AggregateReduceFunctionsRule]] for details.
+ */
+public class WindowAggregateReduceFunctionsRule extends AggregateReduceFunctionsRule {
+ private static final RelBuilderFactory LOGICAL_BUILDER_WITHOUT_AGG_INPUT_PRUNE =
+ RelBuilder.proto(
+ Contexts.of(
+ RelFactories.DEFAULT_STRUCT,
+ RelBuilder.Config.DEFAULT.withPruneInputOfAggregate(false)));
+
+ public static final WindowAggregateReduceFunctionsRule
Review Comment:
Didn't get what Config class do you mean? The only Config used in this class is coming from Calcite and can not be changed. Or do you mean something like a holder where the logic could be put?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org