You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@asterixdb.apache.org by "Dmitry Lychagin (Code Review)" <do...@asterixdb.incubator.apache.org> on 2019/01/08 00:50:32 UTC
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Dmitry Lychagin has uploaded a new change for review.
https://asterix-gerrit.ics.uci.edu/3107
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
[ASTERIXDB-2505][COMP] Consolidate window operators rule
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
- Add optimizer rules that merge two window operators into one
- Improve existing optimizer rules to handle window operators
- Add tests for distinct aggregate in window operator
Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/LoadRecordFieldsRule.java
R asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_01.sqlpp
R asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_02.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_1.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_2.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_3.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_4.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_5.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_6.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_7.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_01.plan
R asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_02.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_1.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_2.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_3.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_4.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_5.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_6.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_7.plan
D asterixdb/asterix-app/src/test/resources/optimizerts/results/window/window_01.plan
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/misc_01/misc_01.7.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.1.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.2.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.3.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.5.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.6.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.7.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/misc_01/misc_01.7.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.2.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.3.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.4.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.5.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.6.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.7.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppDistinctAggregationSugarVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/VariableCheckAndRewriteVisitor.java
M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/WindowOperator.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismVariableMappingVisitor.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorManipulationUtil.java
A hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ConsolidateWindowOperatorsRule.java
M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineAssignIntoAggregateRule.java
M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
A hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ReuseWindowAggregateRule.java
49 files changed, 1,162 insertions(+), 109 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/07/3107/1
diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
index ece78f8..80469a7 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
@@ -94,6 +94,8 @@
import org.apache.hyracks.algebricks.rewriter.rules.ComplexUnnestToProductRule;
import org.apache.hyracks.algebricks.rewriter.rules.ConsolidateAssignsRule;
import org.apache.hyracks.algebricks.rewriter.rules.ConsolidateSelectsRule;
+import org.apache.hyracks.algebricks.rewriter.rules.ReuseWindowAggregateRule;
+import org.apache.hyracks.algebricks.rewriter.rules.ConsolidateWindowOperatorsRule;
import org.apache.hyracks.algebricks.rewriter.rules.CopyLimitDownRule;
import org.apache.hyracks.algebricks.rewriter.rules.EliminateGroupByEmptyKeyRule;
import org.apache.hyracks.algebricks.rewriter.rules.EnforceOrderByAfterSubplan;
@@ -291,6 +293,10 @@
//PushUnnestDownUnion => RemoveRedundantListifyRule cause these rules are correlated
consolidation.add(new PushUnnestDownThroughUnionRule());
consolidation.add(new RemoveRedundantListifyRule());
+ // Window operator consolidation rules
+ consolidation.add(new ConsolidateWindowOperatorsRule());
+ consolidation.add(new ReuseWindowAggregateRule());
+ consolidation.add(new RemoveRedundantVariablesRule());
return consolidation;
}
diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/LoadRecordFieldsRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/LoadRecordFieldsRule.java
index 32fa744..4d61307 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/LoadRecordFieldsRule.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/LoadRecordFieldsRule.java
@@ -321,10 +321,10 @@
}
} else if (opChild.getOperatorTag() == LogicalOperatorTag.NESTEDTUPLESOURCE) {
NestedTupleSourceOperator nts = (NestedTupleSourceOperator) opChild;
- AbstractLogicalOperator opBelowNestedPlan =
- (AbstractLogicalOperator) nts.getDataSourceReference().getValue().getInputs().get(0).getValue();
+ AbstractLogicalOperator opWithNestedPlan =
+ (AbstractLogicalOperator) nts.getDataSourceReference().getValue();
ILogicalExpression expr1 =
- findFieldExpression(opBelowNestedPlan, recordVar, accessKey, typeEnvironment, resolver);
+ findFieldExpression(opWithNestedPlan, recordVar, accessKey, typeEnvironment, resolver);
if (expr1 != null) {
return expr1;
}
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/window_01.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_01.sqlpp
similarity index 100%
rename from asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/window_01.sqlpp
rename to asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_01.sqlpp
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/window_02.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_02.sqlpp
similarity index 100%
rename from asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/window_02.sqlpp
rename to asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_02.sqlpp
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_1.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_1.sqlpp
new file mode 100644
index 0000000..d53d092
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_1.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of two running aggregates
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ round_half_to_even(percent_rank() over (partition by m order by t), 2) as `percent_rank`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_2.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_2.sqlpp
new file mode 100644
index 0000000..c403726
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_2.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of one running aggregate and one regular aggregate
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ sum(t) over (partition by m order by t) as `sum`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_3.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_3.sqlpp
new file mode 100644
index 0000000..25d559d
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_3.sqlpp
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of one running aggregate
+ * and one regular aggregate with complex argument
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ sum(t + m) over (partition by m order by t) as `sum`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_4.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_4.sqlpp
new file mode 100644
index 0000000..06f5ac3
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_4.sqlpp
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of one running aggregate and
+ * two regular aggregates with complex arguments
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ sum(t + m) over (partition by m order by t) as `sum`,
+ round_half_to_even(avg(t - m) over (partition by m order by t), 2) as `avg`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_5.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_5.sqlpp
new file mode 100644
index 0000000..b6bfd66
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_5.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of two window functions in LET clauses
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t, row_number, ntile
+FROM range(1, 16) t
+LET m = t % 4,
+ row_number = row_number() over (partition by m order by t),
+ ntile = ntile(2) over (partition by m order by t)
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_6.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_6.sqlpp
new file mode 100644
index 0000000..0019ba8
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_6.sqlpp
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of data dependency between two window functions.
+ * No operator consolidation in this case.
+ * Expected Res : SUCCESS (two window operators in the optimized plan)
+ */
+
+SELECT m, t, row_number, ntile
+FROM range(1, 16) t
+LET m = t % 4,
+ row_number = row_number() over (partition by m order by t),
+ ntile = ntile(row_number + 1) over (partition by m order by t)
+ORDER BY m, t;
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_7.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_7.sqlpp
new file mode 100644
index 0000000..f139ab8
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_7.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of ROW frame mode
+ * Expected Res : SUCCESS (two window operators in the optimized plan)
+ */
+
+SELECT m, t,
+ sum(t) over (partition by m order by t rows between unbounded preceding and current row) as `sum`,
+ round_half_to_even(avg(t) over (partition by m order by t rows between unbounded preceding and current row), 2) as `avg`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_01.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_01.plan
new file mode 100644
index 0000000..8191e74
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_01.plan
@@ -0,0 +1,29 @@
+-- DISTRIBUTE_RESULT |UNPARTITIONED|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- STREAM_PROJECT |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- AGGREGATE |UNPARTITIONED|
+ -- RANDOM_MERGE_EXCHANGE |PARTITIONED|
+ -- AGGREGATE |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- ASSIGN |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- ASSIGN |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- ASSIGN |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- WINDOW |PARTITIONED|
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ -- STABLE_SORT [$$t1.c2(ASC), $$p(ASC), $$n(DESC)] |PARTITIONED|
+ -- HASH_PARTITION_EXCHANGE [$$t1.c2, $$p] |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- UNNEST |PARTITIONED|
+ -- UNNEST |PARTITIONED|
+ -- UNNEST |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- ASSIGN |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ -- DATASOURCE_SCAN |PARTITIONED|
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |PARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/window_02.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_02.plan
similarity index 100%
rename from asterixdb/asterix-app/src/test/resources/optimizerts/results/window/window_02.plan
rename to asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_02.plan
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_1.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_1.plan
new file mode 100644
index 0000000..871ea8e
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_1.plan
@@ -0,0 +1,11 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- WINDOW |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_2.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_2.plan
new file mode 100644
index 0000000..a131487
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_2.plan
@@ -0,0 +1,15 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- WINDOW |LOCAL|
+ {
+ -- AGGREGATE |LOCAL|
+ -- NESTED_TUPLE_SOURCE |LOCAL|
+ }
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_3.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_3.plan
new file mode 100644
index 0000000..a131487
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_3.plan
@@ -0,0 +1,15 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- WINDOW |LOCAL|
+ {
+ -- AGGREGATE |LOCAL|
+ -- NESTED_TUPLE_SOURCE |LOCAL|
+ }
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_4.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_4.plan
new file mode 100644
index 0000000..a131487
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_4.plan
@@ -0,0 +1,15 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- WINDOW |LOCAL|
+ {
+ -- AGGREGATE |LOCAL|
+ -- NESTED_TUPLE_SOURCE |LOCAL|
+ }
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_5.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_5.plan
new file mode 100644
index 0000000..871ea8e
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_5.plan
@@ -0,0 +1,11 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- WINDOW |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_6.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_6.plan
new file mode 100644
index 0000000..ab78ecc
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_6.plan
@@ -0,0 +1,12 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- WINDOW |LOCAL|
+ -- WINDOW |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_7.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_7.plan
new file mode 100644
index 0000000..5b3d480
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_7.plan
@@ -0,0 +1,17 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- WINDOW |LOCAL|
+ {
+ -- AGGREGATE |LOCAL|
+ -- NESTED_TUPLE_SOURCE |LOCAL|
+ }
+ -- WINDOW |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/window_01.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/window_01.plan
deleted file mode 100644
index 91e9cfc..0000000
--- a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/window_01.plan
+++ /dev/null
@@ -1,30 +0,0 @@
--- DISTRIBUTE_RESULT |UNPARTITIONED|
- -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
- -- STREAM_PROJECT |UNPARTITIONED|
- -- ASSIGN |UNPARTITIONED|
- -- AGGREGATE |UNPARTITIONED|
- -- RANDOM_MERGE_EXCHANGE |PARTITIONED|
- -- AGGREGATE |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- ASSIGN |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- ASSIGN |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- ASSIGN |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- WINDOW |PARTITIONED|
- -- WINDOW |PARTITIONED|
- -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
- -- STABLE_SORT [$$t1.c2(ASC), $$p(ASC), $$n(DESC)] |PARTITIONED|
- -- HASH_PARTITION_EXCHANGE [$$t1.c2, $$p] |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- UNNEST |PARTITIONED|
- -- UNNEST |PARTITIONED|
- -- UNNEST |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- ASSIGN |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
- -- DATASOURCE_SCAN |PARTITIONED|
- -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
- -- EMPTY_TUPLE_SOURCE |PARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/misc_01/misc_01.7.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/misc_01/misc_01.7.query.sqlpp
new file mode 100644
index 0000000..a04ab08
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/misc_01/misc_01.7.query.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window function distinct aggregate
+ * Expected Res : SUCCESS
+ */
+
+SELECT m, t,
+ sum(t % 4) over (partition by m order by t rows between unbounded preceding and current row) as `sum`,
+ sum(distinct t % 4) over (partition by m order by t rows between unbounded preceding and current row) as `sum_distinct`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.1.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.1.query.sqlpp
new file mode 100644
index 0000000..d53d092
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.1.query.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of two running aggregates
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ round_half_to_even(percent_rank() over (partition by m order by t), 2) as `percent_rank`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.2.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.2.query.sqlpp
new file mode 100644
index 0000000..c403726
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.2.query.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of one running aggregate and one regular aggregate
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ sum(t) over (partition by m order by t) as `sum`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.3.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.3.query.sqlpp
new file mode 100644
index 0000000..25d559d
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.3.query.sqlpp
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of one running aggregate
+ * and one regular aggregate with complex argument
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ sum(t + m) over (partition by m order by t) as `sum`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.4.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.4.query.sqlpp
new file mode 100644
index 0000000..06f5ac3
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.4.query.sqlpp
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of one running aggregate and
+ * two regular aggregates with complex arguments
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ sum(t + m) over (partition by m order by t) as `sum`,
+ round_half_to_even(avg(t - m) over (partition by m order by t), 2) as `avg`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.5.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.5.query.sqlpp
new file mode 100644
index 0000000..b6bfd66
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.5.query.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of two window functions in LET clauses
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t, row_number, ntile
+FROM range(1, 16) t
+LET m = t % 4,
+ row_number = row_number() over (partition by m order by t),
+ ntile = ntile(2) over (partition by m order by t)
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.6.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.6.query.sqlpp
new file mode 100644
index 0000000..0019ba8
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.6.query.sqlpp
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of data dependency between two window functions.
+ * No operator consolidation in this case.
+ * Expected Res : SUCCESS (two window operators in the optimized plan)
+ */
+
+SELECT m, t, row_number, ntile
+FROM range(1, 16) t
+LET m = t % 4,
+ row_number = row_number() over (partition by m order by t),
+ ntile = ntile(row_number + 1) over (partition by m order by t)
+ORDER BY m, t;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.7.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.7.query.sqlpp
new file mode 100644
index 0000000..f139ab8
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.7.query.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of ROW frame mode
+ * Expected Res : SUCCESS (two window operators in the optimized plan)
+ */
+
+SELECT m, t,
+ sum(t) over (partition by m order by t rows between unbounded preceding and current row) as `sum`,
+ round_half_to_even(avg(t) over (partition by m order by t rows between unbounded preceding and current row), 2) as `avg`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/misc_01/misc_01.7.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/misc_01/misc_01.7.adm
new file mode 100644
index 0000000..9deb329
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/misc_01/misc_01.7.adm
@@ -0,0 +1,20 @@
+{ "m": 0, "t": 4, "sum": 0, "sum_distinct": 0 }
+{ "m": 0, "t": 8, "sum": 0, "sum_distinct": 0 }
+{ "m": 0, "t": 12, "sum": 0, "sum_distinct": 0 }
+{ "m": 0, "t": 16, "sum": 0, "sum_distinct": 0 }
+{ "m": 0, "t": 20, "sum": 0, "sum_distinct": 0 }
+{ "m": 1, "t": 1, "sum": 1, "sum_distinct": 1 }
+{ "m": 1, "t": 5, "sum": 2, "sum_distinct": 1 }
+{ "m": 1, "t": 9, "sum": 3, "sum_distinct": 1 }
+{ "m": 1, "t": 13, "sum": 4, "sum_distinct": 1 }
+{ "m": 1, "t": 17, "sum": 5, "sum_distinct": 1 }
+{ "m": 2, "t": 2, "sum": 2, "sum_distinct": 2 }
+{ "m": 2, "t": 6, "sum": 4, "sum_distinct": 2 }
+{ "m": 2, "t": 10, "sum": 6, "sum_distinct": 2 }
+{ "m": 2, "t": 14, "sum": 8, "sum_distinct": 2 }
+{ "m": 2, "t": 18, "sum": 10, "sum_distinct": 2 }
+{ "m": 3, "t": 3, "sum": 3, "sum_distinct": 3 }
+{ "m": 3, "t": 7, "sum": 6, "sum_distinct": 3 }
+{ "m": 3, "t": 11, "sum": 9, "sum_distinct": 3 }
+{ "m": 3, "t": 15, "sum": 12, "sum_distinct": 3 }
+{ "m": 3, "t": 19, "sum": 15, "sum_distinct": 3 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.1.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.1.adm
new file mode 100644
index 0000000..75263ed
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.1.adm
@@ -0,0 +1,20 @@
+{ "m": 0, "t": 4, "row_nubmer": 1, "percent_rank": 0.0 }
+{ "m": 0, "t": 8, "row_nubmer": 2, "percent_rank": 0.25 }
+{ "m": 0, "t": 12, "row_nubmer": 3, "percent_rank": 0.5 }
+{ "m": 0, "t": 16, "row_nubmer": 4, "percent_rank": 0.75 }
+{ "m": 0, "t": 20, "row_nubmer": 5, "percent_rank": 1.0 }
+{ "m": 1, "t": 1, "row_nubmer": 1, "percent_rank": 0.0 }
+{ "m": 1, "t": 5, "row_nubmer": 2, "percent_rank": 0.25 }
+{ "m": 1, "t": 9, "row_nubmer": 3, "percent_rank": 0.5 }
+{ "m": 1, "t": 13, "row_nubmer": 4, "percent_rank": 0.75 }
+{ "m": 1, "t": 17, "row_nubmer": 5, "percent_rank": 1.0 }
+{ "m": 2, "t": 2, "row_nubmer": 1, "percent_rank": 0.0 }
+{ "m": 2, "t": 6, "row_nubmer": 2, "percent_rank": 0.25 }
+{ "m": 2, "t": 10, "row_nubmer": 3, "percent_rank": 0.5 }
+{ "m": 2, "t": 14, "row_nubmer": 4, "percent_rank": 0.75 }
+{ "m": 2, "t": 18, "row_nubmer": 5, "percent_rank": 1.0 }
+{ "m": 3, "t": 3, "row_nubmer": 1, "percent_rank": 0.0 }
+{ "m": 3, "t": 7, "row_nubmer": 2, "percent_rank": 0.25 }
+{ "m": 3, "t": 11, "row_nubmer": 3, "percent_rank": 0.5 }
+{ "m": 3, "t": 15, "row_nubmer": 4, "percent_rank": 0.75 }
+{ "m": 3, "t": 19, "row_nubmer": 5, "percent_rank": 1.0 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.2.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.2.adm
new file mode 100644
index 0000000..a08ac96
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.2.adm
@@ -0,0 +1,20 @@
+{ "m": 0, "t": 4, "row_nubmer": 1, "sum": 4 }
+{ "m": 0, "t": 8, "row_nubmer": 2, "sum": 12 }
+{ "m": 0, "t": 12, "row_nubmer": 3, "sum": 24 }
+{ "m": 0, "t": 16, "row_nubmer": 4, "sum": 40 }
+{ "m": 0, "t": 20, "row_nubmer": 5, "sum": 60 }
+{ "m": 1, "t": 1, "row_nubmer": 1, "sum": 1 }
+{ "m": 1, "t": 5, "row_nubmer": 2, "sum": 6 }
+{ "m": 1, "t": 9, "row_nubmer": 3, "sum": 15 }
+{ "m": 1, "t": 13, "row_nubmer": 4, "sum": 28 }
+{ "m": 1, "t": 17, "row_nubmer": 5, "sum": 45 }
+{ "m": 2, "t": 2, "row_nubmer": 1, "sum": 2 }
+{ "m": 2, "t": 6, "row_nubmer": 2, "sum": 8 }
+{ "m": 2, "t": 10, "row_nubmer": 3, "sum": 18 }
+{ "m": 2, "t": 14, "row_nubmer": 4, "sum": 32 }
+{ "m": 2, "t": 18, "row_nubmer": 5, "sum": 50 }
+{ "m": 3, "t": 3, "row_nubmer": 1, "sum": 3 }
+{ "m": 3, "t": 7, "row_nubmer": 2, "sum": 10 }
+{ "m": 3, "t": 11, "row_nubmer": 3, "sum": 21 }
+{ "m": 3, "t": 15, "row_nubmer": 4, "sum": 36 }
+{ "m": 3, "t": 19, "row_nubmer": 5, "sum": 55 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.3.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.3.adm
new file mode 100644
index 0000000..f356720
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.3.adm
@@ -0,0 +1,20 @@
+{ "m": 0, "t": 4, "row_nubmer": 1, "sum": 4 }
+{ "m": 0, "t": 8, "row_nubmer": 2, "sum": 12 }
+{ "m": 0, "t": 12, "row_nubmer": 3, "sum": 24 }
+{ "m": 0, "t": 16, "row_nubmer": 4, "sum": 40 }
+{ "m": 0, "t": 20, "row_nubmer": 5, "sum": 60 }
+{ "m": 1, "t": 1, "row_nubmer": 1, "sum": 2 }
+{ "m": 1, "t": 5, "row_nubmer": 2, "sum": 8 }
+{ "m": 1, "t": 9, "row_nubmer": 3, "sum": 18 }
+{ "m": 1, "t": 13, "row_nubmer": 4, "sum": 32 }
+{ "m": 1, "t": 17, "row_nubmer": 5, "sum": 50 }
+{ "m": 2, "t": 2, "row_nubmer": 1, "sum": 4 }
+{ "m": 2, "t": 6, "row_nubmer": 2, "sum": 12 }
+{ "m": 2, "t": 10, "row_nubmer": 3, "sum": 24 }
+{ "m": 2, "t": 14, "row_nubmer": 4, "sum": 40 }
+{ "m": 2, "t": 18, "row_nubmer": 5, "sum": 60 }
+{ "m": 3, "t": 3, "row_nubmer": 1, "sum": 6 }
+{ "m": 3, "t": 7, "row_nubmer": 2, "sum": 16 }
+{ "m": 3, "t": 11, "row_nubmer": 3, "sum": 30 }
+{ "m": 3, "t": 15, "row_nubmer": 4, "sum": 48 }
+{ "m": 3, "t": 19, "row_nubmer": 5, "sum": 70 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.4.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.4.adm
new file mode 100644
index 0000000..e14d3a2
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.4.adm
@@ -0,0 +1,20 @@
+{ "m": 0, "t": 4, "row_nubmer": 1, "sum": 4, "avg": 4.0 }
+{ "m": 0, "t": 8, "row_nubmer": 2, "sum": 12, "avg": 6.0 }
+{ "m": 0, "t": 12, "row_nubmer": 3, "sum": 24, "avg": 8.0 }
+{ "m": 0, "t": 16, "row_nubmer": 4, "sum": 40, "avg": 10.0 }
+{ "m": 0, "t": 20, "row_nubmer": 5, "sum": 60, "avg": 12.0 }
+{ "m": 1, "t": 1, "row_nubmer": 1, "sum": 2, "avg": 0.0 }
+{ "m": 1, "t": 5, "row_nubmer": 2, "sum": 8, "avg": 2.0 }
+{ "m": 1, "t": 9, "row_nubmer": 3, "sum": 18, "avg": 4.0 }
+{ "m": 1, "t": 13, "row_nubmer": 4, "sum": 32, "avg": 6.0 }
+{ "m": 1, "t": 17, "row_nubmer": 5, "sum": 50, "avg": 8.0 }
+{ "m": 2, "t": 2, "row_nubmer": 1, "sum": 4, "avg": 0.0 }
+{ "m": 2, "t": 6, "row_nubmer": 2, "sum": 12, "avg": 2.0 }
+{ "m": 2, "t": 10, "row_nubmer": 3, "sum": 24, "avg": 4.0 }
+{ "m": 2, "t": 14, "row_nubmer": 4, "sum": 40, "avg": 6.0 }
+{ "m": 2, "t": 18, "row_nubmer": 5, "sum": 60, "avg": 8.0 }
+{ "m": 3, "t": 3, "row_nubmer": 1, "sum": 6, "avg": 0.0 }
+{ "m": 3, "t": 7, "row_nubmer": 2, "sum": 16, "avg": 2.0 }
+{ "m": 3, "t": 11, "row_nubmer": 3, "sum": 30, "avg": 4.0 }
+{ "m": 3, "t": 15, "row_nubmer": 4, "sum": 48, "avg": 6.0 }
+{ "m": 3, "t": 19, "row_nubmer": 5, "sum": 70, "avg": 8.0 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.5.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.5.adm
new file mode 100644
index 0000000..fa8f4af
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.5.adm
@@ -0,0 +1,16 @@
+{ "m": 0, "t": 4, "row_number": 1, "ntile": 1 }
+{ "m": 0, "t": 8, "row_number": 2, "ntile": 1 }
+{ "m": 0, "t": 12, "row_number": 3, "ntile": 2 }
+{ "m": 0, "t": 16, "row_number": 4, "ntile": 2 }
+{ "m": 1, "t": 1, "row_number": 1, "ntile": 1 }
+{ "m": 1, "t": 5, "row_number": 2, "ntile": 1 }
+{ "m": 1, "t": 9, "row_number": 3, "ntile": 2 }
+{ "m": 1, "t": 13, "row_number": 4, "ntile": 2 }
+{ "m": 2, "t": 2, "row_number": 1, "ntile": 1 }
+{ "m": 2, "t": 6, "row_number": 2, "ntile": 1 }
+{ "m": 2, "t": 10, "row_number": 3, "ntile": 2 }
+{ "m": 2, "t": 14, "row_number": 4, "ntile": 2 }
+{ "m": 3, "t": 3, "row_number": 1, "ntile": 1 }
+{ "m": 3, "t": 7, "row_number": 2, "ntile": 1 }
+{ "m": 3, "t": 11, "row_number": 3, "ntile": 2 }
+{ "m": 3, "t": 15, "row_number": 4, "ntile": 2 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.6.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.6.adm
new file mode 100644
index 0000000..fa8f4af
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.6.adm
@@ -0,0 +1,16 @@
+{ "m": 0, "t": 4, "row_number": 1, "ntile": 1 }
+{ "m": 0, "t": 8, "row_number": 2, "ntile": 1 }
+{ "m": 0, "t": 12, "row_number": 3, "ntile": 2 }
+{ "m": 0, "t": 16, "row_number": 4, "ntile": 2 }
+{ "m": 1, "t": 1, "row_number": 1, "ntile": 1 }
+{ "m": 1, "t": 5, "row_number": 2, "ntile": 1 }
+{ "m": 1, "t": 9, "row_number": 3, "ntile": 2 }
+{ "m": 1, "t": 13, "row_number": 4, "ntile": 2 }
+{ "m": 2, "t": 2, "row_number": 1, "ntile": 1 }
+{ "m": 2, "t": 6, "row_number": 2, "ntile": 1 }
+{ "m": 2, "t": 10, "row_number": 3, "ntile": 2 }
+{ "m": 2, "t": 14, "row_number": 4, "ntile": 2 }
+{ "m": 3, "t": 3, "row_number": 1, "ntile": 1 }
+{ "m": 3, "t": 7, "row_number": 2, "ntile": 1 }
+{ "m": 3, "t": 11, "row_number": 3, "ntile": 2 }
+{ "m": 3, "t": 15, "row_number": 4, "ntile": 2 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.7.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.7.adm
new file mode 100644
index 0000000..30ed6e2
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.7.adm
@@ -0,0 +1,20 @@
+{ "m": 0, "t": 4, "sum": 4, "avg": 4.0 }
+{ "m": 0, "t": 8, "sum": 12, "avg": 6.0 }
+{ "m": 0, "t": 12, "sum": 24, "avg": 8.0 }
+{ "m": 0, "t": 16, "sum": 40, "avg": 10.0 }
+{ "m": 0, "t": 20, "sum": 60, "avg": 12.0 }
+{ "m": 1, "t": 1, "sum": 1, "avg": 1.0 }
+{ "m": 1, "t": 5, "sum": 6, "avg": 3.0 }
+{ "m": 1, "t": 9, "sum": 15, "avg": 5.0 }
+{ "m": 1, "t": 13, "sum": 28, "avg": 7.0 }
+{ "m": 1, "t": 17, "sum": 45, "avg": 9.0 }
+{ "m": 2, "t": 2, "sum": 2, "avg": 2.0 }
+{ "m": 2, "t": 6, "sum": 8, "avg": 4.0 }
+{ "m": 2, "t": 10, "sum": 18, "avg": 6.0 }
+{ "m": 2, "t": 14, "sum": 32, "avg": 8.0 }
+{ "m": 2, "t": 18, "sum": 50, "avg": 10.0 }
+{ "m": 3, "t": 3, "sum": 3, "avg": 3.0 }
+{ "m": 3, "t": 7, "sum": 10, "avg": 5.0 }
+{ "m": 3, "t": 11, "sum": 21, "avg": 7.0 }
+{ "m": 3, "t": 15, "sum": 36, "avg": 9.0 }
+{ "m": 3, "t": 19, "sum": 55, "avg": 11.0 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
index 7bd1eaf..3a857c1 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
@@ -9260,6 +9260,11 @@
<output-dir compare="Text">row_number_01</output-dir>
</compilation-unit>
</test-case>
+ <test-case FilePath="window">
+ <compilation-unit name="win_opt_01">
+ <output-dir compare="Text">win_opt_01</output-dir>
+ </compilation-unit>
+ </test-case>
</test-group>
<test-group name="writers">
<test-case FilePath="writers">
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppDistinctAggregationSugarVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppDistinctAggregationSugarVisitor.java
index 80844ab..efcd7e0 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppDistinctAggregationSugarVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppDistinctAggregationSugarVisitor.java
@@ -38,6 +38,7 @@
import org.apache.asterix.lang.sqlpp.clause.SelectElement;
import org.apache.asterix.lang.sqlpp.clause.SelectSetOperation;
import org.apache.asterix.lang.sqlpp.expression.SelectExpression;
+import org.apache.asterix.lang.sqlpp.expression.WindowExpression;
import org.apache.asterix.lang.sqlpp.struct.SetOperationInput;
import org.apache.asterix.lang.sqlpp.visitor.base.AbstractSqlppSimpleExpressionVisitor;
import org.apache.asterix.om.functions.BuiltinFunctions;
@@ -59,25 +60,45 @@
@Override
public Expression visit(CallExpr callExpr, ILangExpression arg) throws CompilationException {
- FunctionSignature signature = callExpr.getFunctionSignature();
- IFunctionInfo finfo = FunctionUtil.getFunctionInfo(signature);
- FunctionIdentifier aggFn =
- finfo != null ? BuiltinFunctions.getAggregateFunction(finfo.getFunctionIdentifier()) : null;
- FunctionIdentifier newAggFn = aggFn != null ? BuiltinFunctions.getAggregateFunctionForDistinct(aggFn) : null;
+ FunctionIdentifier newAggFn = getAggregateFunctionForDistinct(callExpr.getFunctionSignature());
if (newAggFn == null) {
return super.visit(callExpr, arg);
}
- List<Expression> exprList = callExpr.getExprList();
- List<Expression> newExprList = new ArrayList<>(exprList.size());
- for (Expression expr : exprList) {
- Expression newExpr = rewriteArgument(expr);
- newExprList.add(newExpr.accept(this, arg));
- }
+ List<Expression> newExprList = rewriteArgumentList(callExpr.getExprList(), arg);
callExpr.setFunctionSignature(new FunctionSignature(newAggFn));
callExpr.setExprList(newExprList);
return callExpr;
}
+ @Override
+ public Expression visit(WindowExpression winExpr, ILangExpression arg) throws CompilationException {
+ FunctionIdentifier newAggFn = getAggregateFunctionForDistinct(winExpr.getFunctionSignature());
+ if (newAggFn == null) {
+ return super.visit(winExpr, arg);
+ }
+ List<Expression> newExprList = rewriteArgumentList(winExpr.getExprList(), arg);
+ winExpr.setFunctionSignature(new FunctionSignature(newAggFn));
+ winExpr.setExprList(newExprList);
+ return winExpr;
+ }
+
+ private FunctionIdentifier getAggregateFunctionForDistinct(FunctionSignature signature) {
+ IFunctionInfo finfo = FunctionUtil.getFunctionInfo(signature);
+ FunctionIdentifier aggFn =
+ finfo != null ? BuiltinFunctions.getAggregateFunction(finfo.getFunctionIdentifier()) : null;
+ return aggFn != null ? BuiltinFunctions.getAggregateFunctionForDistinct(aggFn) : null;
+ }
+
+ private List<Expression> rewriteArgumentList(List<Expression> exprList, ILangExpression arg)
+ throws CompilationException {
+ List<Expression> result = new ArrayList<>(exprList.size());
+ for (Expression expr : exprList) {
+ Expression newExpr = rewriteArgument(expr);
+ result.add(newExpr.accept(this, arg));
+ }
+ return result;
+ }
+
/**
* rewrites {@code expr -> FROM expr AS i SELECT DISTINCT VALUE i}
*/
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/VariableCheckAndRewriteVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/VariableCheckAndRewriteVisitor.java
index b48e024..7ffebc2 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/VariableCheckAndRewriteVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/VariableCheckAndRewriteVisitor.java
@@ -204,6 +204,7 @@
return path.length == 2 && metadataProvider.findDataset(path[0], path[1]) != null;
}
+ @Override
public Expression visit(CallExpr callExpr, ILangExpression arg) throws CompilationException {
// skip variables inside SQL-92 aggregates (they will be resolved by SqlppGroupByAggregationSugarVisitor)
if (FunctionMapUtil.isSql92AggregateFunction(callExpr.getFunctionSignature())) {
diff --git a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java
index 08f5139..54c8330 100644
--- a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java
+++ b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java
@@ -1790,9 +1790,9 @@
addFunction(SCALAR_SQL_COUNT_DISTINCT, AInt64TypeComputer.INSTANCE, true);
addFunction(SUM_DISTINCT, NumericSumAggTypeComputer.INSTANCE, true);
- addFunction(SCALAR_SUM_DISTINCT, NumericSumAggTypeComputer.INSTANCE, true);
+ addFunction(SCALAR_SUM_DISTINCT, ScalarVersionOfAggregateResultType.INSTANCE, true);
addFunction(SQL_SUM_DISTINCT, NumericSumAggTypeComputer.INSTANCE, true);
- addFunction(SCALAR_SQL_SUM_DISTINCT, NumericSumAggTypeComputer.INSTANCE, true);
+ addFunction(SCALAR_SQL_SUM_DISTINCT, ScalarVersionOfAggregateResultType.INSTANCE, true);
addFunction(AVG_DISTINCT, NullableDoubleTypeComputer.INSTANCE, true);
addFunction(SCALAR_AVG_DISTINCT, NullableDoubleTypeComputer.INSTANCE, true);
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/WindowOperator.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/WindowOperator.java
index e09b471..8f563b3 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/WindowOperator.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/WindowOperator.java
@@ -68,7 +68,7 @@
private final List<Mutable<ILogicalExpression>> frameExcludeExpressions;
- private final int frameExcludeNegationStartIdx;
+ private int frameExcludeNegationStartIdx;
private final Mutable<ILogicalExpression> frameOffset;
@@ -175,6 +175,10 @@
return frameExcludeNegationStartIdx;
}
+ public void setFrameExcludeNegationStartIdx(int value) {
+ this.frameExcludeNegationStartIdx = value;
+ }
+
public Mutable<ILogicalExpression> getFrameOffset() {
return frameOffset;
}
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java
index 764fe49..a39af41 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java
@@ -624,32 +624,10 @@
return false;
}
WindowOperator windowOpArg = (WindowOperator) copyAndSubstituteVar(op, arg);
- if (!VariableUtilities.varListEqualUnordered(op.getPartitionExpressions(),
- windowOpArg.getPartitionExpressions())) {
+ if (!compareWindowPartitionSpec(op, windowOpArg)) {
return false;
}
- if (!compareIOrderAndExpressions(op.getOrderExpressions(), windowOpArg.getOrderExpressions())) {
- return false;
- }
- if (!compareIOrderAndExpressions(op.getFrameValueExpressions(), windowOpArg.getFrameValueExpressions())) {
- return false;
- }
- if (!compareExpressions(op.getFrameStartExpressions(), windowOpArg.getFrameStartExpressions())) {
- return false;
- }
- if (!compareExpressions(op.getFrameEndExpressions(), windowOpArg.getFrameEndExpressions())) {
- return false;
- }
- if (!compareExpressions(op.getFrameExcludeExpressions(), windowOpArg.getFrameExcludeExpressions())) {
- return false;
- }
- if (op.getFrameExcludeNegationStartIdx() != windowOpArg.getFrameExcludeNegationStartIdx()) {
- return false;
- }
- if (!Objects.equals(op.getFrameOffset().getValue(), windowOpArg.getFrameOffset().getValue())) {
- return false;
- }
- if (op.getFrameMaxObjects() != windowOpArg.getFrameMaxObjects()) {
+ if (!compareWindowFrameSpec(op, windowOpArg)) {
return false;
}
if (!VariableUtilities.varListEqualUnordered(getPairList(op.getVariables(), op.getExpressions()),
@@ -662,36 +640,52 @@
return isomorphic;
}
- private Boolean compareExpressions(List<Mutable<ILogicalExpression>> opExprs,
+ public static boolean compareWindowPartitionSpec(WindowOperator winOp1, WindowOperator winOp2) {
+ return VariableUtilities.varListEqualUnordered(winOp1.getPartitionExpressions(),
+ winOp2.getPartitionExpressions())
+ && compareIOrderAndExpressions(winOp1.getOrderExpressions(), winOp2.getOrderExpressions());
+ }
+
+ public static boolean compareWindowFrameSpec(WindowOperator winOp1, WindowOperator winOp2) {
+ return compareIOrderAndExpressions(winOp1.getFrameValueExpressions(), winOp2.getFrameValueExpressions())
+ && compareExpressions(winOp1.getFrameStartExpressions(), winOp2.getFrameStartExpressions())
+ && compareExpressions(winOp1.getFrameEndExpressions(), winOp2.getFrameEndExpressions())
+ && compareExpressions(winOp1.getFrameExcludeExpressions(), winOp2.getFrameExcludeExpressions())
+ && winOp1.getFrameExcludeNegationStartIdx() == winOp2.getFrameExcludeNegationStartIdx()
+ && Objects.equals(winOp1.getFrameOffset().getValue(), winOp2.getFrameOffset().getValue())
+ && winOp1.getFrameMaxObjects() == winOp2.getFrameMaxObjects();
+ }
+
+ private static boolean compareExpressions(List<Mutable<ILogicalExpression>> opExprs,
List<Mutable<ILogicalExpression>> argExprs) {
if (opExprs.size() != argExprs.size()) {
- return Boolean.FALSE;
+ return false;
}
for (int i = 0; i < opExprs.size(); i++) {
boolean isomorphic = opExprs.get(i).getValue().equals(argExprs.get(i).getValue());
if (!isomorphic) {
- return Boolean.FALSE;
+ return false;
}
}
- return Boolean.TRUE;
+ return true;
}
- private Boolean compareIOrderAndExpressions(List<Pair<IOrder, Mutable<ILogicalExpression>>> opOrderExprs,
+ private static boolean compareIOrderAndExpressions(List<Pair<IOrder, Mutable<ILogicalExpression>>> opOrderExprs,
List<Pair<IOrder, Mutable<ILogicalExpression>>> argOrderExprs) {
if (opOrderExprs.size() != argOrderExprs.size()) {
- return Boolean.FALSE;
+ return false;
}
for (int i = 0; i < opOrderExprs.size(); i++) {
boolean isomorphic = opOrderExprs.get(i).first.equals(argOrderExprs.get(i).first);
if (!isomorphic) {
- return Boolean.FALSE;
+ return false;
}
isomorphic = opOrderExprs.get(i).second.getValue().equals(argOrderExprs.get(i).second.getValue());
if (!isomorphic) {
- return Boolean.FALSE;
+ return false;
}
}
- return Boolean.TRUE;
+ return true;
}
private boolean compareSubplans(List<ILogicalPlan> plans, List<ILogicalPlan> plansArg) throws AlgebricksException {
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismVariableMappingVisitor.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismVariableMappingVisitor.java
index 5591053..eca2508 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismVariableMappingVisitor.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismVariableMappingVisitor.java
@@ -99,7 +99,7 @@
@Override
public Void visitWindowOperator(WindowOperator op, ILogicalOperator arg) throws AlgebricksException {
mapChildren(op, arg);
- mapVariablesForAbstractAssign(op, arg);
+ mapVariablesForWindow(op, arg);
mapVariablesInNestedPlans(op, arg);
return null;
}
@@ -359,7 +359,7 @@
rightOp.getExpressions());
}
- private void mapVariablesForGroupBy(ILogicalOperator left, ILogicalOperator right) throws AlgebricksException {
+ private void mapVariablesForGroupBy(ILogicalOperator left, ILogicalOperator right) {
if (left.getOperatorTag() != right.getOperatorTag()) {
return;
}
@@ -371,6 +371,16 @@
leftPairs = leftOp.getDecorList();
rightPairs = rightOp.getDecorList();
mapVarExprPairList(leftPairs, rightPairs);
+ }
+
+ private void mapVariablesForWindow(ILogicalOperator left, ILogicalOperator right) {
+ if (left.getOperatorTag() != right.getOperatorTag()) {
+ return;
+ }
+ WindowOperator leftOp = (WindowOperator) left;
+ WindowOperator rightOp = (WindowOperator) right;
+ mapVariablesForAbstractAssign(leftOp.getVariables(), leftOp.getExpressions(), rightOp.getVariables(),
+ rightOp.getExpressions());
}
private void mapVarExprPairList(List<Pair<LogicalVariable, Mutable<ILogicalExpression>>> leftPairs,
@@ -420,6 +430,9 @@
private void mapVariablesInNestedPlans(AbstractOperatorWithNestedPlans op, ILogicalOperator arg)
throws AlgebricksException {
+ if (op.getOperatorTag() != arg.getOperatorTag()) {
+ return;
+ }
AbstractOperatorWithNestedPlans argOp = (AbstractOperatorWithNestedPlans) arg;
List<ILogicalPlan> plans = op.getNestedPlans();
List<ILogicalPlan> plansArg = argOp.getNestedPlans();
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorManipulationUtil.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorManipulationUtil.java
index 4d05a1b..967ad6f 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorManipulationUtil.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorManipulationUtil.java
@@ -36,6 +36,7 @@
import org.apache.hyracks.algebricks.core.algebra.base.IOptimizationContext;
import org.apache.hyracks.algebricks.core.algebra.base.LogicalOperatorTag;
import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
+import org.apache.hyracks.algebricks.core.algebra.expressions.VariableReferenceExpression;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractOperatorWithNestedPlans;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AggregateOperator;
@@ -407,4 +408,25 @@
}
return clonedExprList;
}
+
+ /**
+ * Finds a variable assigned to a given expression and returns a new {@link VariableReferenceExpression}
+ * referring to this variable.
+ * @param assignVarList list of variables
+ * @param assignExprList list of expressions assigned to those variables
+ * @param searchExpr expression to search for
+ * @return said value, {@code null} if a variable is not found
+ */
+ public static VariableReferenceExpression findAssignedVariable(List<LogicalVariable> assignVarList,
+ List<Mutable<ILogicalExpression>> assignExprList, ILogicalExpression searchExpr) {
+ for (int i = 0, n = assignExprList.size(); i < n; i++) {
+ ILogicalExpression expr = assignExprList.get(i).getValue();
+ if (expr.equals(searchExpr)) {
+ VariableReferenceExpression result = new VariableReferenceExpression(assignVarList.get(i));
+ result.setSourceLocation(expr.getSourceLocation());
+ return result;
+ }
+ }
+ return null;
+ }
}
diff --git a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ConsolidateWindowOperatorsRule.java b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ConsolidateWindowOperatorsRule.java
new file mode 100644
index 0000000..61880a2
--- /dev/null
+++ b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ConsolidateWindowOperatorsRule.java
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hyracks.algebricks.rewriter.rules;
+
+import java.util.Collection;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+import org.apache.commons.lang3.mutable.Mutable;
+import org.apache.hyracks.algebricks.common.exceptions.AlgebricksException;
+import org.apache.hyracks.algebricks.core.algebra.base.ILogicalOperator;
+import org.apache.hyracks.algebricks.core.algebra.base.ILogicalPlan;
+import org.apache.hyracks.algebricks.core.algebra.base.IOptimizationContext;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalOperatorTag;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.AggregateOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.WindowOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.visitors.IsomorphismOperatorVisitor;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.visitors.IsomorphismUtilities;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.visitors.VariableUtilities;
+import org.apache.hyracks.algebricks.core.algebra.util.OperatorPropertiesUtil;
+import org.apache.hyracks.algebricks.core.rewriter.base.IAlgebraicRewriteRule;
+
+/**
+ * Merges two adjacent window operators into one if their window specifications are compatible.
+ * <pre>
+ * window [$x] <- [f()] with nested plan (aggergate [$a] <- [agg_1()] - ... - nts )
+ * window [$y] <- [g()] with nesedd plan (aggregate [$b] <- [agg_2()] - ... - nts )
+ * -->
+ * window [$x, $y] <- [f(), g()] with nested plan ( aggregate [$a, $b] <- [agg_1(), agg_2()] - ... - nts )
+ * </pre>
+ */
+public class ConsolidateWindowOperatorsRule implements IAlgebraicRewriteRule {
+
+ @Override
+ public boolean rewritePre(Mutable<ILogicalOperator> opRef, IOptimizationContext context) {
+ return false;
+ }
+
+ @Override
+ public boolean rewritePost(Mutable<ILogicalOperator> opRef, IOptimizationContext context)
+ throws AlgebricksException {
+ AbstractLogicalOperator op1 = (AbstractLogicalOperator) opRef.getValue();
+ if (op1.getOperatorTag() != LogicalOperatorTag.WINDOW) {
+ return false;
+ }
+ WindowOperator winOp1 = (WindowOperator) op1;
+
+ AbstractLogicalOperator op2 = (AbstractLogicalOperator) op1.getInputs().get(0).getValue();
+ if (op2.getOperatorTag() != LogicalOperatorTag.WINDOW) {
+ return false;
+ }
+
+ WindowOperator winOp2 = (WindowOperator) op2;
+
+ if (!IsomorphismOperatorVisitor.compareWindowPartitionSpec(winOp1, winOp2)) {
+ return false;
+ }
+ if (winOp1.hasNestedPlans() && winOp2.hasNestedPlans()
+ && !IsomorphismOperatorVisitor.compareWindowFrameSpec(winOp1, winOp2)) {
+ return false;
+ }
+
+ Set<LogicalVariable> used1 = new HashSet<>();
+ VariableUtilities.getUsedVariables(winOp1, used1);
+ if (!OperatorPropertiesUtil.disjoint(winOp2.getVariables(), used1)) {
+ return false;
+ }
+
+ if (winOp2.hasNestedPlans() && !consolidateNestedPlans(winOp1, winOp2, context)) {
+ return false;
+ }
+
+ winOp1.getVariables().addAll(winOp2.getVariables());
+ winOp1.getExpressions().addAll(winOp2.getExpressions());
+
+ winOp1.getInputs().clear();
+ winOp1.getInputs().addAll(winOp2.getInputs());
+ context.computeAndSetTypeEnvironmentForOperator(winOp1);
+
+ return true;
+ }
+
+ private boolean consolidateNestedPlans(WindowOperator winOpTo, WindowOperator winOpFrom,
+ IOptimizationContext context) throws AlgebricksException {
+ if (winOpTo.hasNestedPlans()) {
+ AggregateOperator aggTo = getAggregateRoot(winOpTo.getNestedPlans());
+ if (aggTo == null) {
+ return false;
+ }
+ AggregateOperator aggFrom = getAggregateRoot(winOpFrom.getNestedPlans());
+ if (aggFrom == null) {
+ return false;
+ }
+ if (!IsomorphismUtilities.isOperatorIsomorphicPlanSegment(aggTo.getInputs().get(0).getValue(),
+ aggFrom.getInputs().get(0).getValue())) {
+ return false;
+ }
+ aggTo.getVariables().addAll(aggFrom.getVariables());
+ aggTo.getExpressions().addAll(aggFrom.getExpressions());
+ context.computeAndSetTypeEnvironmentForOperator(aggTo);
+ } else {
+ setAll(winOpTo.getNestedPlans(), winOpFrom.getNestedPlans());
+ setAll(winOpTo.getFrameValueExpressions(), winOpFrom.getFrameValueExpressions());
+ setAll(winOpTo.getFrameStartExpressions(), winOpFrom.getFrameStartExpressions());
+ setAll(winOpTo.getFrameEndExpressions(), winOpFrom.getFrameEndExpressions());
+ setAll(winOpTo.getFrameExcludeExpressions(), winOpFrom.getFrameExcludeExpressions());
+ winOpTo.setFrameExcludeNegationStartIdx(winOpFrom.getFrameExcludeNegationStartIdx());
+ winOpTo.getFrameOffset().setValue(winOpFrom.getFrameOffset().getValue());
+ winOpTo.setFrameMaxObjects(winOpFrom.getFrameMaxObjects());
+ }
+ return true;
+ }
+
+ private AggregateOperator getAggregateRoot(List<ILogicalPlan> nestedPlans) {
+ if (nestedPlans.size() != 1) {
+ return null;
+ }
+ List<Mutable<ILogicalOperator>> roots = nestedPlans.get(0).getRoots();
+ if (roots.size() != 1) {
+ return null;
+ }
+ ILogicalOperator rootOp = roots.get(0).getValue();
+ if (rootOp.getOperatorTag() != LogicalOperatorTag.AGGREGATE) {
+ return null;
+ }
+ return (AggregateOperator) rootOp;
+ }
+
+ private <T> void setAll(Collection<? super T> to, Collection<? extends T> from) {
+ if (!to.isEmpty()) {
+ throw new IllegalStateException(String.valueOf(to.size()));
+ }
+ to.addAll(from);
+ }
+}
diff --git a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineAssignIntoAggregateRule.java b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineAssignIntoAggregateRule.java
index d0dff03..a97c38b 100644
--- a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineAssignIntoAggregateRule.java
+++ b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineAssignIntoAggregateRule.java
@@ -34,9 +34,9 @@
import org.apache.hyracks.algebricks.core.algebra.expressions.ConstantExpression;
import org.apache.hyracks.algebricks.core.algebra.expressions.VariableReferenceExpression;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractOperatorWithNestedPlans;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AggregateOperator;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AssignOperator;
-import org.apache.hyracks.algebricks.core.algebra.operators.logical.GroupByOperator;
import org.apache.hyracks.algebricks.core.algebra.visitors.AbstractConstVarFunVisitor;
import org.apache.hyracks.algebricks.core.rewriter.base.IAlgebraicRewriteRule;
@@ -51,48 +51,71 @@
public boolean rewritePost(Mutable<ILogicalOperator> opRef, IOptimizationContext context)
throws AlgebricksException {
AbstractLogicalOperator op = (AbstractLogicalOperator) opRef.getValue();
- if (op.getOperatorTag() != LogicalOperatorTag.GROUP) {
+ if (op.getOperatorTag() != LogicalOperatorTag.GROUP && op.getOperatorTag() != LogicalOperatorTag.WINDOW) {
return false;
}
boolean changed = false;
- GroupByOperator gbyOp = (GroupByOperator) op;
- for (ILogicalPlan p : gbyOp.getNestedPlans()) {
+ AbstractOperatorWithNestedPlans opWithNestedPlan = (AbstractOperatorWithNestedPlans) op;
+ for (ILogicalPlan p : opWithNestedPlan.getNestedPlans()) {
for (Mutable<ILogicalOperator> r : p.getRoots()) {
- if (inlined(r)) {
- changed = true;
- }
+ changed |= inlined(r.getValue(), opWithNestedPlan);
}
}
return changed;
}
- private boolean inlined(Mutable<ILogicalOperator> r) throws AlgebricksException {
- AbstractLogicalOperator op1 = (AbstractLogicalOperator) r.getValue();
+ private boolean inlined(ILogicalOperator planRootOp, AbstractOperatorWithNestedPlans opWithNestedPlan)
+ throws AlgebricksException {
+ AbstractLogicalOperator op1 = (AbstractLogicalOperator) planRootOp;
if (op1.getOperatorTag() != LogicalOperatorTag.AGGREGATE) {
return false;
}
- AbstractLogicalOperator op2 = (AbstractLogicalOperator) op1.getInputs().get(0).getValue();
+ AggregateOperator aggOp = (AggregateOperator) op1;
+ boolean inlined = inlineInputAssignIntoAgg(aggOp);
+ if (opWithNestedPlan.getOperatorTag() == LogicalOperatorTag.WINDOW) {
+ inlined |= inlineOuterInputAssignIntoAgg(aggOp, opWithNestedPlan);
+ }
+ return inlined;
+ }
+
+ private boolean inlineInputAssignIntoAgg(AggregateOperator aggOp) throws AlgebricksException {
+ AbstractLogicalOperator op2 = (AbstractLogicalOperator) aggOp.getInputs().get(0).getValue();
if (op2.getOperatorTag() != LogicalOperatorTag.ASSIGN) {
return false;
}
- AggregateOperator agg = (AggregateOperator) op1;
- AssignOperator assign = (AssignOperator) op2;
- VarExprSubstitution ves = new VarExprSubstitution(assign.getVariables(), assign.getExpressions());
- for (Mutable<ILogicalExpression> exprRef : agg.getExpressions()) {
- ILogicalExpression expr = exprRef.getValue();
- Pair<Boolean, ILogicalExpression> p = expr.accept(ves, null);
- if (p.first == true) {
- exprRef.setValue(p.second);
- }
- // AbstractLogicalExpression ale = (AbstractLogicalExpression) expr;
- // ale.accept(ves, null);
- }
- List<Mutable<ILogicalOperator>> op1InpList = op1.getInputs();
+ AssignOperator assignOp = (AssignOperator) op2;
+ VarExprSubstitution ves = new VarExprSubstitution(assignOp.getVariables(), assignOp.getExpressions());
+ inlineVariables(aggOp, ves);
+ List<Mutable<ILogicalOperator>> op1InpList = aggOp.getInputs();
op1InpList.clear();
op1InpList.add(op2.getInputs().get(0));
return true;
}
+ private boolean inlineOuterInputAssignIntoAgg(AggregateOperator aggOp,
+ AbstractOperatorWithNestedPlans opWithNestedPlans) throws AlgebricksException {
+ AbstractLogicalOperator op2 = (AbstractLogicalOperator) opWithNestedPlans.getInputs().get(0).getValue();
+ if (op2.getOperatorTag() != LogicalOperatorTag.ASSIGN) {
+ return false;
+ }
+ AssignOperator assignOp = (AssignOperator) op2;
+ VarExprSubstitution ves = new VarExprSubstitution(assignOp.getVariables(), assignOp.getExpressions());
+ return inlineVariables(aggOp, ves);
+ }
+
+ private boolean inlineVariables(AggregateOperator aggOp, VarExprSubstitution ves) throws AlgebricksException {
+ boolean inlined = false;
+ for (Mutable<ILogicalExpression> exprRef : aggOp.getExpressions()) {
+ ILogicalExpression expr = exprRef.getValue();
+ Pair<Boolean, ILogicalExpression> p = expr.accept(ves, null);
+ if (p.first) {
+ exprRef.setValue(p.second);
+ inlined = true;
+ }
+ }
+ return inlined;
+ }
+
private class VarExprSubstitution extends AbstractConstVarFunVisitor<Pair<Boolean, ILogicalExpression>, Void> {
private List<LogicalVariable> variables;
diff --git a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
index 5d6237a..729d6f9 100644
--- a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
+++ b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
@@ -39,8 +39,8 @@
import org.apache.hyracks.algebricks.core.algebra.expressions.VariableReferenceExpression;
import org.apache.hyracks.algebricks.core.algebra.functions.FunctionIdentifier;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractOperatorWithNestedPlans;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AssignOperator;
-import org.apache.hyracks.algebricks.core.algebra.operators.logical.SubplanOperator;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.visitors.VariableUtilities;
import org.apache.hyracks.algebricks.core.algebra.visitors.ILogicalExpressionReferenceTransform;
import org.apache.hyracks.algebricks.core.rewriter.base.IAlgebraicRewriteRule;
@@ -157,9 +157,9 @@
}
// Descend into subplan
- if (op.getOperatorTag() == LogicalOperatorTag.SUBPLAN) {
- SubplanOperator subplanOp = (SubplanOperator) op;
- for (ILogicalPlan nestedPlan : subplanOp.getNestedPlans()) {
+ if (op.getOperatorTag() == LogicalOperatorTag.SUBPLAN || op.getOperatorTag() == LogicalOperatorTag.WINDOW) {
+ List<ILogicalPlan> nestedPlans = ((AbstractOperatorWithNestedPlans) op).getNestedPlans();
+ for (ILogicalPlan nestedPlan : nestedPlans) {
for (Mutable<ILogicalOperator> root : nestedPlan.getRoots()) {
if (inlineVariables(root, context)) {
modified = true;
diff --git a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ReuseWindowAggregateRule.java b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ReuseWindowAggregateRule.java
new file mode 100644
index 0000000..59272ec
--- /dev/null
+++ b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ReuseWindowAggregateRule.java
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hyracks.algebricks.rewriter.rules;
+
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+
+import org.apache.commons.lang3.mutable.Mutable;
+import org.apache.commons.lang3.mutable.MutableObject;
+import org.apache.hyracks.algebricks.common.exceptions.AlgebricksException;
+import org.apache.hyracks.algebricks.core.algebra.base.ILogicalExpression;
+import org.apache.hyracks.algebricks.core.algebra.base.ILogicalOperator;
+import org.apache.hyracks.algebricks.core.algebra.base.IOptimizationContext;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalOperatorTag;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
+import org.apache.hyracks.algebricks.core.algebra.expressions.VariableReferenceExpression;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.AssignOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.WindowOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.visitors.IsomorphismOperatorVisitor;
+import org.apache.hyracks.algebricks.core.algebra.util.OperatorManipulationUtil;
+import org.apache.hyracks.algebricks.core.rewriter.base.IAlgebraicRewriteRule;
+
+/**
+ * If two adjacent window operators compute the same running aggregate then replace the second computation with
+ * the assign operator referring to the first operator's output variable:
+ *
+ * <pre>
+ * window [$x] <- [f()] ...
+ * window [$y] <- [f()] ...
+ * -->
+ * assign [$x] <- [$y]
+ * window [] <- [] ...
+ * window [$y] <- [f()] ...
+ * </pre>
+ *
+ * Both window operators must have the same partitioning specification.
+ *
+ * This rule must be followed by {@link RemoveRedundantVariablesRule} to substitute {@code $x} references with
+ * {@code $y} and then {@link RemoveUnusedAssignAndAggregateRule} to eliminate the new assign operator.
+ */
+public class ReuseWindowAggregateRule implements IAlgebraicRewriteRule {
+ @Override
+ public boolean rewritePre(Mutable<ILogicalOperator> opRef, IOptimizationContext context) {
+ return false;
+ }
+
+ @Override
+ public boolean rewritePost(Mutable<ILogicalOperator> opRef, IOptimizationContext context)
+ throws AlgebricksException {
+ AbstractLogicalOperator op1 = (AbstractLogicalOperator) opRef.getValue();
+ if (op1.getOperatorTag() != LogicalOperatorTag.WINDOW) {
+ return false;
+ }
+ WindowOperator winOp1 = (WindowOperator) op1;
+
+ AbstractLogicalOperator op2 = (AbstractLogicalOperator) op1.getInputs().get(0).getValue();
+ if (op2.getOperatorTag() != LogicalOperatorTag.WINDOW) {
+ return false;
+ }
+
+ WindowOperator winOp2 = (WindowOperator) op2;
+
+ if (!IsomorphismOperatorVisitor.compareWindowPartitionSpec(winOp1, winOp2)) {
+ return false;
+ }
+
+ List<LogicalVariable> assignVars = new ArrayList<>();
+ List<Mutable<ILogicalExpression>> assignExprs = new ArrayList<>();
+
+ List<LogicalVariable> varsOp1 = winOp1.getVariables();
+ List<LogicalVariable> varsOp2 = winOp2.getVariables();
+ List<Mutable<ILogicalExpression>> exprsOp1 = winOp1.getExpressions();
+ List<Mutable<ILogicalExpression>> exprsOp2 = winOp2.getExpressions();
+ Iterator<LogicalVariable> varsOp1Iter = varsOp1.iterator();
+ Iterator<Mutable<ILogicalExpression>> exprsOp1Iter = exprsOp1.iterator();
+ while (varsOp1Iter.hasNext()) {
+ LogicalVariable varOp1 = varsOp1Iter.next();
+ Mutable<ILogicalExpression> exprOp1 = exprsOp1Iter.next();
+ VariableReferenceExpression varOp2Ref =
+ OperatorManipulationUtil.findAssignedVariable(varsOp2, exprsOp2, exprOp1.getValue());
+ if (varOp2Ref != null) {
+ varsOp1Iter.remove();
+ exprsOp1Iter.remove();
+ assignVars.add(varOp1);
+ assignExprs.add(new MutableObject<>(varOp2Ref));
+ }
+ }
+
+ if (assignVars.isEmpty()) {
+ return false;
+ }
+
+ AssignOperator assignOp = new AssignOperator(assignVars, assignExprs);
+ assignOp.getInputs().add(new MutableObject<>(winOp1));
+ assignOp.setSourceLocation(winOp1.getSourceLocation());
+ context.computeAndSetTypeEnvironmentForOperator(assignOp);
+ opRef.setValue(assignOp);
+ return true;
+ }
+}
\ No newline at end of file
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2: Contrib+1
BAD Compatibility Tests Successful
https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/3891/ : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-stabilization-f69489-compat/337/ (5/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Ali Alsuliman (Code Review)" <do...@asterixdb.incubator.apache.org>.
Ali Alsuliman has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
(2 comments)
https://asterix-gerrit.ics.uci.edu/#/c/3107/2/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ConsolidateWindowOperatorsRule.java
File hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ConsolidateWindowOperatorsRule.java:
PS2, Line 46: aggergate
Typo?
PS2, Line 47: nesedd
Typo?
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage/5498/ (4/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Anon. E. Moose (Code Review)" <do...@asterixdb.incubator.apache.org>.
Anon. E. Moose #1000171 has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1: Contrib-2
Analytics Compatibility Tests Failed
https://goo.gl/mMTvMf : UNSTABLE
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack
PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-cancellation-test/4926/ (1/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Anon. E. Moose (Code Review)" <do...@asterixdb.incubator.apache.org>.
Anon. E. Moose #1000171 has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2: Contrib+1
Analytics Compatibility Tests Successful
https://goo.gl/HnkPSU : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/hyracks-gerrit/4838/ (2/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-sql-execution/4936/ (12/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1: Integration-Tests+1
Integration Tests Successful
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/7750/ : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-openjdk11/334/ (13/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/8875/ (9/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2: Integration-Tests+1
Integration Tests Successful
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/7752/ : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-format/4892/ (12/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-assemblies/5148/ (11/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/7752/
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-verify-txnlog/94/ (1/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-ensure-ancestor/2938/ (2/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Dmitry Lychagin (Code Review)" <do...@asterixdb.incubator.apache.org>.
Hello Anon. E. Moose #1000171, Jenkins,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/3107
to look at the new patch set (#2).
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
[ASTERIXDB-2505][COMP] Consolidate window operators rule
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
- Add optimizer rules that merge two window operators into one
- Improve existing optimizer rules to handle window operators
- Add tests for distinct aggregate in window operator
Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/LoadRecordFieldsRule.java
R asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_01.sqlpp
R asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_02.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_1.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_2.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_3.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_4.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_5.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_6.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_7.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_01.plan
R asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_02.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_1.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_2.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_3.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_4.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_5.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_6.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_7.plan
D asterixdb/asterix-app/src/test/resources/optimizerts/results/window/window_01.plan
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/misc_01/misc_01.7.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.1.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.2.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.3.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.5.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.6.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.7.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/misc_01/misc_01.7.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.2.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.3.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.4.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.5.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.6.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.7.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/window/misc_01/misc_01.7.ast
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppDistinctAggregationSugarVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/VariableCheckAndRewriteVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppAstPrintVisitor.java
M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/WindowOperator.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismVariableMappingVisitor.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorManipulationUtil.java
A hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ConsolidateWindowOperatorsRule.java
M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineAssignIntoAggregateRule.java
M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
A hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ReuseWindowAggregateRule.java
51 files changed, 1,222 insertions(+), 111 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/07/3107/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Dmitry Lychagin (Code Review)" <do...@asterixdb.incubator.apache.org>.
Dmitry Lychagin has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
(1 comment)
https://asterix-gerrit.ics.uci.edu/#/c/3107/2/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ConsolidateWindowOperatorsRule.java
File hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ConsolidateWindowOperatorsRule.java:
PS2, Line 46: aggergate
> Typo?
yes. will address both in a follow up change.
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/5265/ (14/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/10410/ (9/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Anon. E. Moose (Code Review)" <do...@asterixdb.incubator.apache.org>.
Anon. E. Moose #1000171 has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2: Contrib-2
Analytics Compatibility Tests Failed
https://goo.gl/yUitEC : UNSTABLE
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Anon. E. Moose (Code Review)" <do...@asterixdb.incubator.apache.org>.
Anon. E. Moose #1000171 has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Analytics Compatibility Compilation Successful
https://goo.gl/NxxPfu : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-format/4894/ (7/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-assemblies/5150/ (8/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Integration Tests Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/7750/
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-openjdk11/332/ (8/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Anon. E. Moose (Code Review)" <do...@asterixdb.incubator.apache.org>.
Anon. E. Moose #1000171 has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Analytics Compatibility Compilation Successful
https://goo.gl/uB74HM : SUCCESS
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-assemblies/5151/ (14/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/5312/ (11/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
BAD Compatibility Tests Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/3891/
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage/5496/ (6/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/hyracks-gerrit/4840/ (6/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-cancellation-test/4928/ (10/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/5267/ (14/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-stabilization-f69489-compat/339/ (5/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack
PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Dmitry Lychagin (Code Review)" <do...@asterixdb.incubator.apache.org>.
Dmitry Lychagin has submitted this change and it was merged.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
[ASTERIXDB-2505][COMP] Consolidate window operators rule
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
- Add optimizer rules that merge two window operators into one
- Improve existing optimizer rules to handle window operators
- Add tests for distinct aggregate in window operator
Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3107
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Contrib: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Ali Alsuliman <al...@gmail.com>
---
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/LoadRecordFieldsRule.java
R asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_01.sqlpp
R asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_02.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_1.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_2.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_3.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_4.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_5.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_6.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_7.sqlpp
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_01.plan
R asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_02.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_1.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_2.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_3.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_4.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_5.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_6.plan
A asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_7.plan
D asterixdb/asterix-app/src/test/resources/optimizerts/results/window/window_01.plan
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/misc_01/misc_01.7.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.1.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.2.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.3.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.4.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.5.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.6.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.7.query.sqlpp
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/misc_01/misc_01.7.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.1.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.2.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.3.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.4.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.5.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.6.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.7.adm
A asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/window/misc_01/misc_01.7.ast
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppDistinctAggregationSugarVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/VariableCheckAndRewriteVisitor.java
M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppAstPrintVisitor.java
M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/WindowOperator.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismVariableMappingVisitor.java
M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorManipulationUtil.java
A hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ConsolidateWindowOperatorsRule.java
M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineAssignIntoAggregateRule.java
M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
A hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ReuseWindowAggregateRule.java
51 files changed, 1,222 insertions(+), 111 deletions(-)
Approvals:
Anon. E. Moose #1000171:
Ali Alsuliman: Looks good to me, approved
Jenkins: Verified; ; Verified
Objections:
Jenkins: Violations found
diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
index ece78f8..80469a7 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
@@ -94,6 +94,8 @@
import org.apache.hyracks.algebricks.rewriter.rules.ComplexUnnestToProductRule;
import org.apache.hyracks.algebricks.rewriter.rules.ConsolidateAssignsRule;
import org.apache.hyracks.algebricks.rewriter.rules.ConsolidateSelectsRule;
+import org.apache.hyracks.algebricks.rewriter.rules.ReuseWindowAggregateRule;
+import org.apache.hyracks.algebricks.rewriter.rules.ConsolidateWindowOperatorsRule;
import org.apache.hyracks.algebricks.rewriter.rules.CopyLimitDownRule;
import org.apache.hyracks.algebricks.rewriter.rules.EliminateGroupByEmptyKeyRule;
import org.apache.hyracks.algebricks.rewriter.rules.EnforceOrderByAfterSubplan;
@@ -291,6 +293,10 @@
//PushUnnestDownUnion => RemoveRedundantListifyRule cause these rules are correlated
consolidation.add(new PushUnnestDownThroughUnionRule());
consolidation.add(new RemoveRedundantListifyRule());
+ // Window operator consolidation rules
+ consolidation.add(new ConsolidateWindowOperatorsRule());
+ consolidation.add(new ReuseWindowAggregateRule());
+ consolidation.add(new RemoveRedundantVariablesRule());
return consolidation;
}
diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/LoadRecordFieldsRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/LoadRecordFieldsRule.java
index 32fa744..4d61307 100644
--- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/LoadRecordFieldsRule.java
+++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/LoadRecordFieldsRule.java
@@ -321,10 +321,10 @@
}
} else if (opChild.getOperatorTag() == LogicalOperatorTag.NESTEDTUPLESOURCE) {
NestedTupleSourceOperator nts = (NestedTupleSourceOperator) opChild;
- AbstractLogicalOperator opBelowNestedPlan =
- (AbstractLogicalOperator) nts.getDataSourceReference().getValue().getInputs().get(0).getValue();
+ AbstractLogicalOperator opWithNestedPlan =
+ (AbstractLogicalOperator) nts.getDataSourceReference().getValue();
ILogicalExpression expr1 =
- findFieldExpression(opBelowNestedPlan, recordVar, accessKey, typeEnvironment, resolver);
+ findFieldExpression(opWithNestedPlan, recordVar, accessKey, typeEnvironment, resolver);
if (expr1 != null) {
return expr1;
}
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/window_01.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_01.sqlpp
similarity index 100%
rename from asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/window_01.sqlpp
rename to asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_01.sqlpp
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/window_02.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_02.sqlpp
similarity index 100%
rename from asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/window_02.sqlpp
rename to asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_misc/win_misc_02.sqlpp
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_1.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_1.sqlpp
new file mode 100644
index 0000000..d53d092
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_1.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of two running aggregates
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ round_half_to_even(percent_rank() over (partition by m order by t), 2) as `percent_rank`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_2.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_2.sqlpp
new file mode 100644
index 0000000..c403726
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_2.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of one running aggregate and one regular aggregate
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ sum(t) over (partition by m order by t) as `sum`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_3.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_3.sqlpp
new file mode 100644
index 0000000..25d559d
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_3.sqlpp
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of one running aggregate
+ * and one regular aggregate with complex argument
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ sum(t + m) over (partition by m order by t) as `sum`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_4.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_4.sqlpp
new file mode 100644
index 0000000..06f5ac3
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_4.sqlpp
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of one running aggregate and
+ * two regular aggregates with complex arguments
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ sum(t + m) over (partition by m order by t) as `sum`,
+ round_half_to_even(avg(t - m) over (partition by m order by t), 2) as `avg`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_5.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_5.sqlpp
new file mode 100644
index 0000000..b6bfd66
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_5.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of two window functions in LET clauses
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t, row_number, ntile
+FROM range(1, 16) t
+LET m = t % 4,
+ row_number = row_number() over (partition by m order by t),
+ ntile = ntile(2) over (partition by m order by t)
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_6.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_6.sqlpp
new file mode 100644
index 0000000..0019ba8
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_6.sqlpp
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of data dependency between two window functions.
+ * No operator consolidation in this case.
+ * Expected Res : SUCCESS (two window operators in the optimized plan)
+ */
+
+SELECT m, t, row_number, ntile
+FROM range(1, 16) t
+LET m = t % 4,
+ row_number = row_number() over (partition by m order by t),
+ ntile = ntile(row_number + 1) over (partition by m order by t)
+ORDER BY m, t;
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_7.sqlpp b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_7.sqlpp
new file mode 100644
index 0000000..f139ab8
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/queries/window/win_opt_01/win_opt_01_7.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of ROW frame mode
+ * Expected Res : SUCCESS (two window operators in the optimized plan)
+ */
+
+SELECT m, t,
+ sum(t) over (partition by m order by t rows between unbounded preceding and current row) as `sum`,
+ round_half_to_even(avg(t) over (partition by m order by t rows between unbounded preceding and current row), 2) as `avg`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_01.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_01.plan
new file mode 100644
index 0000000..8191e74
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_01.plan
@@ -0,0 +1,29 @@
+-- DISTRIBUTE_RESULT |UNPARTITIONED|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- STREAM_PROJECT |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- AGGREGATE |UNPARTITIONED|
+ -- RANDOM_MERGE_EXCHANGE |PARTITIONED|
+ -- AGGREGATE |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- ASSIGN |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- ASSIGN |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- ASSIGN |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- WINDOW |PARTITIONED|
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ -- STABLE_SORT [$$t1.c2(ASC), $$p(ASC), $$n(DESC)] |PARTITIONED|
+ -- HASH_PARTITION_EXCHANGE [$$t1.c2, $$p] |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- UNNEST |PARTITIONED|
+ -- UNNEST |PARTITIONED|
+ -- UNNEST |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- ASSIGN |PARTITIONED|
+ -- STREAM_PROJECT |PARTITIONED|
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ -- DATASOURCE_SCAN |PARTITIONED|
+ -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |PARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/window_02.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_02.plan
similarity index 100%
rename from asterixdb/asterix-app/src/test/resources/optimizerts/results/window/window_02.plan
rename to asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_misc/win_misc_02.plan
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_1.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_1.plan
new file mode 100644
index 0000000..871ea8e
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_1.plan
@@ -0,0 +1,11 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- WINDOW |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_2.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_2.plan
new file mode 100644
index 0000000..a131487
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_2.plan
@@ -0,0 +1,15 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- WINDOW |LOCAL|
+ {
+ -- AGGREGATE |LOCAL|
+ -- NESTED_TUPLE_SOURCE |LOCAL|
+ }
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_3.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_3.plan
new file mode 100644
index 0000000..a131487
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_3.plan
@@ -0,0 +1,15 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- WINDOW |LOCAL|
+ {
+ -- AGGREGATE |LOCAL|
+ -- NESTED_TUPLE_SOURCE |LOCAL|
+ }
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_4.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_4.plan
new file mode 100644
index 0000000..a131487
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_4.plan
@@ -0,0 +1,15 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- WINDOW |LOCAL|
+ {
+ -- AGGREGATE |LOCAL|
+ -- NESTED_TUPLE_SOURCE |LOCAL|
+ }
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_5.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_5.plan
new file mode 100644
index 0000000..871ea8e
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_5.plan
@@ -0,0 +1,11 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- WINDOW |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_6.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_6.plan
new file mode 100644
index 0000000..ab78ecc
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_6.plan
@@ -0,0 +1,12 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- WINDOW |LOCAL|
+ -- WINDOW |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_7.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_7.plan
new file mode 100644
index 0000000..5b3d480
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/win_opt_01/win_opt_01_7.plan
@@ -0,0 +1,17 @@
+-- DISTRIBUTE_RESULT |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- ASSIGN |LOCAL|
+ -- STREAM_PROJECT |LOCAL|
+ -- WINDOW |LOCAL|
+ {
+ -- AGGREGATE |LOCAL|
+ -- NESTED_TUPLE_SOURCE |LOCAL|
+ }
+ -- WINDOW |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |LOCAL|
+ -- STABLE_SORT [$$m(ASC), $$t(ASC)] |LOCAL|
+ -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
+ -- ASSIGN |UNPARTITIONED|
+ -- UNNEST |UNPARTITIONED|
+ -- EMPTY_TUPLE_SOURCE |UNPARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/window_01.plan b/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/window_01.plan
deleted file mode 100644
index 91e9cfc..0000000
--- a/asterixdb/asterix-app/src/test/resources/optimizerts/results/window/window_01.plan
+++ /dev/null
@@ -1,30 +0,0 @@
--- DISTRIBUTE_RESULT |UNPARTITIONED|
- -- ONE_TO_ONE_EXCHANGE |UNPARTITIONED|
- -- STREAM_PROJECT |UNPARTITIONED|
- -- ASSIGN |UNPARTITIONED|
- -- AGGREGATE |UNPARTITIONED|
- -- RANDOM_MERGE_EXCHANGE |PARTITIONED|
- -- AGGREGATE |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- ASSIGN |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- ASSIGN |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- ASSIGN |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- WINDOW |PARTITIONED|
- -- WINDOW |PARTITIONED|
- -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
- -- STABLE_SORT [$$t1.c2(ASC), $$p(ASC), $$n(DESC)] |PARTITIONED|
- -- HASH_PARTITION_EXCHANGE [$$t1.c2, $$p] |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- UNNEST |PARTITIONED|
- -- UNNEST |PARTITIONED|
- -- UNNEST |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- ASSIGN |PARTITIONED|
- -- STREAM_PROJECT |PARTITIONED|
- -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
- -- DATASOURCE_SCAN |PARTITIONED|
- -- ONE_TO_ONE_EXCHANGE |PARTITIONED|
- -- EMPTY_TUPLE_SOURCE |PARTITIONED|
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/misc_01/misc_01.7.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/misc_01/misc_01.7.query.sqlpp
new file mode 100644
index 0000000..a04ab08
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/misc_01/misc_01.7.query.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window function distinct aggregate
+ * Expected Res : SUCCESS
+ */
+
+SELECT m, t,
+ sum(t % 4) over (partition by m order by t rows between unbounded preceding and current row) as `sum`,
+ sum(distinct t % 4) over (partition by m order by t rows between unbounded preceding and current row) as `sum_distinct`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.1.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.1.query.sqlpp
new file mode 100644
index 0000000..d53d092
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.1.query.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of two running aggregates
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ round_half_to_even(percent_rank() over (partition by m order by t), 2) as `percent_rank`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.2.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.2.query.sqlpp
new file mode 100644
index 0000000..c403726
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.2.query.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of one running aggregate and one regular aggregate
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ sum(t) over (partition by m order by t) as `sum`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.3.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.3.query.sqlpp
new file mode 100644
index 0000000..25d559d
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.3.query.sqlpp
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of one running aggregate
+ * and one regular aggregate with complex argument
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ sum(t + m) over (partition by m order by t) as `sum`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.4.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.4.query.sqlpp
new file mode 100644
index 0000000..06f5ac3
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.4.query.sqlpp
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of one running aggregate and
+ * two regular aggregates with complex arguments
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t,
+ row_number() over (partition by m order by t) as `row_nubmer`,
+ sum(t + m) over (partition by m order by t) as `sum`,
+ round_half_to_even(avg(t - m) over (partition by m order by t), 2) as `avg`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.5.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.5.query.sqlpp
new file mode 100644
index 0000000..b6bfd66
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.5.query.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of two window functions in LET clauses
+ * Expected Res : SUCCESS (one window operator in the optimized plan)
+ */
+
+SELECT m, t, row_number, ntile
+FROM range(1, 16) t
+LET m = t % 4,
+ row_number = row_number() over (partition by m order by t),
+ ntile = ntile(2) over (partition by m order by t)
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.6.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.6.query.sqlpp
new file mode 100644
index 0000000..0019ba8
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.6.query.sqlpp
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of data dependency between two window functions.
+ * No operator consolidation in this case.
+ * Expected Res : SUCCESS (two window operators in the optimized plan)
+ */
+
+SELECT m, t, row_number, ntile
+FROM range(1, 16) t
+LET m = t % 4,
+ row_number = row_number() over (partition by m order by t),
+ ntile = ntile(row_number + 1) over (partition by m order by t)
+ORDER BY m, t;
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.7.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.7.query.sqlpp
new file mode 100644
index 0000000..f139ab8
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/window/win_opt_01/win_opt_01.7.query.sqlpp
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+/*
+ * Description : Test window operator consolidation rule handling of ROW frame mode
+ * Expected Res : SUCCESS (two window operators in the optimized plan)
+ */
+
+SELECT m, t,
+ sum(t) over (partition by m order by t rows between unbounded preceding and current row) as `sum`,
+ round_half_to_even(avg(t) over (partition by m order by t rows between unbounded preceding and current row), 2) as `avg`
+FROM range(1, 20) t
+LET m = t % 4
+ORDER BY m, t;
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/misc_01/misc_01.7.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/misc_01/misc_01.7.adm
new file mode 100644
index 0000000..9deb329
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/misc_01/misc_01.7.adm
@@ -0,0 +1,20 @@
+{ "m": 0, "t": 4, "sum": 0, "sum_distinct": 0 }
+{ "m": 0, "t": 8, "sum": 0, "sum_distinct": 0 }
+{ "m": 0, "t": 12, "sum": 0, "sum_distinct": 0 }
+{ "m": 0, "t": 16, "sum": 0, "sum_distinct": 0 }
+{ "m": 0, "t": 20, "sum": 0, "sum_distinct": 0 }
+{ "m": 1, "t": 1, "sum": 1, "sum_distinct": 1 }
+{ "m": 1, "t": 5, "sum": 2, "sum_distinct": 1 }
+{ "m": 1, "t": 9, "sum": 3, "sum_distinct": 1 }
+{ "m": 1, "t": 13, "sum": 4, "sum_distinct": 1 }
+{ "m": 1, "t": 17, "sum": 5, "sum_distinct": 1 }
+{ "m": 2, "t": 2, "sum": 2, "sum_distinct": 2 }
+{ "m": 2, "t": 6, "sum": 4, "sum_distinct": 2 }
+{ "m": 2, "t": 10, "sum": 6, "sum_distinct": 2 }
+{ "m": 2, "t": 14, "sum": 8, "sum_distinct": 2 }
+{ "m": 2, "t": 18, "sum": 10, "sum_distinct": 2 }
+{ "m": 3, "t": 3, "sum": 3, "sum_distinct": 3 }
+{ "m": 3, "t": 7, "sum": 6, "sum_distinct": 3 }
+{ "m": 3, "t": 11, "sum": 9, "sum_distinct": 3 }
+{ "m": 3, "t": 15, "sum": 12, "sum_distinct": 3 }
+{ "m": 3, "t": 19, "sum": 15, "sum_distinct": 3 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.1.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.1.adm
new file mode 100644
index 0000000..75263ed
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.1.adm
@@ -0,0 +1,20 @@
+{ "m": 0, "t": 4, "row_nubmer": 1, "percent_rank": 0.0 }
+{ "m": 0, "t": 8, "row_nubmer": 2, "percent_rank": 0.25 }
+{ "m": 0, "t": 12, "row_nubmer": 3, "percent_rank": 0.5 }
+{ "m": 0, "t": 16, "row_nubmer": 4, "percent_rank": 0.75 }
+{ "m": 0, "t": 20, "row_nubmer": 5, "percent_rank": 1.0 }
+{ "m": 1, "t": 1, "row_nubmer": 1, "percent_rank": 0.0 }
+{ "m": 1, "t": 5, "row_nubmer": 2, "percent_rank": 0.25 }
+{ "m": 1, "t": 9, "row_nubmer": 3, "percent_rank": 0.5 }
+{ "m": 1, "t": 13, "row_nubmer": 4, "percent_rank": 0.75 }
+{ "m": 1, "t": 17, "row_nubmer": 5, "percent_rank": 1.0 }
+{ "m": 2, "t": 2, "row_nubmer": 1, "percent_rank": 0.0 }
+{ "m": 2, "t": 6, "row_nubmer": 2, "percent_rank": 0.25 }
+{ "m": 2, "t": 10, "row_nubmer": 3, "percent_rank": 0.5 }
+{ "m": 2, "t": 14, "row_nubmer": 4, "percent_rank": 0.75 }
+{ "m": 2, "t": 18, "row_nubmer": 5, "percent_rank": 1.0 }
+{ "m": 3, "t": 3, "row_nubmer": 1, "percent_rank": 0.0 }
+{ "m": 3, "t": 7, "row_nubmer": 2, "percent_rank": 0.25 }
+{ "m": 3, "t": 11, "row_nubmer": 3, "percent_rank": 0.5 }
+{ "m": 3, "t": 15, "row_nubmer": 4, "percent_rank": 0.75 }
+{ "m": 3, "t": 19, "row_nubmer": 5, "percent_rank": 1.0 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.2.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.2.adm
new file mode 100644
index 0000000..a08ac96
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.2.adm
@@ -0,0 +1,20 @@
+{ "m": 0, "t": 4, "row_nubmer": 1, "sum": 4 }
+{ "m": 0, "t": 8, "row_nubmer": 2, "sum": 12 }
+{ "m": 0, "t": 12, "row_nubmer": 3, "sum": 24 }
+{ "m": 0, "t": 16, "row_nubmer": 4, "sum": 40 }
+{ "m": 0, "t": 20, "row_nubmer": 5, "sum": 60 }
+{ "m": 1, "t": 1, "row_nubmer": 1, "sum": 1 }
+{ "m": 1, "t": 5, "row_nubmer": 2, "sum": 6 }
+{ "m": 1, "t": 9, "row_nubmer": 3, "sum": 15 }
+{ "m": 1, "t": 13, "row_nubmer": 4, "sum": 28 }
+{ "m": 1, "t": 17, "row_nubmer": 5, "sum": 45 }
+{ "m": 2, "t": 2, "row_nubmer": 1, "sum": 2 }
+{ "m": 2, "t": 6, "row_nubmer": 2, "sum": 8 }
+{ "m": 2, "t": 10, "row_nubmer": 3, "sum": 18 }
+{ "m": 2, "t": 14, "row_nubmer": 4, "sum": 32 }
+{ "m": 2, "t": 18, "row_nubmer": 5, "sum": 50 }
+{ "m": 3, "t": 3, "row_nubmer": 1, "sum": 3 }
+{ "m": 3, "t": 7, "row_nubmer": 2, "sum": 10 }
+{ "m": 3, "t": 11, "row_nubmer": 3, "sum": 21 }
+{ "m": 3, "t": 15, "row_nubmer": 4, "sum": 36 }
+{ "m": 3, "t": 19, "row_nubmer": 5, "sum": 55 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.3.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.3.adm
new file mode 100644
index 0000000..f356720
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.3.adm
@@ -0,0 +1,20 @@
+{ "m": 0, "t": 4, "row_nubmer": 1, "sum": 4 }
+{ "m": 0, "t": 8, "row_nubmer": 2, "sum": 12 }
+{ "m": 0, "t": 12, "row_nubmer": 3, "sum": 24 }
+{ "m": 0, "t": 16, "row_nubmer": 4, "sum": 40 }
+{ "m": 0, "t": 20, "row_nubmer": 5, "sum": 60 }
+{ "m": 1, "t": 1, "row_nubmer": 1, "sum": 2 }
+{ "m": 1, "t": 5, "row_nubmer": 2, "sum": 8 }
+{ "m": 1, "t": 9, "row_nubmer": 3, "sum": 18 }
+{ "m": 1, "t": 13, "row_nubmer": 4, "sum": 32 }
+{ "m": 1, "t": 17, "row_nubmer": 5, "sum": 50 }
+{ "m": 2, "t": 2, "row_nubmer": 1, "sum": 4 }
+{ "m": 2, "t": 6, "row_nubmer": 2, "sum": 12 }
+{ "m": 2, "t": 10, "row_nubmer": 3, "sum": 24 }
+{ "m": 2, "t": 14, "row_nubmer": 4, "sum": 40 }
+{ "m": 2, "t": 18, "row_nubmer": 5, "sum": 60 }
+{ "m": 3, "t": 3, "row_nubmer": 1, "sum": 6 }
+{ "m": 3, "t": 7, "row_nubmer": 2, "sum": 16 }
+{ "m": 3, "t": 11, "row_nubmer": 3, "sum": 30 }
+{ "m": 3, "t": 15, "row_nubmer": 4, "sum": 48 }
+{ "m": 3, "t": 19, "row_nubmer": 5, "sum": 70 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.4.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.4.adm
new file mode 100644
index 0000000..e14d3a2
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.4.adm
@@ -0,0 +1,20 @@
+{ "m": 0, "t": 4, "row_nubmer": 1, "sum": 4, "avg": 4.0 }
+{ "m": 0, "t": 8, "row_nubmer": 2, "sum": 12, "avg": 6.0 }
+{ "m": 0, "t": 12, "row_nubmer": 3, "sum": 24, "avg": 8.0 }
+{ "m": 0, "t": 16, "row_nubmer": 4, "sum": 40, "avg": 10.0 }
+{ "m": 0, "t": 20, "row_nubmer": 5, "sum": 60, "avg": 12.0 }
+{ "m": 1, "t": 1, "row_nubmer": 1, "sum": 2, "avg": 0.0 }
+{ "m": 1, "t": 5, "row_nubmer": 2, "sum": 8, "avg": 2.0 }
+{ "m": 1, "t": 9, "row_nubmer": 3, "sum": 18, "avg": 4.0 }
+{ "m": 1, "t": 13, "row_nubmer": 4, "sum": 32, "avg": 6.0 }
+{ "m": 1, "t": 17, "row_nubmer": 5, "sum": 50, "avg": 8.0 }
+{ "m": 2, "t": 2, "row_nubmer": 1, "sum": 4, "avg": 0.0 }
+{ "m": 2, "t": 6, "row_nubmer": 2, "sum": 12, "avg": 2.0 }
+{ "m": 2, "t": 10, "row_nubmer": 3, "sum": 24, "avg": 4.0 }
+{ "m": 2, "t": 14, "row_nubmer": 4, "sum": 40, "avg": 6.0 }
+{ "m": 2, "t": 18, "row_nubmer": 5, "sum": 60, "avg": 8.0 }
+{ "m": 3, "t": 3, "row_nubmer": 1, "sum": 6, "avg": 0.0 }
+{ "m": 3, "t": 7, "row_nubmer": 2, "sum": 16, "avg": 2.0 }
+{ "m": 3, "t": 11, "row_nubmer": 3, "sum": 30, "avg": 4.0 }
+{ "m": 3, "t": 15, "row_nubmer": 4, "sum": 48, "avg": 6.0 }
+{ "m": 3, "t": 19, "row_nubmer": 5, "sum": 70, "avg": 8.0 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.5.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.5.adm
new file mode 100644
index 0000000..fa8f4af
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.5.adm
@@ -0,0 +1,16 @@
+{ "m": 0, "t": 4, "row_number": 1, "ntile": 1 }
+{ "m": 0, "t": 8, "row_number": 2, "ntile": 1 }
+{ "m": 0, "t": 12, "row_number": 3, "ntile": 2 }
+{ "m": 0, "t": 16, "row_number": 4, "ntile": 2 }
+{ "m": 1, "t": 1, "row_number": 1, "ntile": 1 }
+{ "m": 1, "t": 5, "row_number": 2, "ntile": 1 }
+{ "m": 1, "t": 9, "row_number": 3, "ntile": 2 }
+{ "m": 1, "t": 13, "row_number": 4, "ntile": 2 }
+{ "m": 2, "t": 2, "row_number": 1, "ntile": 1 }
+{ "m": 2, "t": 6, "row_number": 2, "ntile": 1 }
+{ "m": 2, "t": 10, "row_number": 3, "ntile": 2 }
+{ "m": 2, "t": 14, "row_number": 4, "ntile": 2 }
+{ "m": 3, "t": 3, "row_number": 1, "ntile": 1 }
+{ "m": 3, "t": 7, "row_number": 2, "ntile": 1 }
+{ "m": 3, "t": 11, "row_number": 3, "ntile": 2 }
+{ "m": 3, "t": 15, "row_number": 4, "ntile": 2 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.6.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.6.adm
new file mode 100644
index 0000000..fa8f4af
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.6.adm
@@ -0,0 +1,16 @@
+{ "m": 0, "t": 4, "row_number": 1, "ntile": 1 }
+{ "m": 0, "t": 8, "row_number": 2, "ntile": 1 }
+{ "m": 0, "t": 12, "row_number": 3, "ntile": 2 }
+{ "m": 0, "t": 16, "row_number": 4, "ntile": 2 }
+{ "m": 1, "t": 1, "row_number": 1, "ntile": 1 }
+{ "m": 1, "t": 5, "row_number": 2, "ntile": 1 }
+{ "m": 1, "t": 9, "row_number": 3, "ntile": 2 }
+{ "m": 1, "t": 13, "row_number": 4, "ntile": 2 }
+{ "m": 2, "t": 2, "row_number": 1, "ntile": 1 }
+{ "m": 2, "t": 6, "row_number": 2, "ntile": 1 }
+{ "m": 2, "t": 10, "row_number": 3, "ntile": 2 }
+{ "m": 2, "t": 14, "row_number": 4, "ntile": 2 }
+{ "m": 3, "t": 3, "row_number": 1, "ntile": 1 }
+{ "m": 3, "t": 7, "row_number": 2, "ntile": 1 }
+{ "m": 3, "t": 11, "row_number": 3, "ntile": 2 }
+{ "m": 3, "t": 15, "row_number": 4, "ntile": 2 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.7.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.7.adm
new file mode 100644
index 0000000..30ed6e2
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/window/win_opt_01/win_opt_01.7.adm
@@ -0,0 +1,20 @@
+{ "m": 0, "t": 4, "sum": 4, "avg": 4.0 }
+{ "m": 0, "t": 8, "sum": 12, "avg": 6.0 }
+{ "m": 0, "t": 12, "sum": 24, "avg": 8.0 }
+{ "m": 0, "t": 16, "sum": 40, "avg": 10.0 }
+{ "m": 0, "t": 20, "sum": 60, "avg": 12.0 }
+{ "m": 1, "t": 1, "sum": 1, "avg": 1.0 }
+{ "m": 1, "t": 5, "sum": 6, "avg": 3.0 }
+{ "m": 1, "t": 9, "sum": 15, "avg": 5.0 }
+{ "m": 1, "t": 13, "sum": 28, "avg": 7.0 }
+{ "m": 1, "t": 17, "sum": 45, "avg": 9.0 }
+{ "m": 2, "t": 2, "sum": 2, "avg": 2.0 }
+{ "m": 2, "t": 6, "sum": 8, "avg": 4.0 }
+{ "m": 2, "t": 10, "sum": 18, "avg": 6.0 }
+{ "m": 2, "t": 14, "sum": 32, "avg": 8.0 }
+{ "m": 2, "t": 18, "sum": 50, "avg": 10.0 }
+{ "m": 3, "t": 3, "sum": 3, "avg": 3.0 }
+{ "m": 3, "t": 7, "sum": 10, "avg": 5.0 }
+{ "m": 3, "t": 11, "sum": 21, "avg": 7.0 }
+{ "m": 3, "t": 15, "sum": 36, "avg": 9.0 }
+{ "m": 3, "t": 19, "sum": 55, "avg": 11.0 }
\ No newline at end of file
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/window/misc_01/misc_01.7.ast b/asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/window/misc_01/misc_01.7.ast
new file mode 100644
index 0000000..2271af5
--- /dev/null
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/results_parser_sqlpp/window/misc_01/misc_01.7.ast
@@ -0,0 +1,58 @@
+Query:
+SELECT [
+Variable [ Name=$m ]
+m
+Variable [ Name=$t ]
+t
+WINDOW null.sum@1[
+ OperatorExpr [
+ Variable [ Name=$t ]
+ mod
+ LiteralExpr [LONG] [4]
+ ]
+]
+OVER (
+ PARTITION BY
+ Variable [ Name=$m ]
+ ORDER BY
+ Variable [ Name=$t ]
+ ASC
+ rows between unbounded preceding and current row exclude no others
+)
+sum
+WINDOW null.sum-distinct@1[
+ OperatorExpr [
+ Variable [ Name=$t ]
+ mod
+ LiteralExpr [LONG] [4]
+ ]
+]
+OVER (
+ PARTITION BY
+ Variable [ Name=$m ]
+ ORDER BY
+ Variable [ Name=$t ]
+ ASC
+ rows between unbounded preceding and current row exclude no others
+)
+sum_distinct
+]
+FROM [ FunctionCall null.range@2[
+ LiteralExpr [LONG] [1]
+ LiteralExpr [LONG] [20]
+ ]
+ AS Variable [ Name=$t ]
+]
+Let Variable [ Name=$m ]
+ :=
+ OperatorExpr [
+ Variable [ Name=$t ]
+ mod
+ LiteralExpr [LONG] [4]
+ ]
+Orderby
+ Variable [ Name=$m ]
+ ASC
+ Variable [ Name=$t ]
+ ASC
+
diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
index 7bd1eaf..3a857c1 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
@@ -9260,6 +9260,11 @@
<output-dir compare="Text">row_number_01</output-dir>
</compilation-unit>
</test-case>
+ <test-case FilePath="window">
+ <compilation-unit name="win_opt_01">
+ <output-dir compare="Text">win_opt_01</output-dir>
+ </compilation-unit>
+ </test-case>
</test-group>
<test-group name="writers">
<test-case FilePath="writers">
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppDistinctAggregationSugarVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppDistinctAggregationSugarVisitor.java
index 80844ab..efcd7e0 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppDistinctAggregationSugarVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppDistinctAggregationSugarVisitor.java
@@ -38,6 +38,7 @@
import org.apache.asterix.lang.sqlpp.clause.SelectElement;
import org.apache.asterix.lang.sqlpp.clause.SelectSetOperation;
import org.apache.asterix.lang.sqlpp.expression.SelectExpression;
+import org.apache.asterix.lang.sqlpp.expression.WindowExpression;
import org.apache.asterix.lang.sqlpp.struct.SetOperationInput;
import org.apache.asterix.lang.sqlpp.visitor.base.AbstractSqlppSimpleExpressionVisitor;
import org.apache.asterix.om.functions.BuiltinFunctions;
@@ -59,25 +60,45 @@
@Override
public Expression visit(CallExpr callExpr, ILangExpression arg) throws CompilationException {
- FunctionSignature signature = callExpr.getFunctionSignature();
- IFunctionInfo finfo = FunctionUtil.getFunctionInfo(signature);
- FunctionIdentifier aggFn =
- finfo != null ? BuiltinFunctions.getAggregateFunction(finfo.getFunctionIdentifier()) : null;
- FunctionIdentifier newAggFn = aggFn != null ? BuiltinFunctions.getAggregateFunctionForDistinct(aggFn) : null;
+ FunctionIdentifier newAggFn = getAggregateFunctionForDistinct(callExpr.getFunctionSignature());
if (newAggFn == null) {
return super.visit(callExpr, arg);
}
- List<Expression> exprList = callExpr.getExprList();
- List<Expression> newExprList = new ArrayList<>(exprList.size());
- for (Expression expr : exprList) {
- Expression newExpr = rewriteArgument(expr);
- newExprList.add(newExpr.accept(this, arg));
- }
+ List<Expression> newExprList = rewriteArgumentList(callExpr.getExprList(), arg);
callExpr.setFunctionSignature(new FunctionSignature(newAggFn));
callExpr.setExprList(newExprList);
return callExpr;
}
+ @Override
+ public Expression visit(WindowExpression winExpr, ILangExpression arg) throws CompilationException {
+ FunctionIdentifier newAggFn = getAggregateFunctionForDistinct(winExpr.getFunctionSignature());
+ if (newAggFn == null) {
+ return super.visit(winExpr, arg);
+ }
+ List<Expression> newExprList = rewriteArgumentList(winExpr.getExprList(), arg);
+ winExpr.setFunctionSignature(new FunctionSignature(newAggFn));
+ winExpr.setExprList(newExprList);
+ return winExpr;
+ }
+
+ private FunctionIdentifier getAggregateFunctionForDistinct(FunctionSignature signature) {
+ IFunctionInfo finfo = FunctionUtil.getFunctionInfo(signature);
+ FunctionIdentifier aggFn =
+ finfo != null ? BuiltinFunctions.getAggregateFunction(finfo.getFunctionIdentifier()) : null;
+ return aggFn != null ? BuiltinFunctions.getAggregateFunctionForDistinct(aggFn) : null;
+ }
+
+ private List<Expression> rewriteArgumentList(List<Expression> exprList, ILangExpression arg)
+ throws CompilationException {
+ List<Expression> result = new ArrayList<>(exprList.size());
+ for (Expression expr : exprList) {
+ Expression newExpr = rewriteArgument(expr);
+ result.add(newExpr.accept(this, arg));
+ }
+ return result;
+ }
+
/**
* rewrites {@code expr -> FROM expr AS i SELECT DISTINCT VALUE i}
*/
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/VariableCheckAndRewriteVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/VariableCheckAndRewriteVisitor.java
index b48e024..7ffebc2 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/VariableCheckAndRewriteVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/VariableCheckAndRewriteVisitor.java
@@ -204,6 +204,7 @@
return path.length == 2 && metadataProvider.findDataset(path[0], path[1]) != null;
}
+ @Override
public Expression visit(CallExpr callExpr, ILangExpression arg) throws CompilationException {
// skip variables inside SQL-92 aggregates (they will be resolved by SqlppGroupByAggregationSugarVisitor)
if (FunctionMapUtil.isSql92AggregateFunction(callExpr.getFunctionSignature())) {
diff --git a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppAstPrintVisitor.java b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppAstPrintVisitor.java
index f1039ff..d906a82 100644
--- a/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppAstPrintVisitor.java
+++ b/asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/visitor/SqlppAstPrintVisitor.java
@@ -379,8 +379,8 @@
}
}
if (winExpr.hasFrameDefinition()) {
- out.println(skip(step + 1) + winExpr.getFrameMode() + ' ' + winExpr.getFrameStartKind() + ' '
- + winExpr.getFrameEndKind() + " EXCLUDE " + winExpr.getFrameExclusionKind());
+ out.println(skip(step + 1) + winExpr.getFrameMode() + " between " + winExpr.getFrameStartKind() + " and "
+ + winExpr.getFrameEndKind() + " exclude " + winExpr.getFrameExclusionKind());
if (winExpr.hasFrameStartExpr()) {
winExpr.getFrameStartExpr().accept(this, step + 2);
}
diff --git a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java
index 08f5139..54c8330 100644
--- a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java
+++ b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java
@@ -1790,9 +1790,9 @@
addFunction(SCALAR_SQL_COUNT_DISTINCT, AInt64TypeComputer.INSTANCE, true);
addFunction(SUM_DISTINCT, NumericSumAggTypeComputer.INSTANCE, true);
- addFunction(SCALAR_SUM_DISTINCT, NumericSumAggTypeComputer.INSTANCE, true);
+ addFunction(SCALAR_SUM_DISTINCT, ScalarVersionOfAggregateResultType.INSTANCE, true);
addFunction(SQL_SUM_DISTINCT, NumericSumAggTypeComputer.INSTANCE, true);
- addFunction(SCALAR_SQL_SUM_DISTINCT, NumericSumAggTypeComputer.INSTANCE, true);
+ addFunction(SCALAR_SQL_SUM_DISTINCT, ScalarVersionOfAggregateResultType.INSTANCE, true);
addFunction(AVG_DISTINCT, NullableDoubleTypeComputer.INSTANCE, true);
addFunction(SCALAR_AVG_DISTINCT, NullableDoubleTypeComputer.INSTANCE, true);
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/WindowOperator.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/WindowOperator.java
index e09b471..8f563b3 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/WindowOperator.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/WindowOperator.java
@@ -68,7 +68,7 @@
private final List<Mutable<ILogicalExpression>> frameExcludeExpressions;
- private final int frameExcludeNegationStartIdx;
+ private int frameExcludeNegationStartIdx;
private final Mutable<ILogicalExpression> frameOffset;
@@ -175,6 +175,10 @@
return frameExcludeNegationStartIdx;
}
+ public void setFrameExcludeNegationStartIdx(int value) {
+ this.frameExcludeNegationStartIdx = value;
+ }
+
public Mutable<ILogicalExpression> getFrameOffset() {
return frameOffset;
}
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java
index 764fe49..a39af41 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java
@@ -624,32 +624,10 @@
return false;
}
WindowOperator windowOpArg = (WindowOperator) copyAndSubstituteVar(op, arg);
- if (!VariableUtilities.varListEqualUnordered(op.getPartitionExpressions(),
- windowOpArg.getPartitionExpressions())) {
+ if (!compareWindowPartitionSpec(op, windowOpArg)) {
return false;
}
- if (!compareIOrderAndExpressions(op.getOrderExpressions(), windowOpArg.getOrderExpressions())) {
- return false;
- }
- if (!compareIOrderAndExpressions(op.getFrameValueExpressions(), windowOpArg.getFrameValueExpressions())) {
- return false;
- }
- if (!compareExpressions(op.getFrameStartExpressions(), windowOpArg.getFrameStartExpressions())) {
- return false;
- }
- if (!compareExpressions(op.getFrameEndExpressions(), windowOpArg.getFrameEndExpressions())) {
- return false;
- }
- if (!compareExpressions(op.getFrameExcludeExpressions(), windowOpArg.getFrameExcludeExpressions())) {
- return false;
- }
- if (op.getFrameExcludeNegationStartIdx() != windowOpArg.getFrameExcludeNegationStartIdx()) {
- return false;
- }
- if (!Objects.equals(op.getFrameOffset().getValue(), windowOpArg.getFrameOffset().getValue())) {
- return false;
- }
- if (op.getFrameMaxObjects() != windowOpArg.getFrameMaxObjects()) {
+ if (!compareWindowFrameSpec(op, windowOpArg)) {
return false;
}
if (!VariableUtilities.varListEqualUnordered(getPairList(op.getVariables(), op.getExpressions()),
@@ -662,36 +640,52 @@
return isomorphic;
}
- private Boolean compareExpressions(List<Mutable<ILogicalExpression>> opExprs,
+ public static boolean compareWindowPartitionSpec(WindowOperator winOp1, WindowOperator winOp2) {
+ return VariableUtilities.varListEqualUnordered(winOp1.getPartitionExpressions(),
+ winOp2.getPartitionExpressions())
+ && compareIOrderAndExpressions(winOp1.getOrderExpressions(), winOp2.getOrderExpressions());
+ }
+
+ public static boolean compareWindowFrameSpec(WindowOperator winOp1, WindowOperator winOp2) {
+ return compareIOrderAndExpressions(winOp1.getFrameValueExpressions(), winOp2.getFrameValueExpressions())
+ && compareExpressions(winOp1.getFrameStartExpressions(), winOp2.getFrameStartExpressions())
+ && compareExpressions(winOp1.getFrameEndExpressions(), winOp2.getFrameEndExpressions())
+ && compareExpressions(winOp1.getFrameExcludeExpressions(), winOp2.getFrameExcludeExpressions())
+ && winOp1.getFrameExcludeNegationStartIdx() == winOp2.getFrameExcludeNegationStartIdx()
+ && Objects.equals(winOp1.getFrameOffset().getValue(), winOp2.getFrameOffset().getValue())
+ && winOp1.getFrameMaxObjects() == winOp2.getFrameMaxObjects();
+ }
+
+ private static boolean compareExpressions(List<Mutable<ILogicalExpression>> opExprs,
List<Mutable<ILogicalExpression>> argExprs) {
if (opExprs.size() != argExprs.size()) {
- return Boolean.FALSE;
+ return false;
}
for (int i = 0; i < opExprs.size(); i++) {
boolean isomorphic = opExprs.get(i).getValue().equals(argExprs.get(i).getValue());
if (!isomorphic) {
- return Boolean.FALSE;
+ return false;
}
}
- return Boolean.TRUE;
+ return true;
}
- private Boolean compareIOrderAndExpressions(List<Pair<IOrder, Mutable<ILogicalExpression>>> opOrderExprs,
+ private static boolean compareIOrderAndExpressions(List<Pair<IOrder, Mutable<ILogicalExpression>>> opOrderExprs,
List<Pair<IOrder, Mutable<ILogicalExpression>>> argOrderExprs) {
if (opOrderExprs.size() != argOrderExprs.size()) {
- return Boolean.FALSE;
+ return false;
}
for (int i = 0; i < opOrderExprs.size(); i++) {
boolean isomorphic = opOrderExprs.get(i).first.equals(argOrderExprs.get(i).first);
if (!isomorphic) {
- return Boolean.FALSE;
+ return false;
}
isomorphic = opOrderExprs.get(i).second.getValue().equals(argOrderExprs.get(i).second.getValue());
if (!isomorphic) {
- return Boolean.FALSE;
+ return false;
}
}
- return Boolean.TRUE;
+ return true;
}
private boolean compareSubplans(List<ILogicalPlan> plans, List<ILogicalPlan> plansArg) throws AlgebricksException {
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismVariableMappingVisitor.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismVariableMappingVisitor.java
index 5591053..eca2508 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismVariableMappingVisitor.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismVariableMappingVisitor.java
@@ -99,7 +99,7 @@
@Override
public Void visitWindowOperator(WindowOperator op, ILogicalOperator arg) throws AlgebricksException {
mapChildren(op, arg);
- mapVariablesForAbstractAssign(op, arg);
+ mapVariablesForWindow(op, arg);
mapVariablesInNestedPlans(op, arg);
return null;
}
@@ -359,7 +359,7 @@
rightOp.getExpressions());
}
- private void mapVariablesForGroupBy(ILogicalOperator left, ILogicalOperator right) throws AlgebricksException {
+ private void mapVariablesForGroupBy(ILogicalOperator left, ILogicalOperator right) {
if (left.getOperatorTag() != right.getOperatorTag()) {
return;
}
@@ -371,6 +371,16 @@
leftPairs = leftOp.getDecorList();
rightPairs = rightOp.getDecorList();
mapVarExprPairList(leftPairs, rightPairs);
+ }
+
+ private void mapVariablesForWindow(ILogicalOperator left, ILogicalOperator right) {
+ if (left.getOperatorTag() != right.getOperatorTag()) {
+ return;
+ }
+ WindowOperator leftOp = (WindowOperator) left;
+ WindowOperator rightOp = (WindowOperator) right;
+ mapVariablesForAbstractAssign(leftOp.getVariables(), leftOp.getExpressions(), rightOp.getVariables(),
+ rightOp.getExpressions());
}
private void mapVarExprPairList(List<Pair<LogicalVariable, Mutable<ILogicalExpression>>> leftPairs,
@@ -420,6 +430,9 @@
private void mapVariablesInNestedPlans(AbstractOperatorWithNestedPlans op, ILogicalOperator arg)
throws AlgebricksException {
+ if (op.getOperatorTag() != arg.getOperatorTag()) {
+ return;
+ }
AbstractOperatorWithNestedPlans argOp = (AbstractOperatorWithNestedPlans) arg;
List<ILogicalPlan> plans = op.getNestedPlans();
List<ILogicalPlan> plansArg = argOp.getNestedPlans();
diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorManipulationUtil.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorManipulationUtil.java
index 4d05a1b..967ad6f 100644
--- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorManipulationUtil.java
+++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorManipulationUtil.java
@@ -36,6 +36,7 @@
import org.apache.hyracks.algebricks.core.algebra.base.IOptimizationContext;
import org.apache.hyracks.algebricks.core.algebra.base.LogicalOperatorTag;
import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
+import org.apache.hyracks.algebricks.core.algebra.expressions.VariableReferenceExpression;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractOperatorWithNestedPlans;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AggregateOperator;
@@ -407,4 +408,25 @@
}
return clonedExprList;
}
+
+ /**
+ * Finds a variable assigned to a given expression and returns a new {@link VariableReferenceExpression}
+ * referring to this variable.
+ * @param assignVarList list of variables
+ * @param assignExprList list of expressions assigned to those variables
+ * @param searchExpr expression to search for
+ * @return said value, {@code null} if a variable is not found
+ */
+ public static VariableReferenceExpression findAssignedVariable(List<LogicalVariable> assignVarList,
+ List<Mutable<ILogicalExpression>> assignExprList, ILogicalExpression searchExpr) {
+ for (int i = 0, n = assignExprList.size(); i < n; i++) {
+ ILogicalExpression expr = assignExprList.get(i).getValue();
+ if (expr.equals(searchExpr)) {
+ VariableReferenceExpression result = new VariableReferenceExpression(assignVarList.get(i));
+ result.setSourceLocation(expr.getSourceLocation());
+ return result;
+ }
+ }
+ return null;
+ }
}
diff --git a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ConsolidateWindowOperatorsRule.java b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ConsolidateWindowOperatorsRule.java
new file mode 100644
index 0000000..61880a2
--- /dev/null
+++ b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ConsolidateWindowOperatorsRule.java
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hyracks.algebricks.rewriter.rules;
+
+import java.util.Collection;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+import org.apache.commons.lang3.mutable.Mutable;
+import org.apache.hyracks.algebricks.common.exceptions.AlgebricksException;
+import org.apache.hyracks.algebricks.core.algebra.base.ILogicalOperator;
+import org.apache.hyracks.algebricks.core.algebra.base.ILogicalPlan;
+import org.apache.hyracks.algebricks.core.algebra.base.IOptimizationContext;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalOperatorTag;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.AggregateOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.WindowOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.visitors.IsomorphismOperatorVisitor;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.visitors.IsomorphismUtilities;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.visitors.VariableUtilities;
+import org.apache.hyracks.algebricks.core.algebra.util.OperatorPropertiesUtil;
+import org.apache.hyracks.algebricks.core.rewriter.base.IAlgebraicRewriteRule;
+
+/**
+ * Merges two adjacent window operators into one if their window specifications are compatible.
+ * <pre>
+ * window [$x] <- [f()] with nested plan (aggergate [$a] <- [agg_1()] - ... - nts )
+ * window [$y] <- [g()] with nesedd plan (aggregate [$b] <- [agg_2()] - ... - nts )
+ * -->
+ * window [$x, $y] <- [f(), g()] with nested plan ( aggregate [$a, $b] <- [agg_1(), agg_2()] - ... - nts )
+ * </pre>
+ */
+public class ConsolidateWindowOperatorsRule implements IAlgebraicRewriteRule {
+
+ @Override
+ public boolean rewritePre(Mutable<ILogicalOperator> opRef, IOptimizationContext context) {
+ return false;
+ }
+
+ @Override
+ public boolean rewritePost(Mutable<ILogicalOperator> opRef, IOptimizationContext context)
+ throws AlgebricksException {
+ AbstractLogicalOperator op1 = (AbstractLogicalOperator) opRef.getValue();
+ if (op1.getOperatorTag() != LogicalOperatorTag.WINDOW) {
+ return false;
+ }
+ WindowOperator winOp1 = (WindowOperator) op1;
+
+ AbstractLogicalOperator op2 = (AbstractLogicalOperator) op1.getInputs().get(0).getValue();
+ if (op2.getOperatorTag() != LogicalOperatorTag.WINDOW) {
+ return false;
+ }
+
+ WindowOperator winOp2 = (WindowOperator) op2;
+
+ if (!IsomorphismOperatorVisitor.compareWindowPartitionSpec(winOp1, winOp2)) {
+ return false;
+ }
+ if (winOp1.hasNestedPlans() && winOp2.hasNestedPlans()
+ && !IsomorphismOperatorVisitor.compareWindowFrameSpec(winOp1, winOp2)) {
+ return false;
+ }
+
+ Set<LogicalVariable> used1 = new HashSet<>();
+ VariableUtilities.getUsedVariables(winOp1, used1);
+ if (!OperatorPropertiesUtil.disjoint(winOp2.getVariables(), used1)) {
+ return false;
+ }
+
+ if (winOp2.hasNestedPlans() && !consolidateNestedPlans(winOp1, winOp2, context)) {
+ return false;
+ }
+
+ winOp1.getVariables().addAll(winOp2.getVariables());
+ winOp1.getExpressions().addAll(winOp2.getExpressions());
+
+ winOp1.getInputs().clear();
+ winOp1.getInputs().addAll(winOp2.getInputs());
+ context.computeAndSetTypeEnvironmentForOperator(winOp1);
+
+ return true;
+ }
+
+ private boolean consolidateNestedPlans(WindowOperator winOpTo, WindowOperator winOpFrom,
+ IOptimizationContext context) throws AlgebricksException {
+ if (winOpTo.hasNestedPlans()) {
+ AggregateOperator aggTo = getAggregateRoot(winOpTo.getNestedPlans());
+ if (aggTo == null) {
+ return false;
+ }
+ AggregateOperator aggFrom = getAggregateRoot(winOpFrom.getNestedPlans());
+ if (aggFrom == null) {
+ return false;
+ }
+ if (!IsomorphismUtilities.isOperatorIsomorphicPlanSegment(aggTo.getInputs().get(0).getValue(),
+ aggFrom.getInputs().get(0).getValue())) {
+ return false;
+ }
+ aggTo.getVariables().addAll(aggFrom.getVariables());
+ aggTo.getExpressions().addAll(aggFrom.getExpressions());
+ context.computeAndSetTypeEnvironmentForOperator(aggTo);
+ } else {
+ setAll(winOpTo.getNestedPlans(), winOpFrom.getNestedPlans());
+ setAll(winOpTo.getFrameValueExpressions(), winOpFrom.getFrameValueExpressions());
+ setAll(winOpTo.getFrameStartExpressions(), winOpFrom.getFrameStartExpressions());
+ setAll(winOpTo.getFrameEndExpressions(), winOpFrom.getFrameEndExpressions());
+ setAll(winOpTo.getFrameExcludeExpressions(), winOpFrom.getFrameExcludeExpressions());
+ winOpTo.setFrameExcludeNegationStartIdx(winOpFrom.getFrameExcludeNegationStartIdx());
+ winOpTo.getFrameOffset().setValue(winOpFrom.getFrameOffset().getValue());
+ winOpTo.setFrameMaxObjects(winOpFrom.getFrameMaxObjects());
+ }
+ return true;
+ }
+
+ private AggregateOperator getAggregateRoot(List<ILogicalPlan> nestedPlans) {
+ if (nestedPlans.size() != 1) {
+ return null;
+ }
+ List<Mutable<ILogicalOperator>> roots = nestedPlans.get(0).getRoots();
+ if (roots.size() != 1) {
+ return null;
+ }
+ ILogicalOperator rootOp = roots.get(0).getValue();
+ if (rootOp.getOperatorTag() != LogicalOperatorTag.AGGREGATE) {
+ return null;
+ }
+ return (AggregateOperator) rootOp;
+ }
+
+ private <T> void setAll(Collection<? super T> to, Collection<? extends T> from) {
+ if (!to.isEmpty()) {
+ throw new IllegalStateException(String.valueOf(to.size()));
+ }
+ to.addAll(from);
+ }
+}
diff --git a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineAssignIntoAggregateRule.java b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineAssignIntoAggregateRule.java
index d0dff03..a97c38b 100644
--- a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineAssignIntoAggregateRule.java
+++ b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineAssignIntoAggregateRule.java
@@ -34,9 +34,9 @@
import org.apache.hyracks.algebricks.core.algebra.expressions.ConstantExpression;
import org.apache.hyracks.algebricks.core.algebra.expressions.VariableReferenceExpression;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractOperatorWithNestedPlans;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AggregateOperator;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AssignOperator;
-import org.apache.hyracks.algebricks.core.algebra.operators.logical.GroupByOperator;
import org.apache.hyracks.algebricks.core.algebra.visitors.AbstractConstVarFunVisitor;
import org.apache.hyracks.algebricks.core.rewriter.base.IAlgebraicRewriteRule;
@@ -51,48 +51,71 @@
public boolean rewritePost(Mutable<ILogicalOperator> opRef, IOptimizationContext context)
throws AlgebricksException {
AbstractLogicalOperator op = (AbstractLogicalOperator) opRef.getValue();
- if (op.getOperatorTag() != LogicalOperatorTag.GROUP) {
+ if (op.getOperatorTag() != LogicalOperatorTag.GROUP && op.getOperatorTag() != LogicalOperatorTag.WINDOW) {
return false;
}
boolean changed = false;
- GroupByOperator gbyOp = (GroupByOperator) op;
- for (ILogicalPlan p : gbyOp.getNestedPlans()) {
+ AbstractOperatorWithNestedPlans opWithNestedPlan = (AbstractOperatorWithNestedPlans) op;
+ for (ILogicalPlan p : opWithNestedPlan.getNestedPlans()) {
for (Mutable<ILogicalOperator> r : p.getRoots()) {
- if (inlined(r)) {
- changed = true;
- }
+ changed |= inlined(r.getValue(), opWithNestedPlan);
}
}
return changed;
}
- private boolean inlined(Mutable<ILogicalOperator> r) throws AlgebricksException {
- AbstractLogicalOperator op1 = (AbstractLogicalOperator) r.getValue();
+ private boolean inlined(ILogicalOperator planRootOp, AbstractOperatorWithNestedPlans opWithNestedPlan)
+ throws AlgebricksException {
+ AbstractLogicalOperator op1 = (AbstractLogicalOperator) planRootOp;
if (op1.getOperatorTag() != LogicalOperatorTag.AGGREGATE) {
return false;
}
- AbstractLogicalOperator op2 = (AbstractLogicalOperator) op1.getInputs().get(0).getValue();
+ AggregateOperator aggOp = (AggregateOperator) op1;
+ boolean inlined = inlineInputAssignIntoAgg(aggOp);
+ if (opWithNestedPlan.getOperatorTag() == LogicalOperatorTag.WINDOW) {
+ inlined |= inlineOuterInputAssignIntoAgg(aggOp, opWithNestedPlan);
+ }
+ return inlined;
+ }
+
+ private boolean inlineInputAssignIntoAgg(AggregateOperator aggOp) throws AlgebricksException {
+ AbstractLogicalOperator op2 = (AbstractLogicalOperator) aggOp.getInputs().get(0).getValue();
if (op2.getOperatorTag() != LogicalOperatorTag.ASSIGN) {
return false;
}
- AggregateOperator agg = (AggregateOperator) op1;
- AssignOperator assign = (AssignOperator) op2;
- VarExprSubstitution ves = new VarExprSubstitution(assign.getVariables(), assign.getExpressions());
- for (Mutable<ILogicalExpression> exprRef : agg.getExpressions()) {
- ILogicalExpression expr = exprRef.getValue();
- Pair<Boolean, ILogicalExpression> p = expr.accept(ves, null);
- if (p.first == true) {
- exprRef.setValue(p.second);
- }
- // AbstractLogicalExpression ale = (AbstractLogicalExpression) expr;
- // ale.accept(ves, null);
- }
- List<Mutable<ILogicalOperator>> op1InpList = op1.getInputs();
+ AssignOperator assignOp = (AssignOperator) op2;
+ VarExprSubstitution ves = new VarExprSubstitution(assignOp.getVariables(), assignOp.getExpressions());
+ inlineVariables(aggOp, ves);
+ List<Mutable<ILogicalOperator>> op1InpList = aggOp.getInputs();
op1InpList.clear();
op1InpList.add(op2.getInputs().get(0));
return true;
}
+ private boolean inlineOuterInputAssignIntoAgg(AggregateOperator aggOp,
+ AbstractOperatorWithNestedPlans opWithNestedPlans) throws AlgebricksException {
+ AbstractLogicalOperator op2 = (AbstractLogicalOperator) opWithNestedPlans.getInputs().get(0).getValue();
+ if (op2.getOperatorTag() != LogicalOperatorTag.ASSIGN) {
+ return false;
+ }
+ AssignOperator assignOp = (AssignOperator) op2;
+ VarExprSubstitution ves = new VarExprSubstitution(assignOp.getVariables(), assignOp.getExpressions());
+ return inlineVariables(aggOp, ves);
+ }
+
+ private boolean inlineVariables(AggregateOperator aggOp, VarExprSubstitution ves) throws AlgebricksException {
+ boolean inlined = false;
+ for (Mutable<ILogicalExpression> exprRef : aggOp.getExpressions()) {
+ ILogicalExpression expr = exprRef.getValue();
+ Pair<Boolean, ILogicalExpression> p = expr.accept(ves, null);
+ if (p.first) {
+ exprRef.setValue(p.second);
+ inlined = true;
+ }
+ }
+ return inlined;
+ }
+
private class VarExprSubstitution extends AbstractConstVarFunVisitor<Pair<Boolean, ILogicalExpression>, Void> {
private List<LogicalVariable> variables;
diff --git a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
index 5d6237a..729d6f9 100644
--- a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
+++ b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/InlineVariablesRule.java
@@ -39,8 +39,8 @@
import org.apache.hyracks.algebricks.core.algebra.expressions.VariableReferenceExpression;
import org.apache.hyracks.algebricks.core.algebra.functions.FunctionIdentifier;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractOperatorWithNestedPlans;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.AssignOperator;
-import org.apache.hyracks.algebricks.core.algebra.operators.logical.SubplanOperator;
import org.apache.hyracks.algebricks.core.algebra.operators.logical.visitors.VariableUtilities;
import org.apache.hyracks.algebricks.core.algebra.visitors.ILogicalExpressionReferenceTransform;
import org.apache.hyracks.algebricks.core.rewriter.base.IAlgebraicRewriteRule;
@@ -157,9 +157,9 @@
}
// Descend into subplan
- if (op.getOperatorTag() == LogicalOperatorTag.SUBPLAN) {
- SubplanOperator subplanOp = (SubplanOperator) op;
- for (ILogicalPlan nestedPlan : subplanOp.getNestedPlans()) {
+ if (op.getOperatorTag() == LogicalOperatorTag.SUBPLAN || op.getOperatorTag() == LogicalOperatorTag.WINDOW) {
+ List<ILogicalPlan> nestedPlans = ((AbstractOperatorWithNestedPlans) op).getNestedPlans();
+ for (ILogicalPlan nestedPlan : nestedPlans) {
for (Mutable<ILogicalOperator> root : nestedPlan.getRoots()) {
if (inlineVariables(root, context)) {
modified = true;
diff --git a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ReuseWindowAggregateRule.java b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ReuseWindowAggregateRule.java
new file mode 100644
index 0000000..59272ec
--- /dev/null
+++ b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ReuseWindowAggregateRule.java
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.hyracks.algebricks.rewriter.rules;
+
+import java.util.ArrayList;
+import java.util.Iterator;
+import java.util.List;
+
+import org.apache.commons.lang3.mutable.Mutable;
+import org.apache.commons.lang3.mutable.MutableObject;
+import org.apache.hyracks.algebricks.common.exceptions.AlgebricksException;
+import org.apache.hyracks.algebricks.core.algebra.base.ILogicalExpression;
+import org.apache.hyracks.algebricks.core.algebra.base.ILogicalOperator;
+import org.apache.hyracks.algebricks.core.algebra.base.IOptimizationContext;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalOperatorTag;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
+import org.apache.hyracks.algebricks.core.algebra.expressions.VariableReferenceExpression;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.AssignOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.WindowOperator;
+import org.apache.hyracks.algebricks.core.algebra.operators.logical.visitors.IsomorphismOperatorVisitor;
+import org.apache.hyracks.algebricks.core.algebra.util.OperatorManipulationUtil;
+import org.apache.hyracks.algebricks.core.rewriter.base.IAlgebraicRewriteRule;
+
+/**
+ * If two adjacent window operators compute the same running aggregate then replace the second computation with
+ * the assign operator referring to the first operator's output variable:
+ *
+ * <pre>
+ * window [$x] <- [f()] ...
+ * window [$y] <- [f()] ...
+ * -->
+ * assign [$x] <- [$y]
+ * window [] <- [] ...
+ * window [$y] <- [f()] ...
+ * </pre>
+ *
+ * Both window operators must have the same partitioning specification.
+ *
+ * This rule must be followed by {@link RemoveRedundantVariablesRule} to substitute {@code $x} references with
+ * {@code $y} and then {@link RemoveUnusedAssignAndAggregateRule} to eliminate the new assign operator.
+ */
+public class ReuseWindowAggregateRule implements IAlgebraicRewriteRule {
+ @Override
+ public boolean rewritePre(Mutable<ILogicalOperator> opRef, IOptimizationContext context) {
+ return false;
+ }
+
+ @Override
+ public boolean rewritePost(Mutable<ILogicalOperator> opRef, IOptimizationContext context)
+ throws AlgebricksException {
+ AbstractLogicalOperator op1 = (AbstractLogicalOperator) opRef.getValue();
+ if (op1.getOperatorTag() != LogicalOperatorTag.WINDOW) {
+ return false;
+ }
+ WindowOperator winOp1 = (WindowOperator) op1;
+
+ AbstractLogicalOperator op2 = (AbstractLogicalOperator) op1.getInputs().get(0).getValue();
+ if (op2.getOperatorTag() != LogicalOperatorTag.WINDOW) {
+ return false;
+ }
+
+ WindowOperator winOp2 = (WindowOperator) op2;
+
+ if (!IsomorphismOperatorVisitor.compareWindowPartitionSpec(winOp1, winOp2)) {
+ return false;
+ }
+
+ List<LogicalVariable> assignVars = new ArrayList<>();
+ List<Mutable<ILogicalExpression>> assignExprs = new ArrayList<>();
+
+ List<LogicalVariable> varsOp1 = winOp1.getVariables();
+ List<LogicalVariable> varsOp2 = winOp2.getVariables();
+ List<Mutable<ILogicalExpression>> exprsOp1 = winOp1.getExpressions();
+ List<Mutable<ILogicalExpression>> exprsOp2 = winOp2.getExpressions();
+ Iterator<LogicalVariable> varsOp1Iter = varsOp1.iterator();
+ Iterator<Mutable<ILogicalExpression>> exprsOp1Iter = exprsOp1.iterator();
+ while (varsOp1Iter.hasNext()) {
+ LogicalVariable varOp1 = varsOp1Iter.next();
+ Mutable<ILogicalExpression> exprOp1 = exprsOp1Iter.next();
+ VariableReferenceExpression varOp2Ref =
+ OperatorManipulationUtil.findAssignedVariable(varsOp2, exprsOp2, exprOp1.getValue());
+ if (varOp2Ref != null) {
+ varsOp1Iter.remove();
+ exprsOp1Iter.remove();
+ assignVars.add(varOp1);
+ assignExprs.add(new MutableObject<>(varOp2Ref));
+ }
+ }
+
+ if (assignVars.isEmpty()) {
+ return false;
+ }
+
+ AssignOperator assignOp = new AssignOperator(assignVars, assignExprs);
+ assignOp.getInputs().add(new MutableObject<>(winOp1));
+ assignOp.setSourceLocation(winOp1.getSourceLocation());
+ context.computeAndSetTypeEnvironmentForOperator(assignOp);
+ opRef.setValue(assignOp);
+ return true;
+ }
+}
\ No newline at end of file
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/10408/ (3/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/8877/ (3/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-ensure-ancestor/2936/ (10/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/5310/ (4/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Ali Alsuliman (Code Review)" <do...@asterixdb.incubator.apache.org>.
Ali Alsuliman has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Ali Alsuliman <al...@gmail.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-sql-execution/4934/ (13/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2505][COMP] Consolidate window operators rule
Posted by "Jenkins (Code Review)" <do...@asterixdb.incubator.apache.org>.
Jenkins has posted comments on this change.
Change subject: [ASTERIXDB-2505][COMP] Consolidate window operators rule
......................................................................
Patch Set 1:
Build Started https://asterix-jenkins.ics.uci.edu/job/asterix-verify-txnlog/92/ (7/14)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3107
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d0739d604d09f5445237d4fa3d56a2af8126bd7
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Dmitry Lychagin <dm...@couchbase.com>
Gerrit-Reviewer: Jenkins <je...@fulliautomatix.ics.uci.edu>
Gerrit-HasComments: No