You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@calcite.apache.org by GitBox <gi...@apache.org> on 2022/06/27 16:52:01 UTC
[GitHub] [calcite] beyond1920 opened a new pull request, #2844: [CALCITE-4865] Extends Table function to support Polymorphic Table Function (PTF)
beyond1920 opened a new pull request, #2844:
URL: https://github.com/apache/calcite/pull/2844
## What is the purpose of the change
This pr aims to extend table function to support Polymorphic Table Function.
## Brief change log
- Update `Parser.jj` to support partition by clause and order by clause for input table with set semantics of PTF
- Introduce `TableCharacteristics` which contains three characteristics of input table of table function
- Update `SqlTableFunction` to add a method `tableCharacteristics`, the method returns the table characteristics for the ordinal-th argument to this table function. Default return value is Optional.empty which means the ordinal-th argument is not table.
- Introduce `SqlSetSemanticsTable` which represents input table with set semantics of Table Function, its `SqlKind` is `SET_SEMANTICS_TABLE`
- Updates `SqlValidatorImpl` to validate only set semantic table of Table Function could have partition by and order by clause
- Update `SqlToRelConverter#substituteSubQuery` to parse subQuery which represents set semantics table.
## Verifying this change
- Add tests in `SqlToRelConveterTest`, `SqlValidatorTest`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r912957334
##########
core/src/main/codegen/templates/Parser.jj:
##########
@@ -1653,6 +1657,108 @@ SqlNode NamedRoutineCall(
}
}
+/**
+ * Table parameter of a table function.
+ * The input table with set semantics may be partitioned/ordered on one or more columns.
+ */
+SqlNode TableArg() :
+{
Review Comment:
TableArg -> TableParam
##########
core/src/main/codegen/templates/Parser.jj:
##########
@@ -1653,6 +1657,108 @@ SqlNode NamedRoutineCall(
}
}
+/**
+ * Table parameter of a table function.
+ * The input table with set semantics may be partitioned/ordered on one or more columns.
+ */
+SqlNode TableArg() :
+{
+ final Span s;
+ SqlNode tableRef;
+ SqlNodeList partitionList = SqlNodeList.EMPTY;
+ SqlNodeList orderList = SqlNodeList.EMPTY;
+}
+{
+ { s = span(); }
+ tableRef = ExplicitTable(getPos())
+ [
+ <PARTITION> <BY>
+ partitionList = SimpleIdentifierOrList()
+ ]
+ [
+ orderList = OrderByInTableParam()
+ ]
+ {
+ if (partitionList.isEmpty() && orderList.isEmpty()) {
+ return tableRef;
+ } else {
+ return SqlStdOperatorTable.SET_SEMANTICS_TABLE.createCall(
+ s.pos(), tableRef, partitionList, orderList);
+ }
+ }
+}
+
+SqlNode PartitionedQueryOrQueryOrExpr(ExprContext exprContext) :
+{
+ SqlNode e;
+}
+{
+ e = OrderedQueryOrExpr(exprContext)
+ e = PartitionedByAndOrderBy(e)
+
+ { return e; }
+}
+
+SqlNode PartitionedByAndOrderBy(SqlNode e):
+{
+ final Span s;
+ SqlNodeList partitionList = SqlNodeList.EMPTY;
+ SqlNodeList orderList = SqlNodeList.EMPTY;
+}
Review Comment:
The over window also has partition by order by clause, can that parse block be reused ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
beyond1920 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913086886
##########
core/src/main/codegen/templates/Parser.jj:
##########
@@ -1653,6 +1657,108 @@ SqlNode NamedRoutineCall(
}
}
+/**
+ * Table parameter of a table function.
+ * The input table with set semantics may be partitioned/ordered on one or more columns.
+ */
+SqlNode TableArg() :
+{
+ final Span s;
+ SqlNode tableRef;
+ SqlNodeList partitionList = SqlNodeList.EMPTY;
+ SqlNodeList orderList = SqlNodeList.EMPTY;
+}
+{
+ { s = span(); }
+ tableRef = ExplicitTable(getPos())
+ [
+ <PARTITION> <BY>
+ partitionList = SimpleIdentifierOrList()
+ ]
+ [
+ orderList = OrderByInTableParam()
+ ]
+ {
+ if (partitionList.isEmpty() && orderList.isEmpty()) {
+ return tableRef;
+ } else {
+ return SqlStdOperatorTable.SET_SEMANTICS_TABLE.createCall(
+ s.pos(), tableRef, partitionList, orderList);
+ }
+ }
+}
+
+SqlNode PartitionedQueryOrQueryOrExpr(ExprContext exprContext) :
+{
+ SqlNode e;
+}
+{
+ e = OrderedQueryOrExpr(exprContext)
+ e = PartitionedByAndOrderBy(e)
+
+ { return e; }
+}
+
+SqlNode PartitionedByAndOrderBy(SqlNode e):
+{
+ final Span s;
+ SqlNodeList partitionList = SqlNodeList.EMPTY;
+ SqlNodeList orderList = SqlNodeList.EMPTY;
+}
Review Comment:
The code could not be reused here because partition by clause and order by clause are different from the partition by clause and order by clause in some other contexts.
Test cases could cover these cases.
Based on the SQL standard, Order by clause of Polymorphic table function is different from other order by clause.
The list of order keys must be parenthesized if more than one column is listed.
Similarly, PARTITION BY clause with a list of zero or more columns. The list can always be enclosed in parentheses. If there is only one partitioning column, then the parentheses are optional.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
beyond1920 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r915649928
##########
core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java:
##########
@@ -1371,13 +1373,153 @@ private void substituteSubQuery(Blackboard bb, SubQuery subQuery) {
//
bb.cursors.add(converted.r);
return;
-
+ case SET_SEMANTICS_TABLE:
+ if (!config.isExpand()) {
+ return;
+ }
+ substituteSubQueryOfSetSemanticsInputTable(bb, subQuery);
+ return;
default:
throw new AssertionError("unexpected kind of sub-query: "
+ subQuery.node);
}
}
+ private void substituteSubQueryOfSetSemanticsInputTable(
+ Blackboard bb,
+ SubQuery subQuery) {
+ SqlBasicCall call;
+ SqlNode query;
+ call = (SqlBasicCall) subQuery.node;
+ query = call.operand(0);
+ final SqlValidatorScope innerTableScope =
+ (query instanceof SqlSelect)
+ ? validator().getSelectScope((SqlSelect) query)
+ : null;
+ final Blackboard setSemanticsTableBb = createBlackboard(
+ innerTableScope, null, false);
+ final RelNode inputOfSetSemanticsTable = convertQueryRecursive(
Review Comment:
Scope is not null here, it's second argument (nameToNodeMap) is null.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913741092
##########
core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java:
##########
@@ -3380,6 +3384,66 @@ protected void validateFrom(
getNamespaceOrThrow(node, scope).validate(targetRowType);
}
+ protected void validateTableFunction(SqlCall node, SqlValidatorScope scope,
+ RelDataType targetRowType) {
+ // Dig out real call; TABLE() wrapper is just syntactic.
+ SqlCall call = node.operand(0);
+ if (call.getOperator() instanceof SqlTableFunction) {
+ SqlTableFunction tableFunction = (SqlTableFunction) call.getOperator();
+ boolean visitedRowSemanticsTable = false;
+ for (int idx = 0; idx < call.operandCount(); idx++) {
+ TableCharacteristic tableCharacteristic = tableFunction.tableCharacteristic(idx);
+ if (tableCharacteristic != null) {
+ // Skip validate if current input table has set semantics
+ if (tableCharacteristic.semantics == TableCharacteristic.Semantics.SET) {
+ continue;
+ }
+ // A table function at most has one input table with row semantics
+ if (visitedRowSemanticsTable) {
+ throw newValidationError(
+ call,
+ RESOURCE.multipleRowSemanticsTables(call.getOperator().getName()));
+ }
+ visitedRowSemanticsTable = true;
+ }
+ // If table function defines the parameter is not table parameter, or is an input table
+ // parameter with row semantics, then it should not be with PARTITION BY OR ORDER BY.
+ SqlNode currentNode = call.operand(idx);
+ if (currentNode instanceof SqlCall) {
+ final SqlOperator op = ((SqlCall) currentNode).getOperator();
+ if (op == SqlStdOperatorTable.SET_SEMANTICS_TABLE) {
+ throwInvalidRowSemanticsTable(call, idx, (SqlCall) currentNode);
+ } else if (op == SqlStdOperatorTable.ARGUMENT_ASSIGNMENT) {
+ // Dig out the underlying operand
+ SqlNode realNode = ((SqlBasicCall) currentNode).operand(0);
+ if (realNode instanceof SqlCall) {
Review Comment:
Can move the digging in the first so that the validation logic only need one action.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913752126
##########
core/src/test/java/org/apache/calcite/test/SqlValidatorTest.java:
##########
@@ -1564,6 +1564,50 @@ public void _testLikeAndSimilarFails() {
.fails("No match found for function signature FOO..");
}
+ @Test void testInvalidTableFunction() {
+ // A table function at most have one input table with row semantics
+ sql("select * from table(^invalid(table orders, table emp)^)")
+ .fails("A table function at most has one input table with row semantics."
+ + " Table function 'INVALID' has multiple input tables with row semantics");
+ // Only tables with set semantics may be partitioned
+ sql("select * from table(^score(table orders partition by productid)^)")
+ .fails("Only tables with set semantics may be partitioned."
+ + " Invalid PARTITION BY clause in the 0-th operand of table function 'SCORE'");
+ // Only tables with set semantics may be ordered
Review Comment:
We need some tests for multiple partition/ordering keys.
We need some tests for multiple table params.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 commented on pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
beyond1920 commented on PR #2844:
URL: https://github.com/apache/calcite/pull/2844#issuecomment-1188526060
Merging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913747288
##########
core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java:
##########
@@ -1371,13 +1373,153 @@ private void substituteSubQuery(Blackboard bb, SubQuery subQuery) {
//
bb.cursors.add(converted.r);
return;
-
+ case SET_SEMANTICS_TABLE:
+ if (!config.isExpand()) {
+ return;
+ }
+ substituteSubQueryOfSetSemanticsInputTable(bb, subQuery);
+ return;
default:
throw new AssertionError("unexpected kind of sub-query: "
+ subQuery.node);
}
}
+ private void substituteSubQueryOfSetSemanticsInputTable(
+ Blackboard bb,
+ SubQuery subQuery) {
+ SqlBasicCall call;
+ SqlNode query;
+ call = (SqlBasicCall) subQuery.node;
+ query = call.operand(0);
+ final SqlValidatorScope innerTableScope =
+ (query instanceof SqlSelect)
+ ? validator().getSelectScope((SqlSelect) query)
+ : null;
+ final Blackboard setSemanticsTableBb = createBlackboard(
+ innerTableScope, null, false);
+ final RelNode inputOfSetSemanticsTable = convertQueryRecursive(
Review Comment:
What is the semantics to create black board with null scope ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913741092
##########
core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java:
##########
@@ -3380,6 +3384,66 @@ protected void validateFrom(
getNamespaceOrThrow(node, scope).validate(targetRowType);
}
+ protected void validateTableFunction(SqlCall node, SqlValidatorScope scope,
+ RelDataType targetRowType) {
+ // Dig out real call; TABLE() wrapper is just syntactic.
+ SqlCall call = node.operand(0);
+ if (call.getOperator() instanceof SqlTableFunction) {
+ SqlTableFunction tableFunction = (SqlTableFunction) call.getOperator();
+ boolean visitedRowSemanticsTable = false;
+ for (int idx = 0; idx < call.operandCount(); idx++) {
+ TableCharacteristic tableCharacteristic = tableFunction.tableCharacteristic(idx);
+ if (tableCharacteristic != null) {
+ // Skip validate if current input table has set semantics
+ if (tableCharacteristic.semantics == TableCharacteristic.Semantics.SET) {
+ continue;
+ }
+ // A table function at most has one input table with row semantics
+ if (visitedRowSemanticsTable) {
+ throw newValidationError(
+ call,
+ RESOURCE.multipleRowSemanticsTables(call.getOperator().getName()));
+ }
+ visitedRowSemanticsTable = true;
+ }
+ // If table function defines the parameter is not table parameter, or is an input table
+ // parameter with row semantics, then it should not be with PARTITION BY OR ORDER BY.
+ SqlNode currentNode = call.operand(idx);
+ if (currentNode instanceof SqlCall) {
+ final SqlOperator op = ((SqlCall) currentNode).getOperator();
+ if (op == SqlStdOperatorTable.SET_SEMANTICS_TABLE) {
+ throwInvalidRowSemanticsTable(call, idx, (SqlCall) currentNode);
+ } else if (op == SqlStdOperatorTable.ARGUMENT_ASSIGNMENT) {
+ // Dig out the underlying operand
+ SqlNode realNode = ((SqlBasicCall) currentNode).operand(0);
+ if (realNode instanceof SqlCall) {
Review Comment:
Can move the digging in the first so that the validation logic only need on.
##########
core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java:
##########
@@ -3380,6 +3384,66 @@ protected void validateFrom(
getNamespaceOrThrow(node, scope).validate(targetRowType);
}
+ protected void validateTableFunction(SqlCall node, SqlValidatorScope scope,
+ RelDataType targetRowType) {
+ // Dig out real call; TABLE() wrapper is just syntactic.
+ SqlCall call = node.operand(0);
+ if (call.getOperator() instanceof SqlTableFunction) {
+ SqlTableFunction tableFunction = (SqlTableFunction) call.getOperator();
+ boolean visitedRowSemanticsTable = false;
+ for (int idx = 0; idx < call.operandCount(); idx++) {
+ TableCharacteristic tableCharacteristic = tableFunction.tableCharacteristic(idx);
+ if (tableCharacteristic != null) {
+ // Skip validate if current input table has set semantics
+ if (tableCharacteristic.semantics == TableCharacteristic.Semantics.SET) {
+ continue;
+ }
+ // A table function at most has one input table with row semantics
+ if (visitedRowSemanticsTable) {
+ throw newValidationError(
+ call,
+ RESOURCE.multipleRowSemanticsTables(call.getOperator().getName()));
+ }
+ visitedRowSemanticsTable = true;
+ }
+ // If table function defines the parameter is not table parameter, or is an input table
+ // parameter with row semantics, then it should not be with PARTITION BY OR ORDER BY.
+ SqlNode currentNode = call.operand(idx);
+ if (currentNode instanceof SqlCall) {
+ final SqlOperator op = ((SqlCall) currentNode).getOperator();
+ if (op == SqlStdOperatorTable.SET_SEMANTICS_TABLE) {
+ throwInvalidRowSemanticsTable(call, idx, (SqlCall) currentNode);
+ } else if (op == SqlStdOperatorTable.ARGUMENT_ASSIGNMENT) {
+ // Dig out the underlying operand
+ SqlNode realNode = ((SqlBasicCall) currentNode).operand(0);
+ if (realNode instanceof SqlCall) {
Review Comment:
Can move the digging in the first so that the validation logic only need one place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913732009
##########
core/src/main/java/org/apache/calcite/sql/SqlTableFunction.java:
##########
@@ -30,4 +32,17 @@ public interface SqlTableFunction {
* @return strategy to infer the row type of a call to this function
*/
SqlReturnTypeInference getRowTypeInference();
+
+ /**
+ * Returns the table parameter characteristics for <code>ordinal</code>th
+ * parameter to this table function.
+ *
+ * <p>Returns <code>null</code> if the <code>ordinal</code>th argument is
+ * not table parameter or the <code>ordinal</code> is smaller than 0 or
+ * the <code>ordinal</code> is greater than or equals to the number of
+ * parameters.
+ */
+ default @Nullable TableCharacteristic tableCharacteristic(int ordinal) {
+ return null;
Review Comment:
Seems the functionality is only supported by window function ? Then moving this interface to `SqlWindowTableFunction` is more proper way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913726617
##########
core/src/main/java/org/apache/calcite/sql/SqlSetSemanticsTableOperator.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql;
+
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.sql.parser.SqlParserPos;
+import org.apache.calcite.sql.validate.SqlValidator;
+import org.apache.calcite.sql.validate.SqlValidatorScope;
+
+import org.checkerframework.checker.nullness.qual.Nullable;
+
+import java.util.List;
+
+/**
+ * SetSemanticsTable appears as an parameter in a table function.
+ * It represents as an input table with set semantics.
+ * Set semantics means that the outcome of the function depends on how
+ * the data is partitioned.
+ * When the table function is called from a query, the table parameter can
+ * optionally be extended with either a PARTITION BY clause or
+ * an ORDER BY clause or both.
+ *
+ */
+public class SqlSetSemanticsTableOperator extends SqlInternalOperator {
+
+ //~ Constructors -----------------------------------------------------------
+
+ public SqlSetSemanticsTableOperator() {
+ super("SET_SEMANTICS_TABLE", SqlKind.SET_SEMANTICS_TABLE);
+ }
+
+ @Override public SqlCall createCall(
+ @Nullable SqlLiteral functionQualifier,
+ SqlParserPos pos,
+ @Nullable SqlNode... operands) {
+ assert operands.length == 3;
+ SqlNode partitionList = operands[1];
+ SqlNode orderList = operands[2];
+ assert (partitionList != null && !SqlNodeList.isEmptyList(partitionList))
+ || (orderList != null && !SqlNodeList.isEmptyList(orderList));
+ return super.createCall(functionQualifier, pos, operands);
+ }
+
+ @Override public void unparse(
+ SqlWriter writer,
+ SqlCall call,
+ int leftPrec,
+ int rightPrec) {
+ call.operand(0).unparse(writer, 0, 0);
+
+ SqlNodeList partitionList = call.operand(1);
+ if (partitionList.size() > 0) {
+ writer.sep("PARTITION BY");
+ final SqlWriter.Frame partitionFrame = writer.startList("", "");
+ partitionList.unparse(writer, 0, 0);
+ writer.endList(partitionFrame);
+ }
+ SqlNodeList orderList = call.operand(2);
+ if (orderList.size() > 0) {
+ writer.sep("ORDER BY");
+ writer.list(
+ SqlWriter.FrameTypeEnum.ORDER_BY_LIST,
+ SqlWriter.COMMA,
+ orderList);
+ }
+ }
+
+ @Override public RelDataType deriveType(
+ SqlValidator validator,
+ SqlValidatorScope scope, SqlCall call) {
+ final List<SqlNode> operands = call.getOperandList();
+ assert operands.size() == 3;
+ RelDataType tableType = validator.deriveType(scope, operands.get(0));
Review Comment:
We already assert the operands number on construction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913749876
##########
core/src/main/java/org/apache/calcite/sql2rel/StandardConvertletTable.java:
##########
@@ -958,7 +960,17 @@ private static List<Integer> elseArgs(int count) {
private static List<RexNode> convertOperands(SqlRexContext cx,
SqlCall call, SqlOperandTypeChecker.Consistency consistency) {
- return convertOperands(cx, call, call.getOperandList(), consistency);
+ List<SqlNode> operandList;
+ if (call.getOperator() instanceof SqlTableFunction) {
+ // skip set semantic table node of table function
Review Comment:
I see too many instance of for `SqlTableFunction`, can it has its own SqlKind ? Or can we use the COLLECTION_TABLE as its sql kind ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
beyond1920 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r915648465
##########
core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java:
##########
@@ -3380,6 +3384,66 @@ protected void validateFrom(
getNamespaceOrThrow(node, scope).validate(targetRowType);
}
+ protected void validateTableFunction(SqlCall node, SqlValidatorScope scope,
+ RelDataType targetRowType) {
+ // Dig out real call; TABLE() wrapper is just syntactic.
+ SqlCall call = node.operand(0);
+ if (call.getOperator() instanceof SqlTableFunction) {
+ SqlTableFunction tableFunction = (SqlTableFunction) call.getOperator();
Review Comment:
Currently, there is no valid case like this.
It requires argument of COLLECTION_TABLE must be a table function which is already checked by ProcedureNamespace#validateImpl.
SqlValidatorTest#testCollectionTable cover these cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
beyond1920 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913082995
##########
core/src/main/codegen/templates/Parser.jj:
##########
@@ -1653,6 +1657,108 @@ SqlNode NamedRoutineCall(
}
}
+/**
+ * Table parameter of a table function.
+ * The input table with set semantics may be partitioned/ordered on one or more columns.
+ */
+SqlNode TableArg() :
+{
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
beyond1920 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r915658421
##########
core/src/main/java/org/apache/calcite/sql2rel/StandardConvertletTable.java:
##########
@@ -958,7 +960,17 @@ private static List<Integer> elseArgs(int count) {
private static List<RexNode> convertOperands(SqlRexContext cx,
SqlCall call, SqlOperandTypeChecker.Consistency consistency) {
- return convertOperands(cx, call, call.getOperandList(), consistency);
+ List<SqlNode> operandList;
+ if (call.getOperator() instanceof SqlTableFunction) {
+ // skip set semantic table node of table function
Review Comment:
> I see too many instance of for SqlTableFunction, can it has its own SqlKind
I think table function has its own SqlKind is a good idea.
However, it is not a compatible change between versions because the SqlKind of existed table functions defined in each implementation instead of a unified SqlKind.
##########
core/src/main/java/org/apache/calcite/sql2rel/StandardConvertletTable.java:
##########
@@ -958,7 +960,17 @@ private static List<Integer> elseArgs(int count) {
private static List<RexNode> convertOperands(SqlRexContext cx,
SqlCall call, SqlOperandTypeChecker.Consistency consistency) {
- return convertOperands(cx, call, call.getOperandList(), consistency);
+ List<SqlNode> operandList;
+ if (call.getOperator() instanceof SqlTableFunction) {
+ // skip set semantic table node of table function
Review Comment:
> I see too many instance of for SqlTableFunction, can it has its own SqlKind
I think table function has its own SqlKind is a good idea.
However, it is not a compatible change between versions because the SqlKind of existed table functions defined in each implementation instead of a unified SqlKind.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
beyond1920 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r915788510
##########
core/src/test/java/org/apache/calcite/test/SqlValidatorTest.java:
##########
@@ -1564,6 +1564,50 @@ public void _testLikeAndSimilarFails() {
.fails("No match found for function signature FOO..");
}
+ @Test void testInvalidTableFunction() {
+ // A table function at most have one input table with row semantics
+ sql("select * from table(^invalid(table orders, table emp)^)")
+ .fails("A table function at most has one input table with row semantics."
+ + " Table function 'INVALID' has multiple input tables with row semantics");
+ // Only tables with set semantics may be partitioned
+ sql("select * from table(^score(table orders partition by productid)^)")
+ .fails("Only tables with set semantics may be partitioned."
+ + " Invalid PARTITION BY clause in the 0-th operand of table function 'SCORE'");
+ // Only tables with set semantics may be ordered
Review Comment:
> We need some tests for multiple partition/ordering keys.
There are already exist cases to cover these in SqlValidatorTest#testTableFunctionWithTableParam.
> We need some tests for multiple table params.
Based on SQL standard, a table function at most has one input table with row semantics, there are already exist cases to cover these based on MockSqlOperatorTable#InvalidTableFunction.
I add some cases to cover a table function with multiple input tables with set semantics based on MockSqlOperatorTable#SimilarlityTableFunction.
##########
testkit/src/main/java/org/apache/calcite/sql/parser/SqlParserTest.java:
##########
@@ -4164,6 +4164,77 @@ void checkPeriodPredicate(Checker checker) {
sql(sql).ok(expected);
}
+ @Test void testTableFunction() {
+ final String sql = "select * from table(score(table orders))";
+ final String expected = "SELECT *\n"
+ + "FROM TABLE(`SCORE`((TABLE `ORDERS`)))";
+ sql(sql).ok(expected);
+ }
+
+ @Test void testTableFunctionWithPartitionKey() {
+ // test one partition key for input table
+ final String sql = "select * from table(topn(table orders partition by productid, 3))";
+ final String expected = "SELECT *\n"
+ + "FROM TABLE(`TOPN`(((TABLE `ORDERS`) PARTITION BY `PRODUCTID`), 3))";
+ sql(sql).ok(expected);
+ }
+
+ @Test void testTableFunctionWithMultiplePartitionKeys() {
+ // test multiple partition keys for input table
+ final String sql =
+ "select * from table(topn(table orders partition by (orderId, productid), 3))";
+ final String expected = "SELECT *\n"
+ + "FROM TABLE(`TOPN`(((TABLE `ORDERS`) PARTITION BY `ORDERID`, `PRODUCTID`), 3))";
+ sql(sql).ok(expected);
+ }
Review Comment:
> We need some tests for multiple partition/ordering keys.
There are already exist cases to cover these, for example, SqlParserTest#testTableFunctionWithMultiplePartitionKeys, SqlParserTest#testTableFunctionWithMultipleOrderKeys
> We need some tests for multiple table params.
Based on SQL standard, a table function at most has one input table with row semantics, there are already exist cases to cover these based on MockSqlOperatorTable#InvalidTableFunction.
I add some cases to cover a table function with multiple input tables with set semantics based on MockSqlOperatorTable#SimilarlityTableFunction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913751422
##########
testkit/src/main/java/org/apache/calcite/sql/parser/SqlParserTest.java:
##########
@@ -4164,6 +4164,77 @@ void checkPeriodPredicate(Checker checker) {
sql(sql).ok(expected);
}
+ @Test void testTableFunction() {
+ final String sql = "select * from table(score(table orders))";
+ final String expected = "SELECT *\n"
+ + "FROM TABLE(`SCORE`((TABLE `ORDERS`)))";
+ sql(sql).ok(expected);
+ }
+
+ @Test void testTableFunctionWithPartitionKey() {
+ // test one partition key for input table
+ final String sql = "select * from table(topn(table orders partition by productid, 3))";
+ final String expected = "SELECT *\n"
+ + "FROM TABLE(`TOPN`(((TABLE `ORDERS`) PARTITION BY `PRODUCTID`), 3))";
+ sql(sql).ok(expected);
+ }
+
+ @Test void testTableFunctionWithMultiplePartitionKeys() {
+ // test multiple partition keys for input table
+ final String sql =
+ "select * from table(topn(table orders partition by (orderId, productid), 3))";
+ final String expected = "SELECT *\n"
+ + "FROM TABLE(`TOPN`(((TABLE `ORDERS`) PARTITION BY `ORDERID`, `PRODUCTID`), 3))";
+ sql(sql).ok(expected);
+ }
Review Comment:
We need some tests for multiple partition/ordering keys.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913742664
##########
core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java:
##########
@@ -3380,6 +3384,66 @@ protected void validateFrom(
getNamespaceOrThrow(node, scope).validate(targetRowType);
}
+ protected void validateTableFunction(SqlCall node, SqlValidatorScope scope,
+ RelDataType targetRowType) {
+ // Dig out real call; TABLE() wrapper is just syntactic.
+ SqlCall call = node.operand(0);
+ if (call.getOperator() instanceof SqlTableFunction) {
+ SqlTableFunction tableFunction = (SqlTableFunction) call.getOperator();
Review Comment:
What kind of function it is if the SqlKind is `COLLECTION_TABLE` but not a `SqlTableFunction` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
beyond1920 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r917343155
##########
core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java:
##########
@@ -3380,6 +3384,66 @@ protected void validateFrom(
getNamespaceOrThrow(node, scope).validate(targetRowType);
}
+ protected void validateTableFunction(SqlCall node, SqlValidatorScope scope,
+ RelDataType targetRowType) {
+ // Dig out real call; TABLE() wrapper is just syntactic.
+ SqlCall call = node.operand(0);
+ if (call.getOperator() instanceof SqlTableFunction) {
+ SqlTableFunction tableFunction = (SqlTableFunction) call.getOperator();
Review Comment:
I prefer to keep `instance of` here.
because `ProcedureNamespace#validateImpl` do a validation after this step.
If I remove `instance of` here, the exception would thrown an exception right here when handling illegal cases like `SqlValidatorTest#testCollectionTable`.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 closed pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
beyond1920 closed pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
URL: https://github.com/apache/calcite/pull/2844
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r917319093
##########
core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java:
##########
@@ -3380,6 +3384,66 @@ protected void validateFrom(
getNamespaceOrThrow(node, scope).validate(targetRowType);
}
+ protected void validateTableFunction(SqlCall node, SqlValidatorScope scope,
+ RelDataType targetRowType) {
+ // Dig out real call; TABLE() wrapper is just syntactic.
+ SqlCall call = node.operand(0);
+ if (call.getOperator() instanceof SqlTableFunction) {
+ SqlTableFunction tableFunction = (SqlTableFunction) call.getOperator();
Review Comment:
So the `instance of ` is not necessary here ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
beyond1920 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r917343155
##########
core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java:
##########
@@ -3380,6 +3384,66 @@ protected void validateFrom(
getNamespaceOrThrow(node, scope).validate(targetRowType);
}
+ protected void validateTableFunction(SqlCall node, SqlValidatorScope scope,
+ RelDataType targetRowType) {
+ // Dig out real call; TABLE() wrapper is just syntactic.
+ SqlCall call = node.operand(0);
+ if (call.getOperator() instanceof SqlTableFunction) {
+ SqlTableFunction tableFunction = (SqlTableFunction) call.getOperator();
Review Comment:
I prefer to keep `instance of` here.
because `ProcedureNamespace#validateImpl` do a validation after this step.
If I remove `instance of` here, I need to add the `else` branch here to handle illegal cases.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 commented on pull request #2844: [CALCITE-4865] Extends Table function to support Polymorphic Table Function (PTF)
Posted by GitBox <gi...@apache.org>.
beyond1920 commented on PR #2844:
URL: https://github.com/apache/calcite/pull/2844#issuecomment-1167612779
This pr is a follow up the [pr#2606](https://github.com/apache/calcite/pull/2606).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913751422
##########
testkit/src/main/java/org/apache/calcite/sql/parser/SqlParserTest.java:
##########
@@ -4164,6 +4164,77 @@ void checkPeriodPredicate(Checker checker) {
sql(sql).ok(expected);
}
+ @Test void testTableFunction() {
+ final String sql = "select * from table(score(table orders))";
+ final String expected = "SELECT *\n"
+ + "FROM TABLE(`SCORE`((TABLE `ORDERS`)))";
+ sql(sql).ok(expected);
+ }
+
+ @Test void testTableFunctionWithPartitionKey() {
+ // test one partition key for input table
+ final String sql = "select * from table(topn(table orders partition by productid, 3))";
+ final String expected = "SELECT *\n"
+ + "FROM TABLE(`TOPN`(((TABLE `ORDERS`) PARTITION BY `PRODUCTID`), 3))";
+ sql(sql).ok(expected);
+ }
+
+ @Test void testTableFunctionWithMultiplePartitionKeys() {
+ // test multiple partition keys for input table
+ final String sql =
+ "select * from table(topn(table orders partition by (orderId, productid), 3))";
+ final String expected = "SELECT *\n"
+ + "FROM TABLE(`TOPN`(((TABLE `ORDERS`) PARTITION BY `ORDERID`, `PRODUCTID`), 3))";
+ sql(sql).ok(expected);
+ }
Review Comment:
We need some tests for multiple partition/ordering keys.
We need some tests for multiple table params.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 commented on pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
beyond1920 commented on PR #2844:
URL: https://github.com/apache/calcite/pull/2844#issuecomment-1186889579
Hi, @danny0405 @julianhyde
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913733981
##########
core/src/main/java/org/apache/calcite/sql/TableCharacteristic.java:
##########
@@ -0,0 +1,333 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql;
+
+import org.checkerframework.checker.nullness.qual.Nullable;
+
+import java.util.Objects;
+
+/**
+ * A table-valued input parameter of a table function is classified by three
+ * characteristics.
+
+ * <p>The first characteristic is semantics. The table has either
+ * row semantics or set semantics.
+ *
+ * <p>Row semantics means that the the result of the table function is decided
+ * on a row-by-row basis.
+ *
+ * <p>Example of a table function with row semantics input table parameter:
+ * We often need to read a CSV file, generally, the first line of the file
+ * contains a list of column names, and subsequent lines of the file contain
+ * data. The data in general can be treated as a large VARCHAR.
+ * However, some of the fields may be numeric or datetime.
+ *
+ * <p>A table function named CSVReader is designed to read a file of
+ * comma-separated values and interpret this file as a table.
+ * The function has three parameters:
+ * <ul>
+ * <li>The first parameter, File, is the name of a file on the query author's
+ * system. This file must contain the comma-separated values that are to be
+ * converted to a table. The first line of the file contains the names of
+ * the resulting columns. Succeeding lines contain the data. Each line after
+ * the first will result in one row of output, with column names as determined
+ * by the first line of the input.
+ * <li>Floats is a descriptor area, which should provide a list of the
+ * column names that are to be interpreted numerically.
+ * These columns will be output with the data type FLOAT.
+ * <li>Dates is a descriptor area, which provides a list of the column
+ * names that are to be interpreted as datetimes.
+ * These columns will be output with the data type DATE.
+ * </ul>
+ *
+ * <p>How to use this table function in query?
+ *
+ * <p>For a csv file which contents are:
+ * <blockquote><pre>
+ * docno,name,due_date,principle,interest
+ * 123,Mary,01/01/2014,234.56,345.67
+ * 234,Edgar,01/01/2014,654.32,543.21
+ * </pre></blockquote>
+ *
+ * <p>The query author may write a query such as the following:
+ * <blockquote><pre>{@code
+ * SELECT *
+ * FROM TABLE (
+ * CSVreader (
+ * 'abc.csv',
+ * DESCRIPTOR ("principle", "interest")
+ * DESCRIPTOR ("due_date")))
+ * }</pre></blockquote>
+ *
+ * <table>
+ * <caption>Results of the query</caption>
+ * <tr>
+ * <th>docno</th>
+ * <th>name</th>
+ * <th>due_date</th>
+ * <th>principle</th>
+ * <th>interest</th>
+ * </tr>
+ * <tr>
+ * <th>123</th>
+ * <th>Mary</th>
+ * <th>01/01/2014</th>
+ * <th>234.56</th>
+ * <th>345.67</th>
+ * </tr>
+ * <tr>
+ * <th>234</th>
+ * <th>Edgar</th>
+ * <th>01/01/2014</th>
+ * <th>654.32</th>
+ * <th>543.21</th>
+ * </tr>
+ * </table>
+ *
+ * <p>Set semantics means that the outcome of the function depends on how
+ * the data is partitioned. Set semantics is useful to implement user-defined
+ * analytics like aggregation or window functions.
+ * They operate on an entire table or a logical partition of it.
+ *
+ * <p>Example of a table function with set semantics input table parameter:
+ * TopN takes an input table that has been sorted on a numeric column.
+ * It copies the first n rows through to the output table. Any additional
+ * rows are summarized in a single output row in which the sort column has
+ * been summed and all other columns are null.
+ * TopN function has two parameters:
+ * <ul>
+ * <li>The first parameter, Input, is the input table. This table has set
+ * semantics, meaning that the result depends on the set of data (since the
+ * last row is a summary row). In addition, the table is marked as PRUNE WHEN
+ * EMPTY, meaning that the result is necessarily empty if the input is empty.
+ * The query author must order this input table on a single numeric column
+ * (syntax below).
+ * <li>The second parameter, Howmany, specifies how many input rows that the
+ * user wants to be copied into the output table; all rows after this will
+ * contribute to the final summary row in the output.
+ * </ul>
+ *
+ * <p>How to use this table function in query?
+ * <table>
+ * <caption>Original records of table orders</caption>
+ * <tr>
+ * <th>region</th>
+ * <th>product</th>
+ * <th>sales</th>
+ * </tr>
+ * <tr>
+ * <th>East</th>
+ * <th>A</th>
+ * <th>1234.56</th>
+ * </tr>
+ * <tr>
+ * <th>East</th>
+ * <th>B</th>
+ * <th>987.65</th>
+ * </tr>
+ * <tr>
+ * <th>East</th>
+ * <th>C</th>
+ * <th>876.54</th>
+ * </tr>
+ * <tr>
+ * <th>East</th>
+ * <th>D</th>
+ * <th>765.43</th>
+ * </tr>
+ * <tr>
+ * <th>East</th>
+ * <th>E</th>
+ * <th>654.32</th>
+ * </tr>
+ * <tr>
+ * <th>West</th>
+ * <th>E</th>
+ * <th>2345.67</th>
+ * </tr>
+ * <tr>
+ * <th>West</th>
+ * <th>D</th>
+ * <th>2001.33</th>
+ * </tr>
+ * <tr>
+ * <th>West</th>
+ * <th>C</th>
+ * <th>1357.99</th>
+ * </tr>
+ * <tr>
+ * <th>West</th>
+ * <th>B</th>
+ * <th>975.35</th>
+ * </tr>
+ * <tr>
+ * <th>West</th>
+ * <th>A</th>
+ * <th>864,22</th>
+ * </tr>
+ * </table>
+ *
+ * <p>The query author may write a query such as the following:
+ * <blockquote><pre>{@code
+ * SELECT *
+ * FROM TABLE(
+ * Topn(
+ * TABLE orders PARTITION BY region ORDER BY sales desc,
+ * 3))
+ * }</pre></blockquote>
+ *
+ * <p>The result will be:
+ * <table>
+ * <caption>Original records of table orders</caption>
+ * <tr>
+ * <th>region</th>
+ * <th>product</th>
+ * <th>sales</th>
+ * </tr>
+ * <tr>
+ * <th>East</th>
+ * <th>A</th>
+ * <th>1234.56</th>
+ * </tr>
+ * <tr>
+ * <th>East</th>
+ * <th>B</th>
+ * <th>987.65</th>
+ * </tr>
+ * <tr>
+ * <th>East</th>
+ * <th>C</th>
+ * <th>876.54</th>
+ * </tr>
+ * <tr>
+ * <th>West</th>
+ * <th>E</th>
+ * <th>2345.67</th>
+ * </tr>
+ * <tr>
+ * <th>West</th>
+ * <th>D</th>
+ * <th>2001.33</th>
+ * </tr>
+ * <tr>
+ * <th>West</th>
+ * <th>C</th>
+ * <th>1357.99</th>
+ * </tr>
+ * </table>
+ *
+ * <p>The second characteristic of input table parameter only applies to input
+ * table with set semantics. It specifies whether the table function can
+ * generate a result row even if the input table is empty.
+ *
+ * <p>The third characteristic is whether the input table supports
+ * pass-through columns or not.
+ */
+public class TableCharacteristic {
+
+ /**
+ * Input table has either row semantics or set semantics.
+ */
+ public final Semantics semantics;
+
+ /**
+ * If the value is true, meaning that the DBMS can prune virtual processors
+ * from the query plan if the input table is empty.
+ * If the value is false, meaning that the DBMS must actually instantiate
+ * a virtual processor (or more than one virtual processor in the presence
+ * of other input tables).
+ */
+ public final boolean pruneIfEmpty;
+
+ /**
+ * If the value is true, for each input row, the table function makes the
+ * entire input row available in the output, qualified by a range variable
+ * associated with the input table. Otherwise the value is false.
+ */
+ public final boolean passColumnsThrough;
+
+ private TableCharacteristic(
+ Semantics semantics,
+ boolean pruneIfEmpty,
+ boolean passColumnsThrough) {
+ this.semantics = semantics;
+ this.pruneIfEmpty = pruneIfEmpty;
+ this.passColumnsThrough = passColumnsThrough;
+ }
+
+ public static TableCharacteristic withRowSemantic(
+ boolean columnsPassThrough) {
+ // Tables with row semantics are always effectively prune when empty.
Review Comment:
Is there any possibility we can add a `Builder` for this clazz ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] danny0405 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r913727713
##########
core/src/main/java/org/apache/calcite/sql/SqlSetSemanticsTableOperator.java:
##########
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.sql;
+
+import org.apache.calcite.rel.type.RelDataType;
+import org.apache.calcite.sql.parser.SqlParserPos;
+import org.apache.calcite.sql.validate.SqlValidator;
+import org.apache.calcite.sql.validate.SqlValidatorScope;
+
+import org.checkerframework.checker.nullness.qual.Nullable;
+
+import java.util.List;
+
+/**
+ * SetSemanticsTable appears as an parameter in a table function.
+ * It represents as an input table with set semantics.
+ * Set semantics means that the outcome of the function depends on how
+ * the data is partitioned.
+ * When the table function is called from a query, the table parameter can
+ * optionally be extended with either a PARTITION BY clause or
+ * an ORDER BY clause or both.
+ *
+ */
+public class SqlSetSemanticsTableOperator extends SqlInternalOperator {
+
+ //~ Constructors -----------------------------------------------------------
+
+ public SqlSetSemanticsTableOperator() {
+ super("SET_SEMANTICS_TABLE", SqlKind.SET_SEMANTICS_TABLE);
+ }
+
+ @Override public SqlCall createCall(
+ @Nullable SqlLiteral functionQualifier,
+ SqlParserPos pos,
+ @Nullable SqlNode... operands) {
+ assert operands.length == 3;
+ SqlNode partitionList = operands[1];
+ SqlNode orderList = operands[2];
+ assert (partitionList != null && !SqlNodeList.isEmptyList(partitionList))
+ || (orderList != null && !SqlNodeList.isEmptyList(orderList));
+ return super.createCall(functionQualifier, pos, operands);
+ }
+
+ @Override public void unparse(
+ SqlWriter writer,
+ SqlCall call,
+ int leftPrec,
+ int rightPrec) {
+ call.operand(0).unparse(writer, 0, 0);
+
+ SqlNodeList partitionList = call.operand(1);
+ if (partitionList.size() > 0) {
+ writer.sep("PARTITION BY");
+ final SqlWriter.Frame partitionFrame = writer.startList("", "");
+ partitionList.unparse(writer, 0, 0);
+ writer.endList(partitionFrame);
+ }
+ SqlNodeList orderList = call.operand(2);
+ if (orderList.size() > 0) {
+ writer.sep("ORDER BY");
+ writer.list(
+ SqlWriter.FrameTypeEnum.ORDER_BY_LIST,
+ SqlWriter.COMMA,
+ orderList);
+ }
+ }
+
+ @Override public RelDataType deriveType(
+ SqlValidator validator,
+ SqlValidatorScope scope, SqlCall call) {
+ final List<SqlNode> operands = call.getOperandList();
+ assert operands.size() == 3;
+ RelDataType tableType = validator.deriveType(scope, operands.get(0));
+ assert tableType != null;
+ return tableType;
+ }
+
+ @Override public boolean argumentMustBeScalar(int ordinal) {
+ return ordinal != 0;
+ }
Review Comment:
Is this right ? Not sure whether the partition by /order by belongs to scalar argument ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 commented on pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
beyond1920 commented on PR #2844:
URL: https://github.com/apache/calcite/pull/2844#issuecomment-1177525405
@danny0405 Thanks for your review.
I've updated pull request to address your comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [calcite] beyond1920 commented on a diff in pull request #2844: [CALCITE-4865] Allow table functions to be polymorphic
Posted by GitBox <gi...@apache.org>.
beyond1920 commented on code in PR #2844:
URL: https://github.com/apache/calcite/pull/2844#discussion_r917343155
##########
core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java:
##########
@@ -3380,6 +3384,66 @@ protected void validateFrom(
getNamespaceOrThrow(node, scope).validate(targetRowType);
}
+ protected void validateTableFunction(SqlCall node, SqlValidatorScope scope,
+ RelDataType targetRowType) {
+ // Dig out real call; TABLE() wrapper is just syntactic.
+ SqlCall call = node.operand(0);
+ if (call.getOperator() instanceof SqlTableFunction) {
+ SqlTableFunction tableFunction = (SqlTableFunction) call.getOperator();
Review Comment:
I prefer to keep `instance of` here.
because `ProcedureNamespace#validateImpl` do a validation after this step.
If I remove `instance of` here, the exception would thrown an exception right here when handling illegal cases like `SqlValidatorTest#testCollectionTable`.
If I want to throw a same exception like the previous behavior, I have to use ` ! instance of` to find illegal cases.
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@calcite.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org