You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "Cpaulyz (via GitHub)" <gi...@apache.org> on 2025/02/05 10:07:23 UTC
[PR] [Draft] Introduce User-defined Polymorphic Table Function [iotdb]
Cpaulyz opened a new pull request, #14791:
URL: https://github.com/apache/iotdb/pull/14791
## Description
Introduct User-defined Polymorphic Table Function
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2714662967
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Draft] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2646334304
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2716251356
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2705522846
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2708053514
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "JackieTien97 (via GitHub)" <gi...@apache.org>.
JackieTien97 commented on code in PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#discussion_r1986751108
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/PartitionRecognizer.java:
##########
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.execution.operator.process.function;
+
+import org.apache.iotdb.commons.udf.utils.UDFDataTypeTransformer;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.PartitionState;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.Slice;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.read.common.block.TsBlock;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Objects;
+
+public class PartitionRecognizer {
+
+ private final List<Integer> partitionChannels;
+ private final List<Object> partitionValues;
+ private final List<Integer> requiredChannels;
+ private final List<Integer> passThroughChannels;
+ private final List<Type> inputDataTypes;
+ private TsBlock currentTsBlock = null;
+ private boolean noMoreData = false;
+ private int currentIndex = 0;
+ private PartitionState state = PartitionState.INIT_STATE;
+
+ public PartitionRecognizer(
+ List<Integer> partitionChannels,
+ List<Integer> requiredChannels,
+ List<Integer> passThroughChannels,
+ List<TSDataType> inputDataTypes) {
+ this.partitionChannels = partitionChannels;
+ this.partitionValues = new ArrayList<>(partitionChannels.size());
+ for (int i = 0; i < partitionChannels.size(); i++) {
+ partitionValues.add(null);
+ }
+ this.requiredChannels = requiredChannels;
+ this.passThroughChannels = passThroughChannels;
+ this.inputDataTypes = UDFDataTypeTransformer.transformToUDFDataTypeList(inputDataTypes);
+ }
+
+ // TsBlock is sorted by partition columns already
+ public void addTsBlock(TsBlock tsBlock) {
+ if (noMoreData) {
+ throw new IllegalArgumentException(
+ "The partition handler is finished, cannot add more data.");
+ }
+ currentTsBlock = tsBlock;
+ }
+
+ /** Marks the handler as finished. */
+ public void noMoreData() {
+ noMoreData = true;
+ }
+
+ public PartitionState nextState() {
+ updateState();
+ return state;
+ }
+
+ private void updateState() {
+ switch (state.getStateType()) {
+ case INIT:
+ state = handleInitState();
+ break;
+ case NEW_PARTITION:
+ state = handleNewPartitionState();
+ break;
+ case ITERATING:
+ state = handleIteratingState();
+ break;
+ case NEED_MORE_DATA:
+ state = handleNeedMoreDataState();
+ break;
+ case FINISHED:
+ // do nothing
+ return;
+ }
+ if (PartitionState.NEED_MORE_DATA_STATE.equals(state)) {
+ currentIndex = 0;
+ }
+ }
+
+ private PartitionState handleInitState() {
+ if (currentTsBlock == null || currentTsBlock.isEmpty()) {
+ return PartitionState.INIT_STATE;
+ }
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+
+ private PartitionState handleNewPartitionState() {
+ if (currentIndex >= currentTsBlock.getPositionCount()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ } else {
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+ }
+
+ private PartitionState handleNeedMoreDataState() {
+ if (noMoreData) {
+ return PartitionState.FINISHED_STATE;
+ } else if (currentTsBlock == null || currentTsBlock.isEmpty()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ }
+ int endPartitionIndex = findNextDifferentRowIndex();
+ if (endPartitionIndex != 0) {
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.iteratingState(slice);
+ } else {
+ currentIndex = endPartitionIndex;
+ endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+ }
+
+ private PartitionState handleIteratingState() {
+ if (currentIndex >= currentTsBlock.getPositionCount()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ } else {
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
Review Comment:
what if endPartitionIndex == endPostion?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2676662024
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2676264331
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2714404427
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "JackieTien97 (via GitHub)" <gi...@apache.org>.
JackieTien97 commented on code in PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#discussion_r1984251960
##########
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/TableFunction.java:
##########
@@ -19,4 +19,69 @@
package org.apache.iotdb.udf.api.relational;
-public interface TableFunction extends SQLFunction {}
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+
+import java.util.List;
+import java.util.Map;
+
+public interface TableFunction extends SQLFunction {
+
+ /**
+ * This method is used to define the specification of the arguments required by the table
+ * function. Each argument is described using a {@link ParameterSpecification} object, which
+ * encapsulates details such as the argument name, whether it is required, its default value (if
+ * any), etc.
+ *
+ * <p>The {@link ParameterSpecification} class is abstract and has two concrete implementations:
+ *
+ * <ul>
+ * <li>{@link TableParameterSpecification}: Used for parameters specific to table functions.
+ * <li>{@link ScalarParameterSpecification}: Used for parameters specific to scalar functions.
Review Comment:
```suggestion
* <li>{@link ScalarParameterSpecification}: Used for parameters specific to scalar arguments.
```
##########
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/TableFunction.java:
##########
@@ -19,4 +19,69 @@
package org.apache.iotdb.udf.api.relational;
-public interface TableFunction extends SQLFunction {}
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+
+import java.util.List;
+import java.util.Map;
+
+public interface TableFunction extends SQLFunction {
+
+ /**
+ * This method is used to define the specification of the arguments required by the table
+ * function. Each argument is described using a {@link ParameterSpecification} object, which
+ * encapsulates details such as the argument name, whether it is required, its default value (if
+ * any), etc.
+ *
+ * <p>The {@link ParameterSpecification} class is abstract and has two concrete implementations:
+ *
+ * <ul>
+ * <li>{@link TableParameterSpecification}: Used for parameters specific to table functions.
+ * <li>{@link ScalarParameterSpecification}: Used for parameters specific to scalar functions.
+ * </ul>
+ *
+ * @return a list of {@link ParameterSpecification} objects describing the function's arguments.
+ * The list should include all parameters that the table function expects, along with their
+ * constraints and default values (if applicable).
+ */
+ List<ParameterSpecification> getArgumentsSpecifications();
+
+ /**
+ * This method is responsible for analyzing the provided arguments and constructing a {@link
+ * TableFunctionAnalysis} object in runtime. During analysis, the method should:
+ *
+ * <ul>
+ * <li>Extract values from scalar arguments (instances of {@link ScalarArgument}) or extract
+ * table descriptions from table arguments (instances of {@link TableArgument}).
+ * <li>Construct a {@link TableFunctionAnalysis} object that contains:
+ * <ul>
+ * <li>A description of proper columns.
Review Comment:
explain what's proper columns. I think they're sub columns of the final output table?
The final table will contain all proper columns and all pass through columns(or partition by columns)?
##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/TableBuiltinTableFunction.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational;
+
+import org.apache.iotdb.commons.udf.builtin.relational.tvf.HOPTableFunction;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+public enum TableBuiltinTableFunction {
+// TODO(UDF): support built in function
Review Comment:
```suggestion
```
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/parser/AstBuilder.java:
##########
@@ -2197,6 +2201,122 @@ public Node visitParenthesizedRelation(RelationalSqlParser.ParenthesizedRelation
return visit(ctx.relation());
}
+ @Override
+ public Node visitTableFunctionInvocation(
+ RelationalSqlParser.TableFunctionInvocationContext context) {
+ return visit(context.tableFunctionCall());
+ }
+
+ @Override
+ public Node visitTableFunctionCall(RelationalSqlParser.TableFunctionCallContext context) {
+ QualifiedName name = getQualifiedName(context.qualifiedName());
+ List<TableFunctionArgument> arguments =
+ visit(context.tableFunctionArgument(), TableFunctionArgument.class);
+ List<List<QualifiedName>> copartitioning = ImmutableList.of();
+ if (context.COPARTITION() != null) {
Review Comment:
Did we support this now? If not, better to throw SematicException here.
##########
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/table/TableFunctionAnalysis.java:
##########
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.udf.api.relational.table;
+
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * An object of this class is produced by the `analyze()` method of a `ConnectorTableFunction`
Review Comment:
```suggestion
* An object of this class is produced by the `analyze()` method of a `TableFunction`
```
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/optimizations/PushLimitOffsetIntoTableScan.java:
##########
@@ -237,6 +238,16 @@ public PlanNode visitInformationSchemaTableScan(
return node;
}
+ @Override
+ public PlanNode visitTableFunctionProcessor(TableFunctionProcessorNode node, Context context) {
+ if (node.getChildren().isEmpty()) {
+ context.enablePushDown = false;
+ return node;
+ } else {
+ return visitPlan(node, context);
Review Comment:
can we push limitOffset through TableFunctionProcessorNode?
##########
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/TableFunction.java:
##########
@@ -19,4 +19,69 @@
package org.apache.iotdb.udf.api.relational;
-public interface TableFunction extends SQLFunction {}
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+
+import java.util.List;
+import java.util.Map;
+
+public interface TableFunction extends SQLFunction {
+
+ /**
+ * This method is used to define the specification of the arguments required by the table
+ * function. Each argument is described using a {@link ParameterSpecification} object, which
+ * encapsulates details such as the argument name, whether it is required, its default value (if
+ * any), etc.
+ *
+ * <p>The {@link ParameterSpecification} class is abstract and has two concrete implementations:
+ *
+ * <ul>
+ * <li>{@link TableParameterSpecification}: Used for parameters specific to table functions.
Review Comment:
also emphasize here, we only support at most one TableParameterSpecification
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/metadata/ShowFunctionsTask.java:
##########
@@ -152,6 +153,11 @@ private static void buildTableModelTsBlock(
TableBuiltinAggregationFunction.getBuiltInAggregateFunctionName(),
BINARY_MAP.get(FUNCTION_TYPE_BUILTIN_AGG_FUNC),
BINARY_MAP.get(FUNCTION_STATE_AVAILABLE));
+ appendFunctions(
+ builder,
+ TableBuiltinTableFunction.getBuiltInTableFunctionName(),
+ BINARY_MAP.get(FUNCTION_TYPE_BUILTIN_AGG_FUNC),
Review Comment:
FUNCTION_TYPE_BUILTIN_TABLE_FUNC
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/relational/aggregation/grouped/GroupedUserDefinedAggregateAccumulator.java:
##########
@@ -88,6 +88,10 @@ public void addInput(int[] groupIds, Column[] arguments, AggregationMask mask) {
int groupId = groupIds[selectedPositions[index]];
index++;
State state = getOrCreateState(groupId);
+ if (state == null) {
+ state = aggregateFunction.createState();
+ stateArray.set(groupId, state);
+ }
Review Comment:
```suggestion
```
Why doing this? already included in `getOrCreateState`
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/execution/config/executor/ClusterConfigTaskExecutor.java:
##########
@@ -613,6 +614,23 @@ public SettableFuture<ConfigTaskResult> createFunction(
functionType = FunctionType.AGGREGATE;
} else if (o instanceof TableFunction) {
functionType = FunctionType.TABLE;
+ // check there is no duplicate argument specification for name
+ TableFunction tableFunction = (TableFunction) o;
+ Set<String> argNames = new HashSet<>();
+ for (ParameterSpecification specification :
+ tableFunction.getArgumentsSpecifications()) {
+ if (!argNames.add(specification.getName())) {
+ future.setException(
+ new IoTDBException(
+ "Failed to create function '"
+ + udfName
+ + "', because there is duplicate argument name '"
+ + specification.getName()
+ + "'.",
+ TSStatusCode.UDF_LOAD_CLASS_ERROR.getStatusCode()));
Review Comment:
add it in

avoid print exception stack in warn log, because it's just user's wrong
##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/HOPTableFunction.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+
+public class HOPTableFunction implements TableFunction {
+
+ private static final String DATA_PARAMETER_NAME = "DATA";
+ private static final String TIMECOL_PARAMETER_NAME = "TIMECOL";
+ private static final String SLIDE_PARAMETER_NAME = "SLIDE";
+ private static final String SIZE_PARAMETER_NAME = "SIZE";
+ private static final String START_PARAMETER_NAME = "START";
Review Comment:
```suggestion
private static final String ORIGIN_PARAMETER_NAME = "ORIGIN";
```
same as date_bin's third parameter
##########
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/TableFunction.java:
##########
@@ -19,4 +19,69 @@
package org.apache.iotdb.udf.api.relational;
-public interface TableFunction extends SQLFunction {}
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+
+import java.util.List;
+import java.util.Map;
+
+public interface TableFunction extends SQLFunction {
+
+ /**
+ * This method is used to define the specification of the arguments required by the table
+ * function. Each argument is described using a {@link ParameterSpecification} object, which
+ * encapsulates details such as the argument name, whether it is required, its default value (if
+ * any), etc.
+ *
+ * <p>The {@link ParameterSpecification} class is abstract and has two concrete implementations:
+ *
+ * <ul>
+ * <li>{@link TableParameterSpecification}: Used for parameters specific to table functions.
Review Comment:
```suggestion
* <li>{@link TableParameterSpecification}: Used for parameter specific to table argument?.
```
##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/HOPTableFunction.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+
+public class HOPTableFunction implements TableFunction {
+
+ private static final String DATA_PARAMETER_NAME = "DATA";
+ private static final String TIMECOL_PARAMETER_NAME = "TIMECOL";
+ private static final String SLIDE_PARAMETER_NAME = "SLIDE";
+ private static final String SIZE_PARAMETER_NAME = "SIZE";
+ private static final String START_PARAMETER_NAME = "START";
+
+ @Override
+ public List<ParameterSpecification> getArgumentsSpecifications() {
+ return Arrays.asList(
+ TableParameterSpecification.builder()
+ .name(DATA_PARAMETER_NAME)
+ .rowSemantics()
+ .passThroughColumns()
+ .build(),
+ ScalarParameterSpecification.builder()
+ .name(TIMECOL_PARAMETER_NAME)
Review Comment:
defaultValue "time" ?
##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/HOPTableFunction.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+
+public class HOPTableFunction implements TableFunction {
+
+ private static final String DATA_PARAMETER_NAME = "DATA";
+ private static final String TIMECOL_PARAMETER_NAME = "TIMECOL";
+ private static final String SLIDE_PARAMETER_NAME = "SLIDE";
+ private static final String SIZE_PARAMETER_NAME = "SIZE";
+ private static final String START_PARAMETER_NAME = "START";
+
+ @Override
+ public List<ParameterSpecification> getArgumentsSpecifications() {
+ return Arrays.asList(
+ TableParameterSpecification.builder()
+ .name(DATA_PARAMETER_NAME)
+ .rowSemantics()
+ .passThroughColumns()
+ .build(),
+ ScalarParameterSpecification.builder()
+ .name(TIMECOL_PARAMETER_NAME)
+ .type(Type.STRING)
+ .build(),
+ ScalarParameterSpecification.builder().name(SLIDE_PARAMETER_NAME).type(Type.INT64).build(),
+ ScalarParameterSpecification.builder().name(SIZE_PARAMETER_NAME).type(Type.INT64).build(),
+ ScalarParameterSpecification.builder()
+ .name(START_PARAMETER_NAME)
+ .type(Type.TIMESTAMP)
+ .defaultValue(0L)
+ .build());
+ }
+
+ private int findTimeColumnIndex(TableArgument tableArgument, String expectedFieldName) {
+ int requiredIndex = -1;
+ for (int i = 0; i < tableArgument.getFieldTypes().size(); i++) {
+ Optional<String> fieldName = tableArgument.getFieldNames().get(i);
+ if (fieldName.isPresent() && expectedFieldName.equalsIgnoreCase(fieldName.get())) {
+ requiredIndex = i;
+ break;
+ }
+ }
+ return requiredIndex;
+ }
+
+ @Override
+ public TableFunctionAnalysis analyze(Map<String, Argument> arguments) {
+ TableArgument tableArgument = (TableArgument) arguments.get(DATA_PARAMETER_NAME);
+ String expectedFieldName =
+ (String) ((ScalarArgument) arguments.get(TIMECOL_PARAMETER_NAME)).getValue();
+ int requiredIndex = findTimeColumnIndex(tableArgument, expectedFieldName);
+ if (requiredIndex == -1) {
+ throw new UDFException("The required field is not found in the input table");
+ }
+ DescribedSchema properColumnSchema =
+ new DescribedSchema.Builder()
+ .addField("window_start", Type.TIMESTAMP)
+ .addField("window_end", Type.TIMESTAMP)
+ .build();
+
+ // outputColumnSchema
+ return TableFunctionAnalysis.builder()
+ .properColumnSchema(properColumnSchema)
+ .requiredColumns(DATA_PARAMETER_NAME, Collections.singletonList(requiredIndex))
+ .build();
+ }
+
+ @Override
+ public TableFunctionProcessorProvider getProcessorProvider(Map<String, Argument> arguments) {
+ return new TableFunctionProcessorProvider() {
+ @Override
+ public TableFunctionDataProcessor getDataProcessor() {
+ return new HOPDataProcessor(
+ (Long) ((ScalarArgument) arguments.get(START_PARAMETER_NAME)).getValue(),
+ (Long) ((ScalarArgument) arguments.get(SLIDE_PARAMETER_NAME)).getValue(),
+ (Long) ((ScalarArgument) arguments.get(SIZE_PARAMETER_NAME)).getValue());
+ }
+ };
+ }
+
+ private static class HOPDataProcessor implements TableFunctionDataProcessor {
+
+ private final long slide;
+ private final long size;
Review Comment:
what if slide or size is year or month level?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2705627815
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "JackieTien97 (via GitHub)" <gi...@apache.org>.
JackieTien97 commented on code in PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#discussion_r1986858760
##########
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/table/processor/TableFunctionDataProcessor.java:
##########
@@ -0,0 +1,64 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.udf.api.relational.table.processor;
+
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.List;
+
+/** Each instance of TableFunctionDataProcessor processes one partition of data. */
+public interface TableFunctionDataProcessor {
+
+ default void beforeStart() {
Review Comment:
need to be call after being inited.
##########
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/table/processor/TableFunctionLeafProcessor.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.udf.api.relational.table.processor;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.List;
+
+public interface TableFunctionLeafProcessor {
+
+ default void beforeStart() {
+ // do nothing
+ }
Review Comment:
need to be call after being inited.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2705682426
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.
Cpaulyz commented on code in PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#discussion_r1984412551
##########
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/TableFunction.java:
##########
@@ -19,4 +19,69 @@
package org.apache.iotdb.udf.api.relational;
-public interface TableFunction extends SQLFunction {}
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+
+import java.util.List;
+import java.util.Map;
+
+public interface TableFunction extends SQLFunction {
+
+ /**
+ * This method is used to define the specification of the arguments required by the table
+ * function. Each argument is described using a {@link ParameterSpecification} object, which
+ * encapsulates details such as the argument name, whether it is required, its default value (if
+ * any), etc.
+ *
+ * <p>The {@link ParameterSpecification} class is abstract and has two concrete implementations:
+ *
+ * <ul>
+ * <li>{@link TableParameterSpecification}: Used for parameters specific to table functions.
+ * <li>{@link ScalarParameterSpecification}: Used for parameters specific to scalar functions.
+ * </ul>
+ *
+ * @return a list of {@link ParameterSpecification} objects describing the function's arguments.
+ * The list should include all parameters that the table function expects, along with their
+ * constraints and default values (if applicable).
+ */
+ List<ParameterSpecification> getArgumentsSpecifications();
+
+ /**
+ * This method is responsible for analyzing the provided arguments and constructing a {@link
+ * TableFunctionAnalysis} object in runtime. During analysis, the method should:
+ *
+ * <ul>
+ * <li>Extract values from scalar arguments (instances of {@link ScalarArgument}) or extract
+ * table descriptions from table arguments (instances of {@link TableArgument}).
+ * <li>Construct a {@link TableFunctionAnalysis} object that contains:
+ * <ul>
+ * <li>A description of proper columns.
Review Comment:
Good idea! I think it is an important knowledge for UDTF developer, so I added this explaination at the top of `TableFunction` interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.
Cpaulyz commented on code in PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#discussion_r1984411617
##########
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/TableFunction.java:
##########
@@ -19,4 +19,69 @@
package org.apache.iotdb.udf.api.relational;
-public interface TableFunction extends SQLFunction {}
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+
+import java.util.List;
+import java.util.Map;
+
+public interface TableFunction extends SQLFunction {
+
+ /**
+ * This method is used to define the specification of the arguments required by the table
+ * function. Each argument is described using a {@link ParameterSpecification} object, which
+ * encapsulates details such as the argument name, whether it is required, its default value (if
+ * any), etc.
+ *
+ * <p>The {@link ParameterSpecification} class is abstract and has two concrete implementations:
+ *
+ * <ul>
+ * <li>{@link TableParameterSpecification}: Used for parameters specific to table functions.
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2715054929
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2676758039
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2676763627
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.
Cpaulyz commented on code in PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#discussion_r1984423188
##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/HOPTableFunction.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+
+public class HOPTableFunction implements TableFunction {
+
+ private static final String DATA_PARAMETER_NAME = "DATA";
+ private static final String TIMECOL_PARAMETER_NAME = "TIMECOL";
+ private static final String SLIDE_PARAMETER_NAME = "SLIDE";
+ private static final String SIZE_PARAMETER_NAME = "SIZE";
+ private static final String START_PARAMETER_NAME = "START";
+
+ @Override
+ public List<ParameterSpecification> getArgumentsSpecifications() {
+ return Arrays.asList(
+ TableParameterSpecification.builder()
+ .name(DATA_PARAMETER_NAME)
+ .rowSemantics()
+ .passThroughColumns()
+ .build(),
+ ScalarParameterSpecification.builder()
+ .name(TIMECOL_PARAMETER_NAME)
+ .type(Type.STRING)
+ .build(),
+ ScalarParameterSpecification.builder().name(SLIDE_PARAMETER_NAME).type(Type.INT64).build(),
+ ScalarParameterSpecification.builder().name(SIZE_PARAMETER_NAME).type(Type.INT64).build(),
+ ScalarParameterSpecification.builder()
+ .name(START_PARAMETER_NAME)
+ .type(Type.TIMESTAMP)
+ .defaultValue(0L)
+ .build());
+ }
+
+ private int findTimeColumnIndex(TableArgument tableArgument, String expectedFieldName) {
+ int requiredIndex = -1;
+ for (int i = 0; i < tableArgument.getFieldTypes().size(); i++) {
+ Optional<String> fieldName = tableArgument.getFieldNames().get(i);
+ if (fieldName.isPresent() && expectedFieldName.equalsIgnoreCase(fieldName.get())) {
+ requiredIndex = i;
+ break;
+ }
+ }
+ return requiredIndex;
+ }
+
+ @Override
+ public TableFunctionAnalysis analyze(Map<String, Argument> arguments) {
+ TableArgument tableArgument = (TableArgument) arguments.get(DATA_PARAMETER_NAME);
+ String expectedFieldName =
+ (String) ((ScalarArgument) arguments.get(TIMECOL_PARAMETER_NAME)).getValue();
+ int requiredIndex = findTimeColumnIndex(tableArgument, expectedFieldName);
+ if (requiredIndex == -1) {
+ throw new UDFException("The required field is not found in the input table");
+ }
+ DescribedSchema properColumnSchema =
+ new DescribedSchema.Builder()
+ .addField("window_start", Type.TIMESTAMP)
+ .addField("window_end", Type.TIMESTAMP)
+ .build();
+
+ // outputColumnSchema
+ return TableFunctionAnalysis.builder()
+ .properColumnSchema(properColumnSchema)
+ .requiredColumns(DATA_PARAMETER_NAME, Collections.singletonList(requiredIndex))
+ .build();
+ }
+
+ @Override
+ public TableFunctionProcessorProvider getProcessorProvider(Map<String, Argument> arguments) {
+ return new TableFunctionProcessorProvider() {
+ @Override
+ public TableFunctionDataProcessor getDataProcessor() {
+ return new HOPDataProcessor(
+ (Long) ((ScalarArgument) arguments.get(START_PARAMETER_NAME)).getValue(),
+ (Long) ((ScalarArgument) arguments.get(SLIDE_PARAMETER_NAME)).getValue(),
+ (Long) ((ScalarArgument) arguments.get(SIZE_PARAMETER_NAME)).getValue());
+ }
+ };
+ }
+
+ private static class HOPDataProcessor implements TableFunctionDataProcessor {
+
+ private final long slide;
+ private final long size;
Review Comment:
Monthly interval is banned for scalar argument now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2710782503
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "JackieTien97 (via GitHub)" <gi...@apache.org>.
JackieTien97 merged PR #14791:
URL: https://github.com/apache/iotdb/pull/14791
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "JackieTien97 (via GitHub)" <gi...@apache.org>.
JackieTien97 commented on code in PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#discussion_r1986566593
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/partition/Slice.java:
##########
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.execution.operator.process.function.partition;
+
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.Column;
+import org.apache.tsfile.common.conf.TSFileConfig;
+import org.apache.tsfile.utils.Binary;
+import org.apache.tsfile.utils.DateUtils;
+
+import java.time.LocalDate;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import java.util.stream.Collectors;
+
+/** Parts of partition. */
+public class Slice {
+
+ private final Column[] requiredColumns;
+ private final Column[] passThroughColumns;
+ private final List<Type> dataTypes;
+ private final long size;
+
+ public Slice(
+ int startIndex,
+ int endIndex,
+ Column[] columns,
+ List<Integer> requiredChannels,
+ List<Integer> passThroughChannels,
+ List<Type> dataTypes) {
+ this.size = endIndex - startIndex;
+ List<Column> partitionColumns =
+ Arrays.stream(columns)
+ .map(i -> i.getRegion(startIndex, (int) size))
+ .collect(Collectors.toList());
+ this.requiredColumns =
+ requiredChannels.stream().map(partitionColumns::get).toArray(Column[]::new);
+ this.passThroughColumns =
+ passThroughChannels.stream().map(partitionColumns::get).toArray(Column[]::new);
+ this.dataTypes = dataTypes;
+ }
+
+ public long getSize() {
+ return size;
+ }
+
+ public Column[] getPassThroughResult(int[] indexes) {
+ return Arrays.stream(passThroughColumns)
+ .map(i -> i.getPositions(indexes, 0, indexes.length))
+ .toArray(Column[]::new);
+ }
+
+ public Iterator<Record> getRequiredRecordIterator() {
+ return new Iterator<Record>() {
+ private int curIndex = 0;
+
+ @Override
+ public boolean hasNext() {
+ return curIndex < size;
+ }
+
+ @Override
+ public Record next() {
+ if (!hasNext()) {
+ throw new NoSuchElementException();
+ }
+ final int idx = curIndex++;
+ return getRecord(idx, requiredColumns);
+ }
+ };
+ }
+
+ private Record getRecord(int offset, Column[] originalColumns) {
+ return new Record() {
Review Comment:
I think it may be better to add a RecordImpl class to implement Record instead of contructing a new class each time.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/PartitionRecognizer.java:
##########
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.execution.operator.process.function;
+
+import org.apache.iotdb.commons.udf.utils.UDFDataTypeTransformer;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.PartitionState;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.Slice;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.read.common.block.TsBlock;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Objects;
+
+public class PartitionRecognizer {
+
+ private final List<Integer> partitionChannels;
+ private final List<Object> partitionValues;
+ private final List<Integer> requiredChannels;
+ private final List<Integer> passThroughChannels;
+ private final List<Type> inputDataTypes;
+ private TsBlock currentTsBlock = null;
+ private boolean noMoreData = false;
+ private int currentIndex = 0;
+ private PartitionState state = PartitionState.INIT_STATE;
+
+ public PartitionRecognizer(
+ List<Integer> partitionChannels,
+ List<Integer> requiredChannels,
+ List<Integer> passThroughChannels,
+ List<TSDataType> inputDataTypes) {
+ this.partitionChannels = partitionChannels;
+ this.partitionValues = new ArrayList<>(partitionChannels.size());
+ for (int i = 0; i < partitionChannels.size(); i++) {
+ partitionValues.add(null);
+ }
+ this.requiredChannels = requiredChannels;
+ this.passThroughChannels = passThroughChannels;
+ this.inputDataTypes = UDFDataTypeTransformer.transformToUDFDataTypeList(inputDataTypes);
+ }
+
+ // TsBlock is sorted by partition columns already
+ public void addTsBlock(TsBlock tsBlock) {
+ if (noMoreData) {
+ throw new IllegalArgumentException(
+ "The partition handler is finished, cannot add more data.");
+ }
+ currentTsBlock = tsBlock;
+ }
+
+ /** Marks the handler as finished. */
+ public void noMoreData() {
+ noMoreData = true;
+ }
+
+ public PartitionState nextState() {
+ updateState();
+ return state;
+ }
+
+ private void updateState() {
+ switch (state.getStateType()) {
+ case INIT:
+ state = handleInitState();
+ break;
+ case NEW_PARTITION:
+ state = handleNewPartitionState();
+ break;
+ case ITERATING:
+ state = handleIteratingState();
+ break;
+ case NEED_MORE_DATA:
+ state = handleNeedMoreDataState();
+ break;
+ case FINISHED:
+ // do nothing
+ return;
+ }
+ if (PartitionState.NEED_MORE_DATA_STATE.equals(state)) {
+ currentIndex = 0;
+ }
+ }
+
+ private PartitionState handleInitState() {
+ if (currentTsBlock == null || currentTsBlock.isEmpty()) {
+ return PartitionState.INIT_STATE;
+ }
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+
+ private PartitionState handleNewPartitionState() {
+ if (currentIndex >= currentTsBlock.getPositionCount()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ } else {
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+ }
+
+ private PartitionState handleNeedMoreDataState() {
+ if (noMoreData) {
+ return PartitionState.FINISHED_STATE;
+ } else if (currentTsBlock == null || currentTsBlock.isEmpty()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ }
+ int endPartitionIndex = findNextDifferentRowIndex();
+ if (endPartitionIndex != 0) {
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.iteratingState(slice);
+ } else {
+ currentIndex = endPartitionIndex;
+ endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+ }
+
+ private PartitionState handleIteratingState() {
+ if (currentIndex >= currentTsBlock.getPositionCount()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ } else {
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+ }
+
+ /**
+ * Find next row index whose partition values are different from the current partition values. If
+ * all rows have the same partition values, return the position count of the current TsBlock.
+ */
+ private int findNextDifferentRowIndex() {
+ int i = currentIndex;
+ while (i < currentTsBlock.getPositionCount()) {
+ for (int j = 0; j < partitionChannels.size(); j++) {
+ if (!Objects.equals(
+ partitionValues.get(j),
+ currentTsBlock.getColumn(partitionChannels.get(j)).getObject(i))) {
Review Comment:
I think it may be better to use Comparator<SortKey> to do this
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/PartitionRecognizer.java:
##########
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.execution.operator.process.function;
+
+import org.apache.iotdb.commons.udf.utils.UDFDataTypeTransformer;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.PartitionState;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.Slice;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.read.common.block.TsBlock;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Objects;
+
+public class PartitionRecognizer {
+
+ private final List<Integer> partitionChannels;
+ private final List<Object> partitionValues;
+ private final List<Integer> requiredChannels;
+ private final List<Integer> passThroughChannels;
+ private final List<Type> inputDataTypes;
+ private TsBlock currentTsBlock = null;
+ private boolean noMoreData = false;
+ private int currentIndex = 0;
+ private PartitionState state = PartitionState.INIT_STATE;
+
+ public PartitionRecognizer(
+ List<Integer> partitionChannels,
+ List<Integer> requiredChannels,
+ List<Integer> passThroughChannels,
+ List<TSDataType> inputDataTypes) {
+ this.partitionChannels = partitionChannels;
+ this.partitionValues = new ArrayList<>(partitionChannels.size());
+ for (int i = 0; i < partitionChannels.size(); i++) {
+ partitionValues.add(null);
+ }
+ this.requiredChannels = requiredChannels;
+ this.passThroughChannels = passThroughChannels;
+ this.inputDataTypes = UDFDataTypeTransformer.transformToUDFDataTypeList(inputDataTypes);
+ }
+
+ // TsBlock is sorted by partition columns already
+ public void addTsBlock(TsBlock tsBlock) {
+ if (noMoreData) {
+ throw new IllegalArgumentException(
+ "The partition handler is finished, cannot add more data.");
+ }
+ currentTsBlock = tsBlock;
+ }
+
+ /** Marks the handler as finished. */
+ public void noMoreData() {
+ noMoreData = true;
+ }
+
+ public PartitionState nextState() {
+ updateState();
+ return state;
+ }
+
+ private void updateState() {
+ switch (state.getStateType()) {
+ case INIT:
+ state = handleInitState();
+ break;
+ case NEW_PARTITION:
+ state = handleNewPartitionState();
+ break;
+ case ITERATING:
+ state = handleIteratingState();
+ break;
+ case NEED_MORE_DATA:
+ state = handleNeedMoreDataState();
+ break;
+ case FINISHED:
+ // do nothing
+ return;
+ }
+ if (PartitionState.NEED_MORE_DATA_STATE.equals(state)) {
+ currentIndex = 0;
+ }
+ }
+
+ private PartitionState handleInitState() {
+ if (currentTsBlock == null || currentTsBlock.isEmpty()) {
+ return PartitionState.INIT_STATE;
+ }
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+
+ private PartitionState handleNewPartitionState() {
+ if (currentIndex >= currentTsBlock.getPositionCount()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ } else {
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+ }
+
+ private PartitionState handleNeedMoreDataState() {
+ if (noMoreData) {
+ return PartitionState.FINISHED_STATE;
+ } else if (currentTsBlock == null || currentTsBlock.isEmpty()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ }
+ int endPartitionIndex = findNextDifferentRowIndex();
+ if (endPartitionIndex != 0) {
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.iteratingState(slice);
+ } else {
+ currentIndex = endPartitionIndex;
+ endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+ }
+
+ private PartitionState handleIteratingState() {
+ if (currentIndex >= currentTsBlock.getPositionCount()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ } else {
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
Review Comment:
what if endPartitionIndex == endPostion?
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/partition/Slice.java:
##########
@@ -0,0 +1,159 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.execution.operator.process.function.partition;
+
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.Column;
+import org.apache.tsfile.common.conf.TSFileConfig;
+import org.apache.tsfile.utils.Binary;
+import org.apache.tsfile.utils.DateUtils;
+
+import java.time.LocalDate;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+import java.util.NoSuchElementException;
+import java.util.stream.Collectors;
+
+/** Parts of partition. */
+public class Slice {
+
+ private final Column[] requiredColumns;
+ private final Column[] passThroughColumns;
+ private final List<Type> dataTypes;
+ private final long size;
+
+ public Slice(
+ int startIndex,
+ int endIndex,
+ Column[] columns,
+ List<Integer> requiredChannels,
+ List<Integer> passThroughChannels,
+ List<Type> dataTypes) {
+ this.size = endIndex - startIndex;
+ List<Column> partitionColumns =
+ Arrays.stream(columns)
+ .map(i -> i.getRegion(startIndex, (int) size))
+ .collect(Collectors.toList());
+ this.requiredColumns =
+ requiredChannels.stream().map(partitionColumns::get).toArray(Column[]::new);
+ this.passThroughColumns =
+ passThroughChannels.stream().map(partitionColumns::get).toArray(Column[]::new);
+ this.dataTypes = dataTypes;
+ }
+
+ public long getSize() {
+ return size;
+ }
+
+ public Column[] getPassThroughResult(int[] indexes) {
+ return Arrays.stream(passThroughColumns)
+ .map(i -> i.getPositions(indexes, 0, indexes.length))
+ .toArray(Column[]::new);
+ }
+
+ public Iterator<Record> getRequiredRecordIterator() {
+ return new Iterator<Record>() {
+ private int curIndex = 0;
+
+ @Override
+ public boolean hasNext() {
+ return curIndex < size;
+ }
+
+ @Override
+ public Record next() {
+ if (!hasNext()) {
+ throw new NoSuchElementException();
+ }
+ final int idx = curIndex++;
+ return getRecord(idx, requiredColumns);
+ }
+ };
+ }
+
+ private Record getRecord(int offset, Column[] originalColumns) {
+ return new Record() {
Review Comment:
same as next() method in `org.apache.iotdb.commons.udf.access.RecordIterator`
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/PartitionRecognizer.java:
##########
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.execution.operator.process.function;
+
+import org.apache.iotdb.commons.udf.utils.UDFDataTypeTransformer;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.PartitionState;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.Slice;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.read.common.block.TsBlock;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Objects;
+
+public class PartitionRecognizer {
+
+ private final List<Integer> partitionChannels;
+ private final List<Object> partitionValues;
+ private final List<Integer> requiredChannels;
+ private final List<Integer> passThroughChannels;
+ private final List<Type> inputDataTypes;
+ private TsBlock currentTsBlock = null;
+ private boolean noMoreData = false;
+ private int currentIndex = 0;
+ private PartitionState state = PartitionState.INIT_STATE;
+
+ public PartitionRecognizer(
+ List<Integer> partitionChannels,
+ List<Integer> requiredChannels,
+ List<Integer> passThroughChannels,
+ List<TSDataType> inputDataTypes) {
+ this.partitionChannels = partitionChannels;
+ this.partitionValues = new ArrayList<>(partitionChannels.size());
+ for (int i = 0; i < partitionChannels.size(); i++) {
+ partitionValues.add(null);
+ }
+ this.requiredChannels = requiredChannels;
+ this.passThroughChannels = passThroughChannels;
+ this.inputDataTypes = UDFDataTypeTransformer.transformToUDFDataTypeList(inputDataTypes);
+ }
+
+ // TsBlock is sorted by partition columns already
+ public void addTsBlock(TsBlock tsBlock) {
+ if (noMoreData) {
+ throw new IllegalArgumentException(
+ "The partition handler is finished, cannot add more data.");
+ }
+ currentTsBlock = tsBlock;
+ }
+
+ /** Marks the handler as finished. */
+ public void noMoreData() {
+ noMoreData = true;
+ }
+
+ public PartitionState nextState() {
+ updateState();
+ return state;
+ }
+
+ private void updateState() {
+ switch (state.getStateType()) {
+ case INIT:
+ state = handleInitState();
+ break;
+ case NEW_PARTITION:
+ state = handleNewPartitionState();
+ break;
+ case ITERATING:
+ state = handleIteratingState();
+ break;
+ case NEED_MORE_DATA:
+ state = handleNeedMoreDataState();
+ break;
+ case FINISHED:
+ // do nothing
+ return;
+ }
+ if (PartitionState.NEED_MORE_DATA_STATE.equals(state)) {
+ currentIndex = 0;
+ }
+ }
+
+ private PartitionState handleInitState() {
+ if (currentTsBlock == null || currentTsBlock.isEmpty()) {
+ return PartitionState.INIT_STATE;
+ }
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+
+ private PartitionState handleNewPartitionState() {
+ if (currentIndex >= currentTsBlock.getPositionCount()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ } else {
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+ }
+
+ private PartitionState handleNeedMoreDataState() {
+ if (noMoreData) {
+ return PartitionState.FINISHED_STATE;
+ } else if (currentTsBlock == null || currentTsBlock.isEmpty()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ }
+ int endPartitionIndex = findNextDifferentRowIndex();
+ if (endPartitionIndex != 0) {
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.iteratingState(slice);
+ } else {
+ currentIndex = endPartitionIndex;
+ endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+ }
+
+ private PartitionState handleIteratingState() {
Review Comment:
It seems that `handleIteratingState` is same as `handleNewPartitionState`
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/PartitionRecognizer.java:
##########
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.execution.operator.process.function;
+
+import org.apache.iotdb.commons.udf.utils.UDFDataTypeTransformer;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.PartitionState;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.Slice;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.read.common.block.TsBlock;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Objects;
+
+public class PartitionRecognizer {
+
+ private final List<Integer> partitionChannels;
+ private final List<Object> partitionValues;
+ private final List<Integer> requiredChannels;
+ private final List<Integer> passThroughChannels;
+ private final List<Type> inputDataTypes;
+ private TsBlock currentTsBlock = null;
+ private boolean noMoreData = false;
+ private int currentIndex = 0;
+ private PartitionState state = PartitionState.INIT_STATE;
+
+ public PartitionRecognizer(
+ List<Integer> partitionChannels,
+ List<Integer> requiredChannels,
+ List<Integer> passThroughChannels,
+ List<TSDataType> inputDataTypes) {
+ this.partitionChannels = partitionChannels;
+ this.partitionValues = new ArrayList<>(partitionChannels.size());
+ for (int i = 0; i < partitionChannels.size(); i++) {
+ partitionValues.add(null);
+ }
+ this.requiredChannels = requiredChannels;
+ this.passThroughChannels = passThroughChannels;
+ this.inputDataTypes = UDFDataTypeTransformer.transformToUDFDataTypeList(inputDataTypes);
+ }
+
+ // TsBlock is sorted by partition columns already
+ public void addTsBlock(TsBlock tsBlock) {
+ if (noMoreData) {
+ throw new IllegalArgumentException(
+ "The partition handler is finished, cannot add more data.");
+ }
+ currentTsBlock = tsBlock;
+ }
+
+ /** Marks the handler as finished. */
+ public void noMoreData() {
+ noMoreData = true;
+ }
+
+ public PartitionState nextState() {
+ updateState();
+ return state;
+ }
+
+ private void updateState() {
+ switch (state.getStateType()) {
+ case INIT:
+ state = handleInitState();
+ break;
+ case NEW_PARTITION:
+ state = handleNewPartitionState();
+ break;
+ case ITERATING:
+ state = handleIteratingState();
+ break;
+ case NEED_MORE_DATA:
+ state = handleNeedMoreDataState();
+ break;
+ case FINISHED:
+ // do nothing
+ return;
+ }
+ if (PartitionState.NEED_MORE_DATA_STATE.equals(state)) {
+ currentIndex = 0;
+ }
+ }
+
+ private PartitionState handleInitState() {
+ if (currentTsBlock == null || currentTsBlock.isEmpty()) {
+ return PartitionState.INIT_STATE;
+ }
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+
+ private PartitionState handleNewPartitionState() {
+ if (currentIndex >= currentTsBlock.getPositionCount()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ } else {
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+ }
+
+ private PartitionState handleNeedMoreDataState() {
+ if (noMoreData) {
+ return PartitionState.FINISHED_STATE;
+ } else if (currentTsBlock == null || currentTsBlock.isEmpty()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ }
+ int endPartitionIndex = findNextDifferentRowIndex();
+ if (endPartitionIndex != 0) {
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.iteratingState(slice);
+ } else {
+ currentIndex = endPartitionIndex;
+ endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+ }
+
+ private PartitionState handleIteratingState() {
+ if (currentIndex >= currentTsBlock.getPositionCount()) {
+ return PartitionState.NEED_MORE_DATA_STATE;
+ } else {
+ int endPartitionIndex = findNextDifferentRowIndex();
+ Slice slice = getSlice(currentIndex, endPartitionIndex);
+ currentIndex = endPartitionIndex;
+ return PartitionState.newPartitionState(slice);
+ }
+ }
+
+ /**
+ * Find next row index whose partition values are different from the current partition values. If
+ * all rows have the same partition values, return the position count of the current TsBlock.
+ */
+ private int findNextDifferentRowIndex() {
+ int i = currentIndex;
+ while (i < currentTsBlock.getPositionCount()) {
+ for (int j = 0; j < partitionChannels.size(); j++) {
+ if (!Objects.equals(
+ partitionValues.get(j),
+ currentTsBlock.getColumn(partitionChannels.get(j)).getObject(i))) {
Review Comment:
what if index i of Column j is null? getObject won't handle this case
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/TableFunctionOperator.java:
##########
@@ -0,0 +1,256 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.execution.operator.process.function;
+
+import org.apache.iotdb.db.queryengine.execution.operator.Operator;
+import org.apache.iotdb.db.queryengine.execution.operator.OperatorContext;
+import org.apache.iotdb.db.queryengine.execution.operator.process.ProcessOperator;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.PartitionState;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.Slice;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.SliceCache;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+
+import com.google.common.util.concurrent.ListenableFuture;
+import org.apache.tsfile.block.column.Column;
+import org.apache.tsfile.block.column.ColumnBuilder;
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.read.common.block.TsBlock;
+import org.apache.tsfile.read.common.block.TsBlockBuilder;
+import org.apache.tsfile.read.common.block.column.LongColumnBuilder;
+import org.apache.tsfile.read.common.block.column.RunLengthEncodedColumn;
+
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+
+import static org.apache.iotdb.db.queryengine.execution.operator.source.relational.TableScanOperator.TIME_COLUMN_TEMPLATE;
+
+// only one input source is supported now
+public class TableFunctionOperator implements ProcessOperator {
+
+ private final OperatorContext operatorContext;
+ private final Operator inputOperator;
+ private final TableFunctionProcessorProvider processorProvider;
+ private final PartitionRecognizer partitionRecognizer;
+ private final TsBlockBuilder blockBuilder;
+ private final int properChannelCount;
+ private final boolean needPassThrough;
+
+ private TableFunctionDataProcessor processor;
+ private PartitionState partitionState;
+ private ListenableFuture<?> isBlocked;
+ private boolean finished = false;
+ private ColumnBuilder passThroughIndexBuilder;
+
+ private SliceCache sliceCache;
+
+ public TableFunctionOperator(
+ OperatorContext operatorContext,
+ TableFunctionProcessorProvider processorProvider,
+ Operator inputOperator,
+ List<TSDataType> inputDataTypes,
+ List<TSDataType> outputDataTypes,
+ int properChannelCount,
+ List<Integer> requiredChannels,
+ List<Integer> passThroughChannels,
+ List<Integer> partitionChannels) {
+ this.operatorContext = operatorContext;
+ this.inputOperator = inputOperator;
+ this.properChannelCount = properChannelCount;
+ this.processorProvider = processorProvider;
+ this.partitionRecognizer =
+ new PartitionRecognizer(
+ partitionChannels, requiredChannels, passThroughChannels, inputDataTypes);
+ this.needPassThrough = properChannelCount != outputDataTypes.size();
+ this.partitionState = null;
+ this.blockBuilder = new TsBlockBuilder(outputDataTypes);
+ this.sliceCache = new SliceCache();
+ }
+
+ @Override
+ public OperatorContext getOperatorContext() {
+ return this.operatorContext;
+ }
+
+ @Override
+ public ListenableFuture<?> isBlocked() {
+ if (isBlocked == null) {
+ isBlocked = tryGetNextTsBlock();
+ }
+ return isBlocked;
+ }
+
+ private ListenableFuture<?> tryGetNextTsBlock() {
+ try {
+ if (inputOperator.isFinished()) {
+ partitionRecognizer.noMoreData();
+ return NOT_BLOCKED;
+ }
+ if (!inputOperator.isBlocked().isDone()) {
+ return inputOperator.isBlocked();
+ }
+ if (inputOperator.hasNextWithTimer()) {
+ partitionRecognizer.addTsBlock(inputOperator.nextWithTimer());
+ }
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ return NOT_BLOCKED;
+ }
+
+ @Override
+ public TsBlock next() throws Exception {
+ PartitionState.StateType stateType = partitionState.getStateType();
+ Slice slice = partitionState.getSlice();
+ if (stateType == PartitionState.StateType.INIT
+ || stateType == PartitionState.StateType.NEED_MORE_DATA) {
+ consumeCurrentPartitionState();
+ consumeCurrentSourceTsBlock();
+ return null;
+ } else {
+ List<ColumnBuilder> properColumnBuilders = getProperColumnBuilders();
+ ColumnBuilder passThroughIndexBuilder = getPassThroughIndexBuilder();
+ if (stateType == PartitionState.StateType.FINISHED) {
+ if (processor != null) {
+ processor.finish(properColumnBuilders, passThroughIndexBuilder);
+ }
+ finished = true;
+ TsBlock tsBlock = buildTsBlock(properColumnBuilders, passThroughIndexBuilder);
+ sliceCache.clear();
+ consumeCurrentPartitionState();
+ return tsBlock;
+ }
+ if (stateType == PartitionState.StateType.NEW_PARTITION) {
+ if (processor != null) {
+ // previous partition state has not finished consuming yet
+ processor.finish(properColumnBuilders, passThroughIndexBuilder);
+ TsBlock tsBlock = buildTsBlock(properColumnBuilders, passThroughIndexBuilder);
+ sliceCache.clear();
+ processor = null;
+ return tsBlock;
+ } else {
+ processor = processorProvider.getDataProcessor();
+ }
+ }
+ sliceCache.addSlice(slice);
+ Iterator<Record> recordIterator = slice.getRequiredRecordIterator();
+ while (recordIterator.hasNext()) {
+ processor.process(recordIterator.next(), properColumnBuilders, passThroughIndexBuilder);
+ }
+ consumeCurrentPartitionState();
+ return buildTsBlock(properColumnBuilders, passThroughIndexBuilder);
+ }
+ }
+
+ private List<ColumnBuilder> getProperColumnBuilders() {
+ blockBuilder.reset();
+ return Arrays.asList(blockBuilder.getValueColumnBuilders()).subList(0, properChannelCount);
+ }
+
+ private ColumnBuilder getPassThroughIndexBuilder() {
+ return new LongColumnBuilder(null, 1);
+ }
+
+ private TsBlock buildTsBlock(
+ List<ColumnBuilder> properColumnBuilders, ColumnBuilder passThroughIndexBuilder) {
+ List<ColumnBuilder> passThroughColumnBuilders =
+ Arrays.asList(blockBuilder.getValueColumnBuilders())
+ .subList(properChannelCount, blockBuilder.getValueColumnBuilders().length);
+ int positionCount = 0;
+ if (properChannelCount > 0) {
+ // if there is proper column, use its position count
+ positionCount = properColumnBuilders.get(0).getPositionCount();
+ } else if (needPassThrough) {
+ // if there is no proper column, use pass through column's position count
+ positionCount = passThroughIndexBuilder.getPositionCount();
+ }
+ if (positionCount == 0) {
+ return null;
+ }
+ blockBuilder.declarePositions(positionCount);
+ if (needPassThrough) {
+ // handle pass through column only if needed
+ Column passThroughIndex = passThroughIndexBuilder.build();
+ for (Column[] passThroughColumns : sliceCache.getPassThroughResult(passThroughIndex)) {
+ for (int i = 0; i < passThroughColumns.length; i++) {
+ ColumnBuilder passThroughColumnBuilder = passThroughColumnBuilders.get(i);
+ for (int j = 0; j < passThroughColumns[i].getPositionCount(); j++) {
+ if (passThroughColumns[i].isNull(j)) {
+ passThroughColumnBuilder.appendNull();
+ } else {
+ passThroughColumnBuilder.write(passThroughColumns[i], j);
+ }
+ }
+ }
+ }
+ }
+ return blockBuilder.build(new RunLengthEncodedColumn(TIME_COLUMN_TEMPLATE, positionCount));
+ }
+
+ private void consumeCurrentPartitionState() {
+ partitionState = null;
+ }
+
+ private void consumeCurrentSourceTsBlock() {
+ isBlocked = null;
+ }
+
+ @Override
+ public boolean hasNext() throws Exception {
+ if (partitionState == null) {
+ isBlocked().get(); // wait for the next TsBlock
Review Comment:
we should never block in hasNext and next
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/TableFunctionOperator.java:
##########
@@ -0,0 +1,256 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.execution.operator.process.function;
+
+import org.apache.iotdb.db.queryengine.execution.operator.Operator;
+import org.apache.iotdb.db.queryengine.execution.operator.OperatorContext;
+import org.apache.iotdb.db.queryengine.execution.operator.process.ProcessOperator;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.PartitionState;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.Slice;
+import org.apache.iotdb.db.queryengine.execution.operator.process.function.partition.SliceCache;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+
+import com.google.common.util.concurrent.ListenableFuture;
+import org.apache.tsfile.block.column.Column;
+import org.apache.tsfile.block.column.ColumnBuilder;
+import org.apache.tsfile.enums.TSDataType;
+import org.apache.tsfile.read.common.block.TsBlock;
+import org.apache.tsfile.read.common.block.TsBlockBuilder;
+import org.apache.tsfile.read.common.block.column.LongColumnBuilder;
+import org.apache.tsfile.read.common.block.column.RunLengthEncodedColumn;
+
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.List;
+
+import static org.apache.iotdb.db.queryengine.execution.operator.source.relational.TableScanOperator.TIME_COLUMN_TEMPLATE;
+
+// only one input source is supported now
+public class TableFunctionOperator implements ProcessOperator {
+
+ private final OperatorContext operatorContext;
+ private final Operator inputOperator;
+ private final TableFunctionProcessorProvider processorProvider;
+ private final PartitionRecognizer partitionRecognizer;
+ private final TsBlockBuilder blockBuilder;
+ private final int properChannelCount;
+ private final boolean needPassThrough;
+
+ private TableFunctionDataProcessor processor;
+ private PartitionState partitionState;
+ private ListenableFuture<?> isBlocked;
+ private boolean finished = false;
+ private ColumnBuilder passThroughIndexBuilder;
+
+ private SliceCache sliceCache;
+
+ public TableFunctionOperator(
+ OperatorContext operatorContext,
+ TableFunctionProcessorProvider processorProvider,
+ Operator inputOperator,
+ List<TSDataType> inputDataTypes,
+ List<TSDataType> outputDataTypes,
+ int properChannelCount,
+ List<Integer> requiredChannels,
+ List<Integer> passThroughChannels,
+ List<Integer> partitionChannels) {
+ this.operatorContext = operatorContext;
+ this.inputOperator = inputOperator;
+ this.properChannelCount = properChannelCount;
+ this.processorProvider = processorProvider;
+ this.partitionRecognizer =
+ new PartitionRecognizer(
+ partitionChannels, requiredChannels, passThroughChannels, inputDataTypes);
+ this.needPassThrough = properChannelCount != outputDataTypes.size();
+ this.partitionState = null;
+ this.blockBuilder = new TsBlockBuilder(outputDataTypes);
+ this.sliceCache = new SliceCache();
+ }
+
+ @Override
+ public OperatorContext getOperatorContext() {
+ return this.operatorContext;
+ }
+
+ @Override
+ public ListenableFuture<?> isBlocked() {
+ if (isBlocked == null) {
+ isBlocked = tryGetNextTsBlock();
+ }
+ return isBlocked;
+ }
+
+ private ListenableFuture<?> tryGetNextTsBlock() {
+ try {
+ if (inputOperator.isFinished()) {
+ partitionRecognizer.noMoreData();
+ return NOT_BLOCKED;
+ }
+ if (!inputOperator.isBlocked().isDone()) {
+ return inputOperator.isBlocked();
+ }
+ if (inputOperator.hasNextWithTimer()) {
+ partitionRecognizer.addTsBlock(inputOperator.nextWithTimer());
+ }
+ } catch (Exception e) {
+ throw new RuntimeException(e);
+ }
+ return NOT_BLOCKED;
+ }
+
+ @Override
+ public TsBlock next() throws Exception {
+ PartitionState.StateType stateType = partitionState.getStateType();
+ Slice slice = partitionState.getSlice();
+ if (stateType == PartitionState.StateType.INIT
+ || stateType == PartitionState.StateType.NEED_MORE_DATA) {
+ consumeCurrentPartitionState();
+ consumeCurrentSourceTsBlock();
+ return null;
+ } else {
+ List<ColumnBuilder> properColumnBuilders = getProperColumnBuilders();
+ ColumnBuilder passThroughIndexBuilder = getPassThroughIndexBuilder();
+ if (stateType == PartitionState.StateType.FINISHED) {
+ if (processor != null) {
+ processor.finish(properColumnBuilders, passThroughIndexBuilder);
+ }
+ finished = true;
+ TsBlock tsBlock = buildTsBlock(properColumnBuilders, passThroughIndexBuilder);
+ sliceCache.clear();
+ consumeCurrentPartitionState();
+ return tsBlock;
+ }
+ if (stateType == PartitionState.StateType.NEW_PARTITION) {
+ if (processor != null) {
+ // previous partition state has not finished consuming yet
+ processor.finish(properColumnBuilders, passThroughIndexBuilder);
+ TsBlock tsBlock = buildTsBlock(properColumnBuilders, passThroughIndexBuilder);
+ sliceCache.clear();
+ processor = null;
+ return tsBlock;
+ } else {
+ processor = processorProvider.getDataProcessor();
+ }
+ }
+ sliceCache.addSlice(slice);
+ Iterator<Record> recordIterator = slice.getRequiredRecordIterator();
+ while (recordIterator.hasNext()) {
+ processor.process(recordIterator.next(), properColumnBuilders, passThroughIndexBuilder);
+ }
+ consumeCurrentPartitionState();
+ return buildTsBlock(properColumnBuilders, passThroughIndexBuilder);
+ }
+ }
+
+ private List<ColumnBuilder> getProperColumnBuilders() {
+ blockBuilder.reset();
+ return Arrays.asList(blockBuilder.getValueColumnBuilders()).subList(0, properChannelCount);
+ }
+
+ private ColumnBuilder getPassThroughIndexBuilder() {
+ return new LongColumnBuilder(null, 1);
+ }
+
+ private TsBlock buildTsBlock(
+ List<ColumnBuilder> properColumnBuilders, ColumnBuilder passThroughIndexBuilder) {
+ List<ColumnBuilder> passThroughColumnBuilders =
+ Arrays.asList(blockBuilder.getValueColumnBuilders())
+ .subList(properChannelCount, blockBuilder.getValueColumnBuilders().length);
+ int positionCount = 0;
+ if (properChannelCount > 0) {
+ // if there is proper column, use its position count
+ positionCount = properColumnBuilders.get(0).getPositionCount();
+ } else if (needPassThrough) {
+ // if there is no proper column, use pass through column's position count
+ positionCount = passThroughIndexBuilder.getPositionCount();
+ }
+ if (positionCount == 0) {
+ return null;
+ }
+ blockBuilder.declarePositions(positionCount);
+ if (needPassThrough) {
+ // handle pass through column only if needed
+ Column passThroughIndex = passThroughIndexBuilder.build();
+ for (Column[] passThroughColumns : sliceCache.getPassThroughResult(passThroughIndex)) {
+ for (int i = 0; i < passThroughColumns.length; i++) {
+ ColumnBuilder passThroughColumnBuilder = passThroughColumnBuilders.get(i);
+ for (int j = 0; j < passThroughColumns[i].getPositionCount(); j++) {
+ if (passThroughColumns[i].isNull(j)) {
+ passThroughColumnBuilder.appendNull();
+ } else {
+ passThroughColumnBuilder.write(passThroughColumns[i], j);
+ }
+ }
+ }
+ }
+ }
+ return blockBuilder.build(new RunLengthEncodedColumn(TIME_COLUMN_TEMPLATE, positionCount));
+ }
+
+ private void consumeCurrentPartitionState() {
+ partitionState = null;
+ }
+
+ private void consumeCurrentSourceTsBlock() {
+ isBlocked = null;
+ }
+
+ @Override
+ public boolean hasNext() throws Exception {
+ if (partitionState == null) {
+ isBlocked().get(); // wait for the next TsBlock
+ partitionState = partitionRecognizer.nextState();
+ }
+ return !finished;
+ }
+
+ @Override
+ public void close() throws Exception {
+ sliceCache.close();
+ inputOperator.close();
+ }
+
+ @Override
+ public boolean isFinished() throws Exception {
+ return finished;
+ }
+
+ @Override
+ public long calculateMaxPeekMemory() {
+ return 0;
+ }
+
+ @Override
+ public long calculateMaxReturnSize() {
+ return 0;
+ }
+
+ @Override
+ public long calculateRetainedSizeAfterCallingNext() {
+ return 0;
+ }
+
+ @Override
+ public long ramBytesUsed() {
+ return 0;
+ }
Review Comment:
implement these
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2692124901
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.
Cpaulyz commented on code in PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#discussion_r1984413506
##########
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/table/TableFunctionAnalysis.java:
##########
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.udf.api.relational.table;
+
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+
+import static java.util.Objects.requireNonNull;
+
+/**
+ * An object of this class is produced by the `analyze()` method of a `ConnectorTableFunction`
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.
Cpaulyz commented on code in PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#discussion_r1984414097
##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/TableBuiltinTableFunction.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational;
+
+import org.apache.iotdb.commons.udf.builtin.relational.tvf.HOPTableFunction;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+public enum TableBuiltinTableFunction {
+// TODO(UDF): support built in function
Review Comment:
Done. TableBuiltinTableFunction is empty now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.
Cpaulyz commented on code in PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#discussion_r1984426833
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/optimizations/PushLimitOffsetIntoTableScan.java:
##########
@@ -237,6 +238,16 @@ public PlanNode visitInformationSchemaTableScan(
return node;
}
+ @Override
+ public PlanNode visitTableFunctionProcessor(TableFunctionProcessorNode node, Context context) {
+ if (node.getChildren().isEmpty()) {
+ context.enablePushDown = false;
+ return node;
+ } else {
+ return visitPlan(node, context);
Review Comment:
No. Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2677003226
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Draft] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2642464544
## [Codecov](https://app.codecov.io/gh/apache/iotdb/pull/14791?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `54.46304%` with `653 lines` in your changes missing coverage. Please review.
> Project coverage is 39.37%. Comparing base [(`69d9dae`)](https://app.codecov.io/gh/apache/iotdb/commit/69d9dae6e93c3b5442474267fc319782c6449f58?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`53709c9`)](https://app.codecov.io/gh/apache/iotdb/commit/53709c9512e7860110d741bb49a1244b3d180aaa?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 7 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/iotdb/pull/14791?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...ne/plan/relational/analyzer/StatementAnalyzer.java](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Frelational%2Fanalyzer%2FStatementAnalyzer.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9yZWxhdGlvbmFsL2FuYWx5emVyL1N0YXRlbWVudEFuYWx5emVyLmphdmE=) | 67.13% | [94 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...erator/process/function/TableFunctionOperator.java](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fexecution%2Foperator%2Fprocess%2Ffunction%2FTableFunctionOperator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvZXhlY3V0aW9uL29wZXJhdG9yL3Byb2Nlc3MvZnVuY3Rpb24vVGFibGVGdW5jdGlvbk9wZXJhdG9yLmphdmE=) | 0.00% | [81 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...yengine/plan/relational/sql/util/SqlFormatter.java](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Frelational%2Fsql%2Futil%2FSqlFormatter.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9yZWxhdGlvbmFsL3NxbC91dGlsL1NxbEZvcm1hdHRlci5qYXZh) | 0.00% | [62 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...eryengine/plan/planner/TableOperatorGenerator.java](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Fplanner%2FTableOperatorGenerator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9wbGFubmVyL1RhYmxlT3BlcmF0b3JHZW5lcmF0b3IuamF2YQ==) | 0.00% | [46 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...planner/optimizations/UnaliasSymbolReferences.java](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Frelational%2Fplanner%2Foptimizations%2FUnaliasSymbolReferences.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9yZWxhdGlvbmFsL3BsYW5uZXIvb3B0aW1pemF0aW9ucy9VbmFsaWFzU3ltYm9sUmVmZXJlbmNlcy5qYXZh) | 55.07% | [31 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...relational/sql/ast/TableFunctionTableArgument.java](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Frelational%2Fsql%2Fast%2FTableFunctionTableArgument.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9yZWxhdGlvbmFsL3NxbC9hc3QvVGFibGVGdW5jdGlvblRhYmxlQXJndW1lbnQuamF2YQ==) | 24.39% | [31 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...yengine/plan/relational/sql/parser/AstBuilder.java](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Frelational%2Fsql%2Fparser%2FAstBuilder.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9yZWxhdGlvbmFsL3NxbC9wYXJzZXIvQXN0QnVpbGRlci5qYXZh) | 55.07% | [31 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ngine/plan/planner/plan/node/PlanGraphPrinter.java](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Fplanner%2Fplan%2Fnode%2FPlanGraphPrinter.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9wbGFubmVyL3BsYW4vbm9kZS9QbGFuR3JhcGhQcmludGVyLmphdmE=) | 0.00% | [29 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...an/relational/sql/ast/TableFunctionInvocation.java](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Frelational%2Fsql%2Fast%2FTableFunctionInvocation.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9yZWxhdGlvbmFsL3NxbC9hc3QvVGFibGVGdW5jdGlvbkludm9jYXRpb24uamF2YQ==) | 23.68% | [29 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ine/execution/function/table/HOPTableFunction.java](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fexecution%2Ffunction%2Ftable%2FHOPTableFunction.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvZXhlY3V0aW9uL2Z1bmN0aW9uL3RhYmxlL0hPUFRhYmxlRnVuY3Rpb24uamF2YQ==) | 59.37% | [26 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| ... and [26 more](https://app.codecov.io/gh/apache/iotdb/pull/14791?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #14791 +/- ##
============================================
+ Coverage 39.36% 39.37% +0.01%
Complexity 193 193
============================================
Files 4490 4518 +28
Lines 286232 288173 +1941
Branches 35484 35712 +228
============================================
+ Hits 112663 113481 +818
- Misses 173569 174692 +1123
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/iotdb/pull/14791?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2677884811
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2709462457
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2714599426
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.
Cpaulyz commented on code in PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#discussion_r1984413286
##########
iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/TableFunction.java:
##########
@@ -19,4 +19,69 @@
package org.apache.iotdb.udf.api.relational;
-public interface TableFunction extends SQLFunction {}
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+
+import java.util.List;
+import java.util.Map;
+
+public interface TableFunction extends SQLFunction {
+
+ /**
+ * This method is used to define the specification of the arguments required by the table
+ * function. Each argument is described using a {@link ParameterSpecification} object, which
+ * encapsulates details such as the argument name, whether it is required, its default value (if
+ * any), etc.
+ *
+ * <p>The {@link ParameterSpecification} class is abstract and has two concrete implementations:
+ *
+ * <ul>
+ * <li>{@link TableParameterSpecification}: Used for parameters specific to table functions.
+ * <li>{@link ScalarParameterSpecification}: Used for parameters specific to scalar functions.
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2713240400
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2710351433
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Draft] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2645890703
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#issuecomment-2708327092
## [](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) **Quality Gate failed**
Failed conditions
 [C Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791) (required ≥ A)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=14791)
##
 Catch issues before they fail your Quality Gate with our IDE extension  [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.
Cpaulyz commented on code in PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#discussion_r1984424985
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/sql/parser/AstBuilder.java:
##########
@@ -2197,6 +2201,122 @@ public Node visitParenthesizedRelation(RelationalSqlParser.ParenthesizedRelation
return visit(ctx.relation());
}
+ @Override
+ public Node visitTableFunctionInvocation(
+ RelationalSqlParser.TableFunctionInvocationContext context) {
+ return visit(context.tableFunctionCall());
+ }
+
+ @Override
+ public Node visitTableFunctionCall(RelationalSqlParser.TableFunctionCallContext context) {
+ QualifiedName name = getQualifiedName(context.qualifiedName());
+ List<TableFunctionArgument> arguments =
+ visit(context.tableFunctionArgument(), TableFunctionArgument.class);
+ List<List<QualifiedName>> copartitioning = ImmutableList.of();
+ if (context.COPARTITION() != null) {
Review Comment:
No, I will delete all related codes about copartition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Introduce User-defined Polymorphic Table Function [iotdb]
Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.
Cpaulyz commented on code in PR #14791:
URL: https://github.com/apache/iotdb/pull/14791#discussion_r1984436811
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/source/relational/aggregation/grouped/GroupedUserDefinedAggregateAccumulator.java:
##########
@@ -88,6 +88,10 @@ public void addInput(int[] groupIds, Column[] arguments, AggregationMask mask) {
int groupId = groupIds[selectedPositions[index]];
index++;
State state = getOrCreateState(groupId);
+ if (state == null) {
+ state = aggregateFunction.createState();
+ stateArray.set(groupId, state);
+ }
Review Comment:
Remove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org