You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@iotdb.apache.org by "Cpaulyz (via GitHub)" <gi...@apache.org> on 2025/03/19 10:19:24 UTC

[PR] Add GroupNode and parallelize TableFunctionProcessorNode 2 [iotdb]

Cpaulyz opened a new pull request, #15136:
URL: https://github.com/apache/iotdb/pull/15136

   ## Description
   
   
   ### Content1 ...
   
   ### Content2 ...
   
   ### Content3 ...
   
   <!--
   In each section, please describe design decisions made, including:
    - Choice of algorithms
    - Behavioral aspects. What configuration values are acceptable? How are corner cases and error 
       conditions handled, such as when there are insufficient resources?
    - Class organization and design (how the logic is split between classes, inheritance, composition, 
       design patterns)
    - Method organization and design (how the logic is split between methods, parameters and return types)
    - Naming (class, method, API, configuration, HTTP endpoint, names of emitted metrics)
   -->
   
   
   <!-- It's good to describe an alternative design (or mention an alternative name) for every design 
   (or naming) decision point and compare the alternatives with the designs that you've implemented 
   (or the names you've chosen) to highlight the advantages of the chosen designs and names. -->
   
   <!-- If there was a discussion of the design of the feature implemented in this PR elsewhere 
   (e. g. a "Proposal" issue, any other issue, or a thread in the development mailing list), 
   link to that discussion from this PR description and explain what have changed in your final design 
   compared to your original proposal or the consensus version in the end of the discussion. 
   If something hasn't changed since the original discussion, you can omit a detailed discussion of 
   those aspects of the design here, perhaps apart from brief mentioning for the sake of readability 
   of this PR description. -->
   
   <!-- Some of the aspects mentioned above may be omitted for simple and small changes. -->
   
   <hr>
   
   This PR has:
   - [ ] been self-reviewed.
       - [ ] concurrent read
       - [ ] concurrent write
       - [ ] concurrent read and write 
   - [ ] added documentation for new or modified features or behaviors.
   - [ ] added Javadocs for most classes and all non-trivial methods. 
   - [ ] added or updated version, __license__, or notice information
   - [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious 
     for an unfamiliar reader.
   - [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold 
     for code coverage.
   - [ ] added integration tests.
   - [ ] been tested in a test IoTDB cluster.
   
   <!-- Check the items by putting "x" in the brackets for the done things. Not all of these items 
   apply to every PR. Remove the items which are not done or not relevant to the PR. None of the items 
   from the checklist above are strictly necessary, but it would be very helpful if you at least 
   self-review the PR. -->
   
   <hr>
   
   ##### Key changed/added classes (or packages if there are too many classes) in this PR
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "JackieTien97 (via GitHub)" <gi...@apache.org>.

JackieTien97 commented on code in PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#discussion_r2011794679


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/PartitionRecognizer.java:
##########
@@ -156,15 +160,39 @@ private PartitionState handleIteratingOrNewPartitionState() {
    * all rows have the same partition values, return the position count of the current TsBlock.
    */
   private int findNextDifferentRowIndex() {
-    SortKey compareKey = new SortKey(currentTsBlock, currentIndex);
-    while (compareKey.rowIndex < currentTsBlock.getPositionCount()) {
-      if (partitionComparator.compare(partitionKey, compareKey) != 0) {
-        partitionKey = compareKey;
-        return compareKey.rowIndex;
+    int totalRows = currentTsBlock.getPositionCount();
+
+    // check if all rows have the same partition values
+    SortKey compareKey = new SortKey(currentTsBlock, totalRows - 1);
+    if (partitionComparator.compare(currentPartitionKey, compareKey) == 0) {
+      return totalRows;
+    }
+
+    // check the first row
+    compareKey.rowIndex = currentIndex;
+    if (partitionComparator.compare(currentPartitionKey, compareKey) != 0) {
+      currentPartitionKey = compareKey;
+      return currentIndex;
+    }
+
+    // binary search to find the next different partition values
+    int low = currentIndex;
+    int high = totalRows - 1;
+    int firstDiff = totalRows;
+    while (low <= high) {
+      compareKey.rowIndex = low + (high - low) / 2;
+      int cmp = partitionComparator.compare(currentPartitionKey, compareKey);
+      if (cmp == 0) {
+        low = compareKey.rowIndex + 1;

Review Comment:
   need update low? otherwise, it will be dead-loop



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/TableFunctionOperator.java:
##########
@@ -61,16 +65,19 @@ public class TableFunctionOperator implements ProcessOperator {
   private final Operator inputOperator;
   private final TableFunctionProcessorProvider processorProvider;
   private final PartitionRecognizer partitionRecognizer;
-  private final TsBlockBuilder blockBuilder;
+  private final TsBlockBuilder properBlockBuilder;
   private final int properChannelCount;
   private final boolean needPassThrough;
-  private final SliceCache sliceCache;
+  private final PartitionCache partitionCache;
+  private final boolean requireRecordSnapshot;
 
   private TableFunctionDataProcessor processor;
   private PartitionState partitionState;
   private ListenableFuture<?> isBlocked;
   private boolean finished = false;
 
+  private Queue<TsBlock> resultTsBlocks;

Review Comment:
   ```suggestion
     private final Queue<TsBlock> resultTsBlocks;
   ```



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/node/GroupNode.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.plan.relational.planner.node;
+
+import org.apache.iotdb.db.queryengine.plan.planner.plan.node.PlanNode;
+import org.apache.iotdb.db.queryengine.plan.planner.plan.node.PlanNodeId;
+import org.apache.iotdb.db.queryengine.plan.planner.plan.node.PlanNodeType;
+import org.apache.iotdb.db.queryengine.plan.planner.plan.node.PlanVisitor;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.OrderingScheme;
+
+import com.google.common.collect.Iterables;
+import org.apache.tsfile.utils.ReadWriteIOUtils;
+
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.List;
+
+/**
+ * GroupNode is a auxiliary node that is used to group data. Currently, it is implemented based on
+ * SortNode. It will only be generated some special node that required grouping source, such as
+ * FillNode and TableFunctionNode.
+ *
+ * <p>GroupNode's ordering schema consists of two parts: PartitionKey and OrderKey. It guarantees to
+ * return data grouped by PartitionKey and sorted by OrderKey. For example, PARTITION BY device_id
+ * ORDER BY time will return data grouped by device_id, and in each group, data will be sorted by
+ * time.
+ */
+public class GroupNode extends SortNode {
+
+  /**
+   * orderingScheme may include two parts: PartitionKey and OrderKey. It marks the number of
+   * PartitionKey.
+   */
+  private int partitionKeyCount;
+
+  public GroupNode(PlanNodeId id, PlanNode child, OrderingScheme scheme, int partitionKeyCount) {
+    super(id, child, scheme, false, false);
+    this.partitionKeyCount = partitionKeyCount;
+  }
+
+  @Override
+  public PlanNode replaceChildren(List<PlanNode> newChildren) {
+    return new GroupNode(
+        id, Iterables.getOnlyElement(newChildren), orderingScheme, partitionKeyCount);
+  }
+
+  public int getPartitionKeyCount() {
+    return partitionKeyCount;
+  }
+
+  @Override
+  public <R, C> R accept(PlanVisitor<R, C> visitor, C context) {
+    return visitor.visitGroup(this, context);
+  }
+
+  @Override
+  public PlanNode clone() {
+    return new GroupNode(id, null, orderingScheme, partitionKeyCount);
+  }
+
+  @Override
+  protected void serializeAttributes(ByteBuffer byteBuffer) {
+    PlanNodeType.TABLE_GROUP_NODE.serialize(byteBuffer);
+    orderingScheme.serialize(byteBuffer);

Review Comment:
   better add a new protected method in SortNode, to extract these common code together, then if SortNode add some new attributes, we won't need to change subClass of it



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.

sonarqubecloud[bot] commented on PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#issuecomment-2765406164

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [6.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=15136&metric=new_duplicated_lines_density&view=list) (required ≤ 5%)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) (required ≥ A)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136)
   
   ##   
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png '') Catch issues before they fail your Quality Gate with our IDE extension ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png '') [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.

sonarqubecloud[bot] commented on PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#issuecomment-2768833625

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [7.3% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=15136&metric=new_duplicated_lines_density&view=list) (required ≤ 5%)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) (required ≥ A)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136)
   
   ##   
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png '') Catch issues before they fail your Quality Gate with our IDE extension ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png '') [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.

Cpaulyz commented on code in PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#discussion_r2020540445


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/optimizations/ParallelizeGrouping.java:
##########
@@ -0,0 +1,270 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.plan.relational.planner.optimizations;
+
+import org.apache.iotdb.db.queryengine.plan.planner.plan.node.PlanNode;
+import org.apache.iotdb.db.queryengine.plan.planner.plan.node.PlanVisitor;
+import org.apache.iotdb.db.queryengine.plan.relational.analyzer.Analysis;
+import org.apache.iotdb.db.queryengine.plan.relational.metadata.ColumnSchema;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.DataOrganizationSpecification;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.OrderingScheme;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.Symbol;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.AggregationNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.AggregationTableScanNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.CorrelatedJoinNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.DeviceTableScanNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.GroupNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.JoinNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.SemiJoinNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.SortNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.StreamSortNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.TableFunctionProcessorNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.TopKNode;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import static org.apache.iotdb.commons.schema.table.column.TsTableColumnCategory.TAG;
+import static org.apache.iotdb.db.queryengine.plan.relational.planner.optimizations.ParallelizeGrouping.CanOptimized.ENABLE_PARALLEL;
+import static org.apache.iotdb.db.queryengine.plan.relational.planner.optimizations.ParallelizeGrouping.CanOptimized.KEEP_GROUP;
+import static org.apache.iotdb.db.queryengine.plan.relational.planner.optimizations.ParallelizeGrouping.CanOptimized.PENDING;
+import static org.apache.iotdb.db.queryengine.plan.relational.planner.optimizations.ParallelizeGrouping.CanOptimized.TO_SORT;
+
+/**
+ * This rule is used to determine whether the GroupNode can be executed parallel.
+ *
+ * <p>Optimization phase: Logical plan planning.
+ *
+ * <p>
+ *
+ * <ul>
+ *   The GroupNode can be paralleled if the lasted offspring that guarantees the data is grouped by
+ *   PartitionKey or is sorted by PartitionKey. For example:
+ *   <ul>
+ *     <li>GroupNode[PK={device_id}, OK={time}] -> ... -> TableDeviceScanNode
+ *     <li>GroupNode[PK={device_id}, OK={s1}] -> ... -> TableDeviceScanNode
+ *     <li>GroupNode[PK={device_id,attr}, OK={time}] -> ... -> TableDeviceScanNode
+ *     <li>GroupNode[PK={tag1,tag2}, OK={tag3}] -> SortNode[sort={tag1,tag2,tag3}]
+ *     <li>GroupNode[PK={tag1,tag2}, OK={s1}] -> TopKNode[sort={tag1,tag2,tag3}]
+ *     <li>GroupNode[PK={tag1,tag2}, OK={}] -> AggregationNode[group={tag1,tag2}]
+ *     <li>GroupNode[PK={tag1,tag2}, OK={s1}] -> AggregationNode[group={tag1,tag2,s1}]
+ *     <li>GroupNode[PK={tag1}, OK={}] -> TableFunctionNode[partition={tag1,tag2}]
+ *   </ul>
+ * </ul>
+ *
+ * <p>Otherwise, the GroupNode should be transformed into a SortNode.
+ */
+public class ParallelizeGrouping implements PlanOptimizer {
+  @Override
+  public PlanNode optimize(PlanNode plan, PlanOptimizer.Context context) {
+    if (!(context.getAnalysis().isQuery())) {

Review Comment:
   How can I know if query plan is with GroupNode or not?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "JackieTien97 (via GitHub)" <gi...@apache.org>.

JackieTien97 merged PR #15136:
URL: https://github.com/apache/iotdb/pull/15136


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.

Cpaulyz commented on code in PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#discussion_r2020558973


##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/CapacityTableFunction.java:
##########
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+public class CapacityTableFunction implements TableFunction {
+  private static final String DATA_PARAMETER_NAME = "DATA";
+  private static final String SIZE_PARAMETER_NAME = "SIZE";
+
+  @Override
+  public List<ParameterSpecification> getArgumentsSpecifications() {
+    return Arrays.asList(
+        TableParameterSpecification.builder()
+            .name(DATA_PARAMETER_NAME)
+            .passThroughColumns()
+            .build(),
+        ScalarParameterSpecification.builder().name(SIZE_PARAMETER_NAME).type(Type.INT64).build());
+  }
+
+  @Override
+  public TableFunctionAnalysis analyze(Map<String, Argument> arguments) throws UDFException {
+    long size = (long) ((ScalarArgument) arguments.get(SIZE_PARAMETER_NAME)).getValue();
+    if (size <= 0) {
+      throw new UDFException("Size must be greater than 0");
+    }
+    return TableFunctionAnalysis.builder()
+        .properColumnSchema(
+            new DescribedSchema.Builder().addField("window_index", Type.INT64).build())
+        .requireRecordSnapshot(false)
+        .requiredColumns(DATA_PARAMETER_NAME, Collections.singletonList(0))
+        .build();
+  }
+
+  @Override
+  public TableFunctionProcessorProvider getProcessorProvider(Map<String, Argument> arguments) {
+    long sz = (long) ((ScalarArgument) arguments.get(SIZE_PARAMETER_NAME)).getValue();
+    return new TableFunctionProcessorProvider() {
+      @Override
+      public TableFunctionDataProcessor getDataProcessor() {
+        return new CountDataProcessor(sz);
+      }
+    };
+  }
+
+  private static class CountDataProcessor implements TableFunctionDataProcessor {
+
+    private final long size;
+    private final List<Long> currentRowIndexes = new ArrayList<>();

Review Comment:
   Good idea! Change ` List<Long> currentRowIndexes ` to `long currentStartIndex`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode 2 [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.

sonarqubecloud[bot] commented on PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#issuecomment-2736073395

   ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) **Quality Gate passed**  
   Issues  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [16 New issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=15136&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_iotdb&pullRequest=15136&issueStatuses=ACCEPTED)
   
   Measures  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=15136&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=15136&metric=new_coverage&view=list)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [3.3% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=15136&metric=new_duplicated_lines_density&view=list)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.

sonarqubecloud[bot] commented on PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#issuecomment-2747155398

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) (required ≥ A)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136)
   
   ##   
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png '') Catch issues before they fail your Quality Gate with our IDE extension ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png '') [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.

sonarqubecloud[bot] commented on PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#issuecomment-2765471449

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) (required ≥ A)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136)
   
   ##   
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png '') Catch issues before they fail your Quality Gate with our IDE extension ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png '') [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.

sonarqubecloud[bot] commented on PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#issuecomment-2748360928

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [6.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=15136&metric=new_duplicated_lines_density&view=list) (required ≤ 5%)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) (required ≥ A)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136)
   
   ##   
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png '') Catch issues before they fail your Quality Gate with our IDE extension ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png '') [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "JackieTien97 (via GitHub)" <gi...@apache.org>.

JackieTien97 commented on code in PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#discussion_r2020469456


##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/CapacityTableFunction.java:
##########
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+public class CapacityTableFunction implements TableFunction {
+  private static final String DATA_PARAMETER_NAME = "DATA";
+  private static final String SIZE_PARAMETER_NAME = "SIZE";
+
+  @Override
+  public List<ParameterSpecification> getArgumentsSpecifications() {
+    return Arrays.asList(
+        TableParameterSpecification.builder()
+            .name(DATA_PARAMETER_NAME)
+            .passThroughColumns()
+            .build(),
+        ScalarParameterSpecification.builder().name(SIZE_PARAMETER_NAME).type(Type.INT64).build());
+  }
+
+  @Override
+  public TableFunctionAnalysis analyze(Map<String, Argument> arguments) throws UDFException {
+    long size = (long) ((ScalarArgument) arguments.get(SIZE_PARAMETER_NAME)).getValue();
+    if (size <= 0) {
+      throw new UDFException("Size must be greater than 0");
+    }
+    return TableFunctionAnalysis.builder()
+        .properColumnSchema(
+            new DescribedSchema.Builder().addField("window_index", Type.INT64).build())
+        .requireRecordSnapshot(false)
+        .requiredColumns(DATA_PARAMETER_NAME, Collections.singletonList(0))
+        .build();
+  }
+
+  @Override
+  public TableFunctionProcessorProvider getProcessorProvider(Map<String, Argument> arguments) {
+    long sz = (long) ((ScalarArgument) arguments.get(SIZE_PARAMETER_NAME)).getValue();
+    return new TableFunctionProcessorProvider() {
+      @Override
+      public TableFunctionDataProcessor getDataProcessor() {
+        return new CountDataProcessor(sz);
+      }
+    };
+  }
+
+  private static class CountDataProcessor implements TableFunctionDataProcessor {
+
+    private final long size;
+    private final List<Long> currentRowIndexes = new ArrayList<>();

Review Comment:
   there is no need to save this list? we can simplely record a curStartIndex and then [startStartIndex, curIndex] is for passThroughIndexBuilder



##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/WindowTVFUtils.java:
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.type.Type;
+
+import java.util.Optional;
+import java.util.Set;
+
+public class WindowTVFUtils {
+  /**
+   * Find the index of the column in the table argument.
+   *
+   * @param tableArgument the table argument
+   * @param expectedFieldName the expected field name
+   * @param expectedTypes the expected types
+   * @return the index of the time column, -1 if not found
+   */
+  public static int findColumnIndex(
+      TableArgument tableArgument, String expectedFieldName, Set<Type> expectedTypes)
+      throws UDFException {
+    for (int i = 0; i < tableArgument.getFieldTypes().size(); i++) {
+      Optional<String> fieldName = tableArgument.getFieldNames().get(i);
+      if (fieldName.isPresent() && expectedFieldName.equalsIgnoreCase(fieldName.get())) {
+        if (!expectedTypes.contains(tableArgument.getFieldTypes().get(i))) {
+          throw new UDFException(
+              String.format("The type of the column [%s] is not as expected.", expectedFieldName));
+        }
+        return i;
+      }
+    }
+    throw new UDFException(
+        String.format(
+            "Required column [%s] not found in the source table argument.", expectedFieldName));

Review Comment:
   ```suggestion
       throw new IoTDBRuntimeException(
           String.format(
               "Required column [%s] not found in the source table argument.", expectedFieldName), TSStatusCode.SEMANTIC_ERROR.getStatusCode());
   ```



##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/SessionTableFunction.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.iotdb.commons.udf.builtin.relational.tvf.WindowTVFUtils.findColumnIndex;
+
+public class SessionTableFunction implements TableFunction {
+  private static final String DATA_PARAMETER_NAME = "DATA";
+  private static final String TIMECOL_PARAMETER_NAME = "TIMECOL";
+  private static final String GAP_PARAMETER_NAME = "GAP";
+
+  @Override
+  public List<ParameterSpecification> getArgumentsSpecifications() {
+    return Arrays.asList(
+        TableParameterSpecification.builder()
+            .name(DATA_PARAMETER_NAME)
+            .passThroughColumns()
+            .build(),
+        ScalarParameterSpecification.builder()
+            .name(TIMECOL_PARAMETER_NAME)
+            .type(Type.STRING)
+            .build(),
+        ScalarParameterSpecification.builder().name(GAP_PARAMETER_NAME).type(Type.INT64).build());
+  }
+
+  @Override
+  public TableFunctionAnalysis analyze(Map<String, Argument> arguments) throws UDFException {
+    TableArgument tableArgument = (TableArgument) arguments.get(DATA_PARAMETER_NAME);
+    String expectedFieldName =
+        (String) ((ScalarArgument) arguments.get(TIMECOL_PARAMETER_NAME)).getValue();
+    int requiredIndex =
+        findColumnIndex(tableArgument, expectedFieldName, Collections.singleton(Type.TIMESTAMP));
+    DescribedSchema properColumnSchema =
+        new DescribedSchema.Builder()
+            .addField("window_start", Type.TIMESTAMP)
+            .addField("window_end", Type.TIMESTAMP)
+            .build();
+
+    // outputColumnSchema
+    return TableFunctionAnalysis.builder()
+        .properColumnSchema(properColumnSchema)
+        .requireRecordSnapshot(false)
+        .requiredColumns(DATA_PARAMETER_NAME, Collections.singletonList(requiredIndex))
+        .build();
+  }
+
+  @Override
+  public TableFunctionProcessorProvider getProcessorProvider(Map<String, Argument> arguments) {
+    long gap = (long) ((ScalarArgument) arguments.get(GAP_PARAMETER_NAME)).getValue();
+    return new TableFunctionProcessorProvider() {
+      @Override
+      public TableFunctionDataProcessor getDataProcessor() {
+        return new SessionDataProcessor(gap);
+      }
+    };
+  }
+
+  private static class SessionDataProcessor implements TableFunctionDataProcessor {
+
+    private final long gap;
+    private final List<Long> currentRowIndexes = new ArrayList<>();

Review Comment:
   no need to use list to save all



##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/WindowTVFUtils.java:
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.type.Type;
+
+import java.util.Optional;
+import java.util.Set;
+
+public class WindowTVFUtils {
+  /**
+   * Find the index of the column in the table argument.
+   *
+   * @param tableArgument the table argument
+   * @param expectedFieldName the expected field name
+   * @param expectedTypes the expected types
+   * @return the index of the time column, -1 if not found
+   */
+  public static int findColumnIndex(
+      TableArgument tableArgument, String expectedFieldName, Set<Type> expectedTypes)
+      throws UDFException {
+    for (int i = 0; i < tableArgument.getFieldTypes().size(); i++) {
+      Optional<String> fieldName = tableArgument.getFieldNames().get(i);
+      if (fieldName.isPresent() && expectedFieldName.equalsIgnoreCase(fieldName.get())) {
+        if (!expectedTypes.contains(tableArgument.getFieldTypes().get(i))) {
+          throw new UDFException(
+              String.format("The type of the column [%s] is not as expected.", expectedFieldName));

Review Comment:
   ```suggestion
   throw new IoTDBRuntimeException(String.format("The type of the column [%s] is not as expected.", expectedFieldName), TSStatusCode.SEMANTIC_ERROR.getStatusCode());
   ```



##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/SessionTableFunction.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.iotdb.commons.udf.builtin.relational.tvf.WindowTVFUtils.findColumnIndex;
+
+public class SessionTableFunction implements TableFunction {
+  private static final String DATA_PARAMETER_NAME = "DATA";
+  private static final String TIMECOL_PARAMETER_NAME = "TIMECOL";
+  private static final String GAP_PARAMETER_NAME = "GAP";
+
+  @Override
+  public List<ParameterSpecification> getArgumentsSpecifications() {
+    return Arrays.asList(
+        TableParameterSpecification.builder()
+            .name(DATA_PARAMETER_NAME)
+            .passThroughColumns()
+            .build(),
+        ScalarParameterSpecification.builder()
+            .name(TIMECOL_PARAMETER_NAME)
+            .type(Type.STRING)
+            .build(),
+        ScalarParameterSpecification.builder().name(GAP_PARAMETER_NAME).type(Type.INT64).build());
+  }
+
+  @Override
+  public TableFunctionAnalysis analyze(Map<String, Argument> arguments) throws UDFException {
+    TableArgument tableArgument = (TableArgument) arguments.get(DATA_PARAMETER_NAME);
+    String expectedFieldName =
+        (String) ((ScalarArgument) arguments.get(TIMECOL_PARAMETER_NAME)).getValue();
+    int requiredIndex =
+        findColumnIndex(tableArgument, expectedFieldName, Collections.singleton(Type.TIMESTAMP));
+    DescribedSchema properColumnSchema =
+        new DescribedSchema.Builder()
+            .addField("window_start", Type.TIMESTAMP)
+            .addField("window_end", Type.TIMESTAMP)
+            .build();
+
+    // outputColumnSchema
+    return TableFunctionAnalysis.builder()
+        .properColumnSchema(properColumnSchema)
+        .requireRecordSnapshot(false)
+        .requiredColumns(DATA_PARAMETER_NAME, Collections.singletonList(requiredIndex))
+        .build();
+  }
+
+  @Override
+  public TableFunctionProcessorProvider getProcessorProvider(Map<String, Argument> arguments) {
+    long gap = (long) ((ScalarArgument) arguments.get(GAP_PARAMETER_NAME)).getValue();
+    return new TableFunctionProcessorProvider() {
+      @Override
+      public TableFunctionDataProcessor getDataProcessor() {
+        return new SessionDataProcessor(gap);
+      }
+    };
+  }
+
+  private static class SessionDataProcessor implements TableFunctionDataProcessor {
+
+    private final long gap;
+    private final List<Long> currentRowIndexes = new ArrayList<>();
+    private long curIndex = 0;
+    private long windowStart = -1;
+    private long windowEnd = -1;

Review Comment:
   what if time is negative? better using Long.MIN_VALUE



##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/VariationTableFunction.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+import org.apache.iotdb.udf.api.type.Type;
+
+import com.google.common.collect.ImmutableSet;
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.iotdb.commons.udf.builtin.relational.tvf.WindowTVFUtils.findColumnIndex;
+
+public class VariationTableFunction implements TableFunction {
+  private static final String DATA_PARAMETER_NAME = "DATA";
+  private static final String COL_PARAMETER_NAME = "COL";
+  private static final String DELTA_PARAMETER_NAME = "DELTA";
+
+  @Override
+  public List<ParameterSpecification> getArgumentsSpecifications() {
+    return Arrays.asList(
+        TableParameterSpecification.builder()
+            .name(DATA_PARAMETER_NAME)
+            .passThroughColumns()
+            .build(),
+        ScalarParameterSpecification.builder().name(COL_PARAMETER_NAME).type(Type.STRING).build(),
+        ScalarParameterSpecification.builder()
+            .name(DELTA_PARAMETER_NAME)
+            .type(Type.DOUBLE)
+            .build());
+  }
+
+  @Override
+  public TableFunctionAnalysis analyze(Map<String, Argument> arguments) throws UDFException {
+    TableArgument tableArgument = (TableArgument) arguments.get(DATA_PARAMETER_NAME);
+    String expectedFieldName =
+        (String) ((ScalarArgument) arguments.get(COL_PARAMETER_NAME)).getValue();
+    int requiredIndex =
+        findColumnIndex(
+            tableArgument,
+            expectedFieldName,
+            ImmutableSet.of(Type.INT32, Type.INT64, Type.FLOAT, Type.DOUBLE));
+    DescribedSchema properColumnSchema =
+        new DescribedSchema.Builder().addField("window_index", Type.INT64).build();
+    // outputColumnSchema
+    return TableFunctionAnalysis.builder()
+        .properColumnSchema(properColumnSchema)
+        .requireRecordSnapshot(false)
+        .requiredColumns(DATA_PARAMETER_NAME, Collections.singletonList(requiredIndex))
+        .build();
+  }
+
+  @Override
+  public TableFunctionProcessorProvider getProcessorProvider(Map<String, Argument> arguments) {
+    double delta = (double) ((ScalarArgument) arguments.get(DELTA_PARAMETER_NAME)).getValue();
+    return new TableFunctionProcessorProvider() {
+      @Override
+      public TableFunctionDataProcessor getDataProcessor() {
+        return new VariationDataProcessor(delta);
+      }
+    };
+  }
+
+  private static class VariationDataProcessor implements TableFunctionDataProcessor {
+
+    private final double gap;
+    private final List<Long> currentRowIndexes = new ArrayList<>();

Review Comment:
   no need to be a list



##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/SessionTableFunction.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.iotdb.commons.udf.builtin.relational.tvf.WindowTVFUtils.findColumnIndex;
+
+public class SessionTableFunction implements TableFunction {
+  private static final String DATA_PARAMETER_NAME = "DATA";
+  private static final String TIMECOL_PARAMETER_NAME = "TIMECOL";
+  private static final String GAP_PARAMETER_NAME = "GAP";
+
+  @Override
+  public List<ParameterSpecification> getArgumentsSpecifications() {
+    return Arrays.asList(
+        TableParameterSpecification.builder()
+            .name(DATA_PARAMETER_NAME)
+            .passThroughColumns()
+            .build(),
+        ScalarParameterSpecification.builder()
+            .name(TIMECOL_PARAMETER_NAME)
+            .type(Type.STRING)
+            .build(),
+        ScalarParameterSpecification.builder().name(GAP_PARAMETER_NAME).type(Type.INT64).build());
+  }
+
+  @Override
+  public TableFunctionAnalysis analyze(Map<String, Argument> arguments) throws UDFException {
+    TableArgument tableArgument = (TableArgument) arguments.get(DATA_PARAMETER_NAME);
+    String expectedFieldName =
+        (String) ((ScalarArgument) arguments.get(TIMECOL_PARAMETER_NAME)).getValue();
+    int requiredIndex =
+        findColumnIndex(tableArgument, expectedFieldName, Collections.singleton(Type.TIMESTAMP));
+    DescribedSchema properColumnSchema =
+        new DescribedSchema.Builder()
+            .addField("window_start", Type.TIMESTAMP)
+            .addField("window_end", Type.TIMESTAMP)
+            .build();
+
+    // outputColumnSchema
+    return TableFunctionAnalysis.builder()
+        .properColumnSchema(properColumnSchema)
+        .requireRecordSnapshot(false)
+        .requiredColumns(DATA_PARAMETER_NAME, Collections.singletonList(requiredIndex))
+        .build();
+  }
+
+  @Override
+  public TableFunctionProcessorProvider getProcessorProvider(Map<String, Argument> arguments) {
+    long gap = (long) ((ScalarArgument) arguments.get(GAP_PARAMETER_NAME)).getValue();
+    return new TableFunctionProcessorProvider() {
+      @Override
+      public TableFunctionDataProcessor getDataProcessor() {
+        return new SessionDataProcessor(gap);
+      }
+    };
+  }
+
+  private static class SessionDataProcessor implements TableFunctionDataProcessor {
+
+    private final long gap;
+    private final List<Long> currentRowIndexes = new ArrayList<>();
+    private long curIndex = 0;
+    private long windowStart = -1;
+    private long windowEnd = -1;
+
+    public SessionDataProcessor(long gap) {
+      this.gap = gap;
+    }
+
+    @Override
+    public void process(
+        Record input,
+        List<ColumnBuilder> properColumnBuilders,
+        ColumnBuilder passThroughIndexBuilder) {
+      long timeValue = input.getLong(0);
+      if (timeValue > windowEnd) {
+        outputWindow(properColumnBuilders, passThroughIndexBuilder);
+      }
+      if (currentRowIndexes.isEmpty()) {
+        windowStart = timeValue;
+      }
+      currentRowIndexes.add(curIndex);
+      windowEnd = timeValue + gap;
+      curIndex++;
+    }
+
+    @Override
+    public void finish(List<ColumnBuilder> columnBuilders, ColumnBuilder passThroughIndexBuilder) {
+      if (!currentRowIndexes.isEmpty()) {
+        outputWindow(columnBuilders, passThroughIndexBuilder);
+      }
+    }
+
+    private void outputWindow(
+        List<ColumnBuilder> properColumnBuilders, ColumnBuilder passThroughIndexBuilder) {
+      for (Long currentRowIndex : currentRowIndexes) {
+        properColumnBuilders.get(0).writeLong(windowStart);
+        properColumnBuilders.get(1).writeLong(windowEnd - gap);
+        passThroughIndexBuilder.writeLong(currentRowIndex);
+      }

Review Comment:
   ```suggestion
         long currentWindowEnd = windowEnd - gap;
         for (Long currentRowIndex : currentRowIndexes) {
           properColumnBuilders.get(0).writeLong(windowStart);
           properColumnBuilders.get(1).writeLong(currentWindowEnd);
           passThroughIndexBuilder.writeLong(currentRowIndex);
         }
   ```



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/optimizations/ParallelizeGrouping.java:
##########
@@ -0,0 +1,270 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.plan.relational.planner.optimizations;
+
+import org.apache.iotdb.db.queryengine.plan.planner.plan.node.PlanNode;
+import org.apache.iotdb.db.queryengine.plan.planner.plan.node.PlanVisitor;
+import org.apache.iotdb.db.queryengine.plan.relational.analyzer.Analysis;
+import org.apache.iotdb.db.queryengine.plan.relational.metadata.ColumnSchema;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.DataOrganizationSpecification;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.OrderingScheme;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.Symbol;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.AggregationNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.AggregationTableScanNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.CorrelatedJoinNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.DeviceTableScanNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.GroupNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.JoinNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.SemiJoinNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.SortNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.StreamSortNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.TableFunctionProcessorNode;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.node.TopKNode;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import static org.apache.iotdb.commons.schema.table.column.TsTableColumnCategory.TAG;
+import static org.apache.iotdb.db.queryengine.plan.relational.planner.optimizations.ParallelizeGrouping.CanOptimized.ENABLE_PARALLEL;
+import static org.apache.iotdb.db.queryengine.plan.relational.planner.optimizations.ParallelizeGrouping.CanOptimized.KEEP_GROUP;
+import static org.apache.iotdb.db.queryengine.plan.relational.planner.optimizations.ParallelizeGrouping.CanOptimized.PENDING;
+import static org.apache.iotdb.db.queryengine.plan.relational.planner.optimizations.ParallelizeGrouping.CanOptimized.TO_SORT;
+
+/**
+ * This rule is used to determine whether the GroupNode can be executed parallel.
+ *
+ * <p>Optimization phase: Logical plan planning.
+ *
+ * <p>
+ *
+ * <ul>
+ *   The GroupNode can be paralleled if the lasted offspring that guarantees the data is grouped by
+ *   PartitionKey or is sorted by PartitionKey. For example:
+ *   <ul>
+ *     <li>GroupNode[PK={device_id}, OK={time}] -> ... -> TableDeviceScanNode
+ *     <li>GroupNode[PK={device_id}, OK={s1}] -> ... -> TableDeviceScanNode
+ *     <li>GroupNode[PK={device_id,attr}, OK={time}] -> ... -> TableDeviceScanNode
+ *     <li>GroupNode[PK={tag1,tag2}, OK={tag3}] -> SortNode[sort={tag1,tag2,tag3}]
+ *     <li>GroupNode[PK={tag1,tag2}, OK={s1}] -> TopKNode[sort={tag1,tag2,tag3}]
+ *     <li>GroupNode[PK={tag1,tag2}, OK={}] -> AggregationNode[group={tag1,tag2}]
+ *     <li>GroupNode[PK={tag1,tag2}, OK={s1}] -> AggregationNode[group={tag1,tag2,s1}]
+ *     <li>GroupNode[PK={tag1}, OK={}] -> TableFunctionNode[partition={tag1,tag2}]
+ *   </ul>
+ * </ul>
+ *
+ * <p>Otherwise, the GroupNode should be transformed into a SortNode.
+ */
+public class ParallelizeGrouping implements PlanOptimizer {
+  @Override
+  public PlanNode optimize(PlanNode plan, PlanOptimizer.Context context) {
+    if (!(context.getAnalysis().isQuery())) {

Review Comment:
   only query plan with `GroupNode` should continue



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.

Cpaulyz commented on code in PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#discussion_r2013358673


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/process/function/TableFunctionOperator.java:
##########
@@ -61,16 +65,19 @@ public class TableFunctionOperator implements ProcessOperator {
   private final Operator inputOperator;
   private final TableFunctionProcessorProvider processorProvider;
   private final PartitionRecognizer partitionRecognizer;
-  private final TsBlockBuilder blockBuilder;
+  private final TsBlockBuilder properBlockBuilder;
   private final int properChannelCount;
   private final boolean needPassThrough;
-  private final SliceCache sliceCache;
+  private final PartitionCache partitionCache;
+  private final boolean requireRecordSnapshot;
 
   private TableFunctionDataProcessor processor;
   private PartitionState partitionState;
   private ListenableFuture<?> isBlocked;
   private boolean finished = false;
 
+  private Queue<TsBlock> resultTsBlocks;

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.

Cpaulyz commented on PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#issuecomment-2753187757

   > ...org/apache/iotdb/db/queryengine/execution/operator/process/function/PartitionRecognizer.java
   
   GroupNode is a subClass of SortNode, so I think it projection push down will be executed in `·ProjectOffPushDownRule<SortNode>`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.

Cpaulyz commented on code in PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#discussion_r2013360504


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/node/GroupNode.java:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.queryengine.plan.relational.planner.node;
+
+import org.apache.iotdb.db.queryengine.plan.planner.plan.node.PlanNode;
+import org.apache.iotdb.db.queryengine.plan.planner.plan.node.PlanNodeId;
+import org.apache.iotdb.db.queryengine.plan.planner.plan.node.PlanNodeType;
+import org.apache.iotdb.db.queryengine.plan.planner.plan.node.PlanVisitor;
+import org.apache.iotdb.db.queryengine.plan.relational.planner.OrderingScheme;
+
+import com.google.common.collect.Iterables;
+import org.apache.tsfile.utils.ReadWriteIOUtils;
+
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.List;
+
+/**
+ * GroupNode is a auxiliary node that is used to group data. Currently, it is implemented based on
+ * SortNode. It will only be generated some special node that required grouping source, such as
+ * FillNode and TableFunctionNode.
+ *
+ * <p>GroupNode's ordering schema consists of two parts: PartitionKey and OrderKey. It guarantees to
+ * return data grouped by PartitionKey and sorted by OrderKey. For example, PARTITION BY device_id
+ * ORDER BY time will return data grouped by device_id, and in each group, data will be sorted by
+ * time.
+ */
+public class GroupNode extends SortNode {
+
+  /**
+   * orderingScheme may include two parts: PartitionKey and OrderKey. It marks the number of
+   * PartitionKey.
+   */
+  private int partitionKeyCount;
+
+  public GroupNode(PlanNodeId id, PlanNode child, OrderingScheme scheme, int partitionKeyCount) {
+    super(id, child, scheme, false, false);
+    this.partitionKeyCount = partitionKeyCount;
+  }
+
+  @Override
+  public PlanNode replaceChildren(List<PlanNode> newChildren) {
+    return new GroupNode(
+        id, Iterables.getOnlyElement(newChildren), orderingScheme, partitionKeyCount);
+  }
+
+  public int getPartitionKeyCount() {
+    return partitionKeyCount;
+  }
+
+  @Override
+  public <R, C> R accept(PlanVisitor<R, C> visitor, C context) {
+    return visitor.visitGroup(this, context);
+  }
+
+  @Override
+  public PlanNode clone() {
+    return new GroupNode(id, null, orderingScheme, partitionKeyCount);
+  }
+
+  @Override
+  protected void serializeAttributes(ByteBuffer byteBuffer) {
+    PlanNodeType.TABLE_GROUP_NODE.serialize(byteBuffer);
+    orderingScheme.serialize(byteBuffer);

Review Comment:
   StreamSortNode has the same issue. Actually, there are some variables in SortNode is not used in StreamSortNode and GroupNode. So if SortNode add some new attributes, maybe it won't affact the logic of subClass?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "Cpaulyz (via GitHub)" <gi...@apache.org>.

Cpaulyz commented on code in PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#discussion_r2020565659


##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/SessionTableFunction.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.iotdb.commons.udf.builtin.relational.tvf.WindowTVFUtils.findColumnIndex;
+
+public class SessionTableFunction implements TableFunction {
+  private static final String DATA_PARAMETER_NAME = "DATA";
+  private static final String TIMECOL_PARAMETER_NAME = "TIMECOL";
+  private static final String GAP_PARAMETER_NAME = "GAP";
+
+  @Override
+  public List<ParameterSpecification> getArgumentsSpecifications() {
+    return Arrays.asList(
+        TableParameterSpecification.builder()
+            .name(DATA_PARAMETER_NAME)
+            .passThroughColumns()
+            .build(),
+        ScalarParameterSpecification.builder()
+            .name(TIMECOL_PARAMETER_NAME)
+            .type(Type.STRING)
+            .build(),
+        ScalarParameterSpecification.builder().name(GAP_PARAMETER_NAME).type(Type.INT64).build());
+  }
+
+  @Override
+  public TableFunctionAnalysis analyze(Map<String, Argument> arguments) throws UDFException {
+    TableArgument tableArgument = (TableArgument) arguments.get(DATA_PARAMETER_NAME);
+    String expectedFieldName =
+        (String) ((ScalarArgument) arguments.get(TIMECOL_PARAMETER_NAME)).getValue();
+    int requiredIndex =
+        findColumnIndex(tableArgument, expectedFieldName, Collections.singleton(Type.TIMESTAMP));
+    DescribedSchema properColumnSchema =
+        new DescribedSchema.Builder()
+            .addField("window_start", Type.TIMESTAMP)
+            .addField("window_end", Type.TIMESTAMP)
+            .build();
+
+    // outputColumnSchema
+    return TableFunctionAnalysis.builder()
+        .properColumnSchema(properColumnSchema)
+        .requireRecordSnapshot(false)
+        .requiredColumns(DATA_PARAMETER_NAME, Collections.singletonList(requiredIndex))
+        .build();
+  }
+
+  @Override
+  public TableFunctionProcessorProvider getProcessorProvider(Map<String, Argument> arguments) {
+    long gap = (long) ((ScalarArgument) arguments.get(GAP_PARAMETER_NAME)).getValue();
+    return new TableFunctionProcessorProvider() {
+      @Override
+      public TableFunctionDataProcessor getDataProcessor() {
+        return new SessionDataProcessor(gap);
+      }
+    };
+  }
+
+  private static class SessionDataProcessor implements TableFunctionDataProcessor {
+
+    private final long gap;
+    private final List<Long> currentRowIndexes = new ArrayList<>();
+    private long curIndex = 0;
+    private long windowStart = -1;
+    private long windowEnd = -1;

Review Comment:
   Fixed



##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/SessionTableFunction.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.iotdb.commons.udf.builtin.relational.tvf.WindowTVFUtils.findColumnIndex;
+
+public class SessionTableFunction implements TableFunction {
+  private static final String DATA_PARAMETER_NAME = "DATA";
+  private static final String TIMECOL_PARAMETER_NAME = "TIMECOL";
+  private static final String GAP_PARAMETER_NAME = "GAP";
+
+  @Override
+  public List<ParameterSpecification> getArgumentsSpecifications() {
+    return Arrays.asList(
+        TableParameterSpecification.builder()
+            .name(DATA_PARAMETER_NAME)
+            .passThroughColumns()
+            .build(),
+        ScalarParameterSpecification.builder()
+            .name(TIMECOL_PARAMETER_NAME)
+            .type(Type.STRING)
+            .build(),
+        ScalarParameterSpecification.builder().name(GAP_PARAMETER_NAME).type(Type.INT64).build());
+  }
+
+  @Override
+  public TableFunctionAnalysis analyze(Map<String, Argument> arguments) throws UDFException {
+    TableArgument tableArgument = (TableArgument) arguments.get(DATA_PARAMETER_NAME);
+    String expectedFieldName =
+        (String) ((ScalarArgument) arguments.get(TIMECOL_PARAMETER_NAME)).getValue();
+    int requiredIndex =
+        findColumnIndex(tableArgument, expectedFieldName, Collections.singleton(Type.TIMESTAMP));
+    DescribedSchema properColumnSchema =
+        new DescribedSchema.Builder()
+            .addField("window_start", Type.TIMESTAMP)
+            .addField("window_end", Type.TIMESTAMP)
+            .build();
+
+    // outputColumnSchema
+    return TableFunctionAnalysis.builder()
+        .properColumnSchema(properColumnSchema)
+        .requireRecordSnapshot(false)
+        .requiredColumns(DATA_PARAMETER_NAME, Collections.singletonList(requiredIndex))
+        .build();
+  }
+
+  @Override
+  public TableFunctionProcessorProvider getProcessorProvider(Map<String, Argument> arguments) {
+    long gap = (long) ((ScalarArgument) arguments.get(GAP_PARAMETER_NAME)).getValue();
+    return new TableFunctionProcessorProvider() {
+      @Override
+      public TableFunctionDataProcessor getDataProcessor() {
+        return new SessionDataProcessor(gap);
+      }
+    };
+  }
+
+  private static class SessionDataProcessor implements TableFunctionDataProcessor {
+
+    private final long gap;
+    private final List<Long> currentRowIndexes = new ArrayList<>();

Review Comment:
   Fixed



##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/SessionTableFunction.java:
##########
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.commons.udf.builtin.relational.tvf;
+
+import org.apache.iotdb.udf.api.exception.UDFException;
+import org.apache.iotdb.udf.api.relational.TableFunction;
+import org.apache.iotdb.udf.api.relational.access.Record;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionAnalysis;
+import org.apache.iotdb.udf.api.relational.table.TableFunctionProcessorProvider;
+import org.apache.iotdb.udf.api.relational.table.argument.Argument;
+import org.apache.iotdb.udf.api.relational.table.argument.DescribedSchema;
+import org.apache.iotdb.udf.api.relational.table.argument.ScalarArgument;
+import org.apache.iotdb.udf.api.relational.table.argument.TableArgument;
+import org.apache.iotdb.udf.api.relational.table.processor.TableFunctionDataProcessor;
+import org.apache.iotdb.udf.api.relational.table.specification.ParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.ScalarParameterSpecification;
+import org.apache.iotdb.udf.api.relational.table.specification.TableParameterSpecification;
+import org.apache.iotdb.udf.api.type.Type;
+
+import org.apache.tsfile.block.column.ColumnBuilder;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.iotdb.commons.udf.builtin.relational.tvf.WindowTVFUtils.findColumnIndex;
+
+public class SessionTableFunction implements TableFunction {
+  private static final String DATA_PARAMETER_NAME = "DATA";
+  private static final String TIMECOL_PARAMETER_NAME = "TIMECOL";
+  private static final String GAP_PARAMETER_NAME = "GAP";
+
+  @Override
+  public List<ParameterSpecification> getArgumentsSpecifications() {
+    return Arrays.asList(
+        TableParameterSpecification.builder()
+            .name(DATA_PARAMETER_NAME)
+            .passThroughColumns()
+            .build(),
+        ScalarParameterSpecification.builder()
+            .name(TIMECOL_PARAMETER_NAME)
+            .type(Type.STRING)
+            .build(),
+        ScalarParameterSpecification.builder().name(GAP_PARAMETER_NAME).type(Type.INT64).build());
+  }
+
+  @Override
+  public TableFunctionAnalysis analyze(Map<String, Argument> arguments) throws UDFException {
+    TableArgument tableArgument = (TableArgument) arguments.get(DATA_PARAMETER_NAME);
+    String expectedFieldName =
+        (String) ((ScalarArgument) arguments.get(TIMECOL_PARAMETER_NAME)).getValue();
+    int requiredIndex =
+        findColumnIndex(tableArgument, expectedFieldName, Collections.singleton(Type.TIMESTAMP));
+    DescribedSchema properColumnSchema =
+        new DescribedSchema.Builder()
+            .addField("window_start", Type.TIMESTAMP)
+            .addField("window_end", Type.TIMESTAMP)
+            .build();
+
+    // outputColumnSchema
+    return TableFunctionAnalysis.builder()
+        .properColumnSchema(properColumnSchema)
+        .requireRecordSnapshot(false)
+        .requiredColumns(DATA_PARAMETER_NAME, Collections.singletonList(requiredIndex))
+        .build();
+  }
+
+  @Override
+  public TableFunctionProcessorProvider getProcessorProvider(Map<String, Argument> arguments) {
+    long gap = (long) ((ScalarArgument) arguments.get(GAP_PARAMETER_NAME)).getValue();
+    return new TableFunctionProcessorProvider() {
+      @Override
+      public TableFunctionDataProcessor getDataProcessor() {
+        return new SessionDataProcessor(gap);
+      }
+    };
+  }
+
+  private static class SessionDataProcessor implements TableFunctionDataProcessor {
+
+    private final long gap;
+    private final List<Long> currentRowIndexes = new ArrayList<>();
+    private long curIndex = 0;
+    private long windowStart = -1;
+    private long windowEnd = -1;
+
+    public SessionDataProcessor(long gap) {
+      this.gap = gap;
+    }
+
+    @Override
+    public void process(
+        Record input,
+        List<ColumnBuilder> properColumnBuilders,
+        ColumnBuilder passThroughIndexBuilder) {
+      long timeValue = input.getLong(0);
+      if (timeValue > windowEnd) {
+        outputWindow(properColumnBuilders, passThroughIndexBuilder);
+      }
+      if (currentRowIndexes.isEmpty()) {
+        windowStart = timeValue;
+      }
+      currentRowIndexes.add(curIndex);
+      windowEnd = timeValue + gap;
+      curIndex++;
+    }
+
+    @Override
+    public void finish(List<ColumnBuilder> columnBuilders, ColumnBuilder passThroughIndexBuilder) {
+      if (!currentRowIndexes.isEmpty()) {
+        outputWindow(columnBuilders, passThroughIndexBuilder);
+      }
+    }
+
+    private void outputWindow(
+        List<ColumnBuilder> properColumnBuilders, ColumnBuilder passThroughIndexBuilder) {
+      for (Long currentRowIndex : currentRowIndexes) {
+        properColumnBuilders.get(0).writeLong(windowStart);
+        properColumnBuilders.get(1).writeLong(windowEnd - gap);
+        passThroughIndexBuilder.writeLong(currentRowIndex);
+      }

Review Comment:
   Fixed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "JackieTien97 (via GitHub)" <gi...@apache.org>.

JackieTien97 commented on code in PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#discussion_r2020878700


##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/builtin/relational/tvf/CapacityTableFunction.java:
##########
@@ -96,28 +95,25 @@ public void process(
         Record input,
         List<ColumnBuilder> properColumnBuilders,
         ColumnBuilder passThroughIndexBuilder) {
-      if (currentRowIndexes.size() >= size) {
+      if (windowIndex - curIndex == size) {

Review Comment:
   ```suggestion
         if (curIndex - currentStartIndex == size) {
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode 2 [iotdb]

Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.

codecov[bot] commented on PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#issuecomment-2736263890

   ## [Codecov](https://app.codecov.io/gh/apache/iotdb/pull/15136?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   Attention: Patch coverage is `44.00786%` with `285 lines` in your changes missing coverage. Please review.
   > Project coverage is 39.35%. Comparing base [(`db614fb`)](https://app.codecov.io/gh/apache/iotdb/commit/db614fb8edae04d7bf37e5538b388011f14e2a23?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`c3c3e39`)](https://app.codecov.io/gh/apache/iotdb/commit/c3c3e39e322694fd0f39186bc474662f77e24ecd?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/iotdb/pull/15136?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [...f/builtin/relational/tvf/SessionTableFunction.java](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&filepath=iotdb-core%2Fnode-commons%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fcommons%2Fudf%2Fbuiltin%2Frelational%2Ftvf%2FSessionTableFunction.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9ub2RlLWNvbW1vbnMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2lvdGRiL2NvbW1vbnMvdWRmL2J1aWx0aW4vcmVsYXRpb25hbC90dmYvU2Vzc2lvblRhYmxlRnVuY3Rpb24uamF2YQ==) | 0.00% | [53 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...nal/planner/optimizations/ParallelizeGrouping.java](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Frelational%2Fplanner%2Foptimizations%2FParallelizeGrouping.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9yZWxhdGlvbmFsL3BsYW5uZXIvb3B0aW1pemF0aW9ucy9QYXJhbGxlbGl6ZUdyb3VwaW5nLmphdmE=) | 56.66% | [52 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...builtin/relational/tvf/VariationTableFunction.java](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&filepath=iotdb-core%2Fnode-commons%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fcommons%2Fudf%2Fbuiltin%2Frelational%2Ftvf%2FVariationTableFunction.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9ub2RlLWNvbW1vbnMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2lvdGRiL2NvbW1vbnMvdWRmL2J1aWx0aW4vcmVsYXRpb25hbC90dmYvVmFyaWF0aW9uVGFibGVGdW5jdGlvbi5qYXZh) | 0.00% | [51 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [.../builtin/relational/tvf/CapacityTableFunction.java](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&filepath=iotdb-core%2Fnode-commons%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fcommons%2Fudf%2Fbuiltin%2Frelational%2Ftvf%2FCapacityTableFunction.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9ub2RlLWNvbW1vbnMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2lvdGRiL2NvbW1vbnMvdWRmL2J1aWx0aW4vcmVsYXRpb25hbC90dmYvQ2FwYWNpdHlUYWJsZUZ1bmN0aW9uLmphdmE=) | 0.00% | [39 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...ngine/plan/planner/plan/node/PlanGraphPrinter.java](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Fplanner%2Fplan%2Fnode%2FPlanGraphPrinter.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9wbGFubmVyL3BsYW4vbm9kZS9QbGFuR3JhcGhQcmludGVyLmphdmE=) | 0.00% | [13 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...engine/plan/relational/planner/node/GroupNode.java](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Frelational%2Fplanner%2Fnode%2FGroupNode.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9yZWxhdGlvbmFsL3BsYW5uZXIvbm9kZS9Hcm91cE5vZGUuamF2YQ==) | 38.09% | [13 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...nner/distribute/TableDistributedPlanGenerator.java](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Frelational%2Fplanner%2Fdistribute%2FTableDistributedPlanGenerator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9yZWxhdGlvbmFsL3BsYW5uZXIvZGlzdHJpYnV0ZS9UYWJsZURpc3RyaWJ1dGVkUGxhbkdlbmVyYXRvci5qYXZh) | 89.52% | [11 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...ptimizations/TransformAggregationToStreamable.java](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fplan%2Frelational%2Fplanner%2Foptimizations%2FTransformAggregationToStreamable.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvcGxhbi9yZWxhdGlvbmFsL3BsYW5uZXIvb3B0aW1pemF0aW9ucy9UcmFuc2Zvcm1BZ2dyZWdhdGlvblRvU3RyZWFtYWJsZS5qYXZh) | 25.00% | [9 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [.../builtin/relational/TableBuiltinTableFunction.java](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&filepath=iotdb-core%2Fnode-commons%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fcommons%2Fudf%2Fbuiltin%2Frelational%2FTableBuiltinTableFunction.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9ub2RlLWNvbW1vbnMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2lvdGRiL2NvbW1vbnMvdWRmL2J1aWx0aW4vcmVsYXRpb25hbC9UYWJsZUJ1aWx0aW5UYWJsZUZ1bmN0aW9uLmphdmE=) | 0.00% | [9 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...erator/process/function/TableFunctionOperator.java](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&filepath=iotdb-core%2Fdatanode%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fiotdb%2Fdb%2Fqueryengine%2Fexecution%2Foperator%2Fprocess%2Ffunction%2FTableFunctionOperator.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aW90ZGItY29yZS9kYXRhbm9kZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaW90ZGIvZGIvcXVlcnllbmdpbmUvZXhlY3V0aW9uL29wZXJhdG9yL3Byb2Nlc3MvZnVuY3Rpb24vVGFibGVGdW5jdGlvbk9wZXJhdG9yLmphdmE=) | 0.00% | [7 Missing :warning: ](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | ... and [9 more](https://app.codecov.io/gh/apache/iotdb/pull/15136?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   ```diff
   @@            Coverage Diff             @@
   ##             master   #15136    +/-   ##
   ==========================================
     Coverage     39.34%   39.35%            
     Complexity      193      193            
   ==========================================
     Files          4598     4604     +6     
     Lines        295862   296322   +460     
     Branches      36883    36933    +50     
   ==========================================
   + Hits         116421   116612   +191     
   - Misses       179441   179710   +269     
   ```
   
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/iotdb/pull/15136?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   
   <details><summary>🚀 New features to boost your workflow: </summary>
   
   - ❄ [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode 2 [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.

sonarqubecloud[bot] commented on PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#issuecomment-2747005669

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [2 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_iotdb&pullRequest=15136&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) (required ≥ A)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136)
   
   ##   
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png '') Catch issues before they fail your Quality Gate with our IDE extension ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png '') [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.

sonarqubecloud[bot] commented on PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#issuecomment-2748986489

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [6.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=15136&metric=new_duplicated_lines_density&view=list) (required ≤ 5%)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) (required ≥ A)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136)
   
   ##   
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png '') Catch issues before they fail your Quality Gate with our IDE extension ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png '') [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add GroupNode and parallelize TableFunctionProcessorNode [iotdb]

Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.

sonarqubecloud[bot] commented on PR #15136:
URL: https://github.com/apache/iotdb/pull/15136#issuecomment-2753211917

   ## [![Quality Gate Failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-failed-20px.png 'Quality Gate Failed')](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) **Quality Gate failed**  
   Failed conditions  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [6.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_iotdb&pullRequest=15136&metric=new_duplicated_lines_density&view=list) (required ≤ 5%)  
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/failed-16px.png '') [B Reliability Rating on New Code](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136) (required ≥ A)  
     
   [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_iotdb&pullRequest=15136)
   
   ##   
   ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/light_bulb-16px.png '') Catch issues before they fail your Quality Gate with our IDE extension ![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/sonarlint-16px.png '') [SonarQube for IDE](https://www.sonarsource.com/products/sonarlint/features/connected-mode/?referrer=pull-request)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org