You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iotdb.apache.org by ha...@apache.org on 2024/11/28 04:06:11 UTC

(iotdb) branch refector_py_table_session updated (44b462a5082 -> b03412bd791)

This is an automated email from the ASF dual-hosted git repository.

haonan pushed a change to branch refector_py_table_session
in repository https://gitbox.apache.org/repos/asf/iotdb.git


    from 44b462a5082 fix IT
     add 559b58a2183 Enhance repair data file scan util (#14167)
     add 68daea60b0a Pipe/Subscription: Filter out non-working DR PipeTasks in CN & Reduce model judgement cost in PipeHistoricalDataRegionTsFileAndDeletionExtractor (#14059)
     add e0c6cf63821 Fix bug in inner join of table model (#14198)
     add c7e2d8e67c5 Optimize insertRelationalTablet performance (#14197)
     add 293746efae3 Refactor mod-file-related FileMetric (#14199)
     add 95d06f6de16 Fix AsyncClientManager selector thread leak #14203
     add 098306cbb9c Optimize the implantation of EXPLAIN ANALYZE
     add bf2c5b134f6 Fix upper-case database name in TableSession error
     add bb99334754f Pipe: implement RenameDatabaseProcessor & WriteBackConnector for table model events (#14131)
     add 0160b65d67b Pipe: make exception message more friendly when creating data sync pipe failed (#14208)
     add d9552365dc2 Set fetchSize to max_tsblock_line_number if it's equal to or less than zero
     add 8a6405cdfa7 [remove datanode] Do not allow regions to inherit the Removing state from datanode (#14185)
     add 77a21a71e7f Fixed the bugs of the table cache
     add b072f9e861c Fix throw exception when compare with null
     add d751364acb3 Refactor UDF management process to support table model user-defined function
     add 8618464d861 Fix table model insert without measurement error (#14222)
     add 5b7f1c66260 Fix flaky LastFlushTimeMapTest (#14225)
     add 864b94edf76 Add extra info for TableScanOperator and AggTableScanOperator in Explain Analyze
     add add7f323f4b Fix query failure if a table deletion involves deviceId with null & Fix status code when deleting illegal time range & support is null in deletion (#14217)
     add f367c729a1b Register TableSchema to TsfileWriter when recover from wal (#14228)
     add 93438081109 Add extra info for HashAggregationOperator and StreamingHashAggregationOperator in Explain Analyze
     add 2b548342ea2 Fixed the bug that the exception may print stack when dropping id or time column
     add f15c30b97fa Fix issue that data is incomplete in full join of table model and add new ExchangeNode for table model
     add 0d8a0d101bf IoTConsensusV2: Fix syncLag error when using steam mode and 2+ replicas (#14219)
     add 0bb51c67c71 IoTConsensusV2: Fix reqBuffer reset and receive concurrent bug (#14215)
     add 4fddd584b20 [IoTConsensusV2 X Pipe Deletion]: Release deletion resource when delete region & Improve for release when buffer is null (#14227)
     add b34b714c80e [IoTConsensusV2 X Region Migration] Start consensus pipe in proper time rather than automatically start it when create it
     new b03412bd791 Merge branch 'master' of github.com:apache/iotdb into refector_py_table_session

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .github/workflows/pipe-it-2cluster.yml             |  18 ++
 .../confignode/it/IoTDBConfigNodeSnapshotIT.java   |  14 +-
 .../it/partition/IoTDBPartitionCreationIT.java     |  59 +++-
 .../it/partition/IoTDBPartitionDurableIT.java      | 130 --------
 .../org/apache/iotdb/db/it/IoTDBRestServiceIT.java |   8 +-
 .../it/IoTDBSyntaxConventionStringLiteralIT.java   |   2 +-
 .../iotdb/db/it/udaf/IoTDBUDAFManagementIT.java    |  12 +-
 .../iotdb/db/it/udf/IoTDBUDFManagementIT.java      |  17 +-
 .../pipe/it/tablemodel/IoTDBPipeDataSinkIT.java    |  59 ++++
 .../relational/it/db/it/IoTDBDeletionTableIT.java  |  71 +++++
 .../db/it/IoTDBMultiIDsWithAttributesTableIT.java  | 148 ++++++++-
 .../it/query/old/IoTDBFilterTableIT.java           |  24 ++
 .../iotdb/relational/it/schema/IoTDBTableIT.java   |   3 +-
 .../it/session/IoTDBTableModelSessionIT.java       |  39 +++
 .../iotdb/session/it/IoTDBSessionRelationalIT.java |  32 ++
 .../udf/api/relational/AggregationFunction.java    |   4 +-
 .../iotdb/udf/api/relational/SQLFunction.java      |   4 +-
 .../iotdb/udf/api/relational/ScalarFunction.java   |   4 +-
 .../iotdb/udf/api/relational/TableFunction.java    |   4 +-
 .../Row.java => relational/access/Record.java}     |  13 +-
 .../java/org/apache/iotdb/rpc/TSStatusCode.java    |   2 +
 .../heartbeat/DataNodeHeartbeatHandler.java        |  14 +-
 .../consensus/request/ConfigPhysicalPlan.java      |  10 +-
 .../consensus/request/ConfigPhysicalPlanType.java  |   5 +-
 ...TablePlan.java => GetAllFunctionTablePlan.java} |   6 +-
 .../read/function/GetFunctionTablePlan.java        |  10 +-
 ...onPlan.java => DropTableModelFunctionPlan.java} |  12 +-
 ...ionPlan.java => DropTreeModelFunctionPlan.java} |  12 +-
 ...teFunctionPlan.java => UpdateFunctionPlan.java} |  45 +--
 .../response/function/FunctionTableResp.java       |  14 +-
 .../iotdb/confignode/manager/ConfigManager.java    |  10 +-
 .../apache/iotdb/confignode/manager/IManager.java  |   6 +-
 .../iotdb/confignode/manager/UDFManager.java       |  95 ++++--
 .../manager/load/cache/AbstractLoadCache.java      |   2 +-
 .../confignode/manager/load/cache/LoadCache.java   |   8 +
 .../load/cache/region/RegionGroupCache.java        |   4 +
 .../iotdb/confignode/manager/node/NodeManager.java |   2 +-
 .../iotdb/confignode/persistence/UDFInfo.java      |  43 ++-
 .../persistence/executor/ConfigPlanExecutor.java   |  20 +-
 .../persistence/schema/ClusterSchemaInfo.java      |   3 +
 .../confignode/persistence/schema/ConfigMTree.java |   2 +-
 .../procedure/env/RemoveDataNodeHandler.java       |  27 +-
 .../impl/pipe/AbstractOperatePipeProcedureV2.java  |  79 ++++-
 .../impl/pipe/task/CreatePipeProcedureV2.java      |   4 +-
 ...bstractOperateSubscriptionAndPipeProcedure.java |   4 +-
 .../thrift/ConfigNodeRPCServiceProcessor.java      |   7 +-
 .../request/ConfigPhysicalPlanSerDeTest.java       |  22 +-
 .../response/ConvertToThriftRespTest.java          |  19 +-
 .../iotdb/confignode/persistence/UDFInfoTest.java  |   6 +-
 .../iot/client/IoTConsensusClientPool.java         |   5 +-
 .../apache/iotdb/consensus/pipe/PipeConsensus.java |  13 +-
 .../consensus/pipe/PipeConsensusServerImpl.java    |  24 +-
 .../consensuspipe/ConsensusPipeDispatcher.java     |   3 +-
 .../pipe/consensuspipe/ConsensusPipeManager.java   |  18 +-
 .../pipe/metric/PipeConsensusSyncLagManager.java   |  88 +++---
 .../service/PipeConsensusRPCServiceProcessor.java  |   3 +-
 .../iotdb/consensus/ratis/RatisConsensus.java      |   3 +-
 .../PipeDataRegionProcessorConstructor.java        |   4 +
 .../protocol/writeback/WriteBackConnector.java     | 120 ++++----
 .../consensus/ConsensusPipeDataNodeDispatcher.java |   4 +-
 .../deletion/persist/PageCacheDeletionBuffer.java  |   4 +-
 .../db/pipe/event/common/PipeInsertionEvent.java   |   4 +
 .../dataregion/DataRegionListeningFilter.java      |  28 ++
 ...oricalDataRegionTsFileAndDeletionExtractor.java |  69 ++---
 .../schemachange/RenameDatabaseProcessor.java      | 101 ++++++
 .../pipeconsensus/PipeConsensusReceiver.java       |  27 +-
 .../iotdb/db/protocol/client/ConfigNodeClient.java |   5 +-
 .../protocol/client/DataNodeClientPoolFactory.java |   5 +-
 .../protocol/thrift/impl/ClientRPCServiceImpl.java |   4 +-
 .../impl/DataNodeInternalRPCServiceImpl.java       |   7 +-
 .../common/header/ColumnHeaderConstant.java        |   4 +-
 .../execution/aggregation/UDAFAccumulator.java     |   2 +-
 .../execution/operator/OperatorContext.java        |   4 +-
 .../TableAggregationTableScanOperator.java         |  45 +--
 .../relational/TableFullOuterJoinOperator.java     | 292 ++++--------------
 .../source/relational/TableInnerJoinOperator.java  | 182 +++++++----
 .../source/relational/TableScanOperator.java       |  44 ++-
 .../grouped/HashAggregationOperator.java           |   2 +
 .../grouped/StreamingAggregationOperator.java      |   5 +-
 .../grouped/StreamingHashAggregationOperator.java  |  18 +-
 .../builder/InMemoryHashAggregationBuilder.java    |  13 +
 .../db/queryengine/plan/analyze/AnalyzeUtils.java  |  56 +++-
 .../plan/execution/config/ConfigExecution.java     |   2 +-
 .../execution/config/TreeConfigTaskVisitor.java    |   5 +-
 .../config/executor/ClusterConfigTaskExecutor.java |  76 +++--
 .../config/executor/IConfigTaskExecutor.java       |  14 +-
 .../config/metadata/CreateFunctionTask.java        |  28 +-
 .../config/metadata/DropFunctionTask.java          |  10 +-
 .../config/metadata/ShowFunctionsTask.java         | 102 +++++--
 .../TableModelStatementMemorySourceVisitor.java    |   7 +-
 .../plan/expression/multi/FunctionExpression.java  |   4 +-
 .../db/queryengine/plan/parser/ASTVisitor.java     |   9 +-
 .../plan/planner/TableOperatorGenerator.java       |   6 +-
 .../planner/distribution/ExchangeNodeAdder.java    |   4 +
 .../plan/planner/plan/node/ExplainAnalyzeNode.java |  26 +-
 .../plan/planner/plan/node/PlanGraphPrinter.java   |  29 +-
 .../plan/planner/plan/node/PlanNodeType.java       |   9 +-
 .../plan/planner/plan/node/PlanVisitor.java        |  11 +
 .../planner/plan/node/process/ExchangeNode.java    |   9 +-
 .../planner/plan/node/write/InsertTabletNode.java  |  12 +-
 .../node/write/RelationalInsertTabletNode.java     |  88 ++++--
 .../plan/relational/analyzer/Analysis.java         |   8 +-
 .../plan/relational/analyzer/Analyzer.java         |   2 +-
 .../relational/analyzer/StatementAnalyzer.java     |   1 +
 .../function/arithmetic/AdditionResolver.java      |  14 +
 .../function/arithmetic/DivisionResolver.java      |  10 +
 .../function/arithmetic/ModulusResolver.java       |  10 +
 .../arithmetic/MultiplicationResolver.java         |  10 +
 .../function/arithmetic/SubtractionResolver.java   |  10 +
 .../relational/metadata/TableMetadataImpl.java     |  17 +-
 .../plan/relational/planner/RelationPlanner.java   |   3 +
 .../relational/planner/TableLogicalPlanner.java    |  10 +-
 .../planner/distribute/AddExchangeNodes.java       |  25 +-
 .../planner/distribute/SubPlanGenerator.java       |   2 +-
 .../distribute/TableDistributedPlanGenerator.java  |  23 +-
 .../distribute/TableDistributedPlanner.java        |   2 +-
 .../distribute/TableModelQueryFragmentPlanner.java |   6 +-
 .../TableModelTypeProviderExtractor.java           |   4 +-
 .../planner/node}/ExchangeNode.java                |  72 +++--
 .../{OffsetNode.java => ExplainAnalyzeNode.java}   |  91 +++---
 .../optimizations/PushPredicateIntoTableScan.java  |  12 +-
 .../optimizations/UnaliasSymbolReferences.java     |   2 +-
 .../plan/relational/sql/ast/ExplainAnalyze.java    |   5 -
 .../plan/relational/sql/ast/InsertTablet.java      |   3 +
 .../queryengine/plan/relational/sql/ast/Query.java |   5 -
 .../plan/relational/sql/ast/Statement.java         |   4 -
 .../plan/statement/crud/InsertTabletStatement.java |  10 +
 .../metadata/CreateFunctionStatement.java          |  24 +-
 .../dag/column/leaf/NullColumnTransformer.java     |   4 +-
 .../AbstractCastFunctionColumnTransformer.java     |   4 +-
 .../dag/udf/UDAFInformationInferrer.java           |   2 +-
 .../transformation/dag/udf/UDTFExecutor.java       |   2 +-
 .../dag/udf/UDTFInformationInferrer.java           |   2 +-
 .../db/schemaengine/table/DataNodeTableCache.java  |   3 +-
 .../java/org/apache/iotdb/db/service/DataNode.java |  16 +-
 .../db/storageengine/dataregion/DataRegion.java    |  24 +-
 .../CompactionStatisticsCheckFailedException.java  |  87 ++++++
 .../task/RepairUnsortedFileCompactionTask.java     |   4 +-
 .../compaction/execute/utils/CompactionUtils.java  |   4 -
 .../compaction/repair/RepairDataFileScanUtil.java  | 338 +++++++++++++++------
 .../repair/RepairTimePartitionScanTask.java        |   4 +-
 .../dataregion/flush/MemTableFlushTask.java        |   2 +-
 .../dataregion/memtable/AbstractMemTable.java      |   7 +-
 .../memtable/AlignedWritableMemChunk.java          |  18 +-
 .../memtable/AlignedWritableMemChunkGroup.java     |  10 +-
 .../dataregion/memtable/IWritableMemChunk.java     |   5 +-
 .../dataregion/memtable/TsFileProcessor.java       |   1 +
 .../dataregion/memtable/WritableMemChunk.java      |   2 +-
 .../dataregion/modification/DeletionPredicate.java |  22 +-
 .../dataregion/modification/IDPredicate.java       |  80 ++---
 .../dataregion/modification/ModEntry.java          |  16 +-
 .../dataregion/modification/ModificationFile.java  |  31 +-
 .../modification/PartitionLevelModFileManager.java |   2 -
 .../modification/TableDeletionEntry.java           |  14 +-
 .../dataregion/modification/TreeDeletionEntry.java |  14 +-
 .../dataregion/tsfile/TsFileResource.java          |   8 +-
 .../file/UnsealedTsFileRecoverPerformer.java       |  22 ++
 .../apache/iotdb/db/utils/QueryDataSetUtils.java   |   6 +
 .../iotdb/db/utils/io/BufferSerializable.java      |   2 +-
 .../iotdb/db/utils/io/StreamSerializable.java      |   2 +-
 .../iotdb/db/metadata/path/PatternTreeMapTest.java |  27 +-
 .../plan/relational/analyzer/AnalyzerTest.java     |   2 +-
 .../plan/relational/analyzer/JoinTest.java         |   2 +-
 .../analyzer/LimitOffsetPushDownTest.java          |   2 +-
 .../plan/relational/analyzer/SortTest.java         |   2 +-
 .../plan/relational/analyzer/SubQueryTest.java     |   2 +-
 .../plan/relational/analyzer/TestUtils.java        |   2 +-
 .../planner/assertions/ExchangeNodeMatcher.java    |   2 +-
 .../planner/assertions/PlanMatchPattern.java       |   2 +-
 .../planner/assertions/PlanMatchingVisitor.java    |   4 +-
 .../storageengine/dataregion/DataRegionTest.java   |  29 +-
 .../dataregion/LastFlushTimeMapTest.java           |  13 +-
 .../inner/InnerSpaceCompactionSelectorTest.java    |   2 +-
 .../repair/RepairDataFileScanUtilTest.java         | 128 +++++++-
 .../modification/ModificationFileTest.java         |   8 +-
 .../file/UnsealedTsFileRecoverPerformerTest.java   |  47 +++
 .../apache/iotdb/commons/client/ClientManager.java |  11 +-
 .../iotdb/commons/client/ClientPoolFactory.java    |  35 +--
 .../iotdb/commons/client/IClientPoolFactory.java   |   4 +-
 .../client/factory/AsyncThriftClientFactory.java   |   6 +
 .../apache/iotdb/commons/conf/IoTDBConstant.java   |   2 +
 .../org/apache/iotdb/commons/path/PartialPath.java |   3 +-
 .../apache/iotdb/commons/path/PatternTreeMap.java  |   2 +-
 .../agent/plugin/builtin/BuiltinPipePlugin.java    |   3 +
 .../RenameDatabaseProcessor.java}                  |  10 +-
 .../config/constant/PipeProcessorConstant.java     |   2 +
 .../connector/client/IoTDBSyncClientManager.java   |  25 +-
 .../schema/table/TsTableInternalRPCUtil.java       |   6 +-
 .../apache/iotdb/commons/udf/UDFInformation.java   |  64 ++--
 .../org/apache/iotdb/commons/udf/UDFTable.java     | 105 ++++---
 .../java/org/apache/iotdb/commons/udf/UDFType.java |  71 +++++
 .../BuiltinTimeSeriesGeneratingFunction.java       |  13 +
 .../commons/udf/service/UDFExecutableManager.java  |  46 +++
 .../commons/udf/service/UDFManagementService.java  | 286 +++++++----------
 .../iotdb/commons/udf/utils/TreeUDFUtils.java}     |  39 +--
 .../iotdb/commons/client/ClientManagerTest.java    |  11 +-
 .../thrift-commons/src/main/thrift/common.thrift   |   5 +
 .../src/main/thrift/confignode.thrift              |   9 +-
 .../src/main/thrift/datanode.thrift                |   1 +
 199 files changed, 3181 insertions(+), 1810 deletions(-)
 copy integration-test/src/main/java/org/apache/iotdb/itbase/category/AIClusterIT.java => iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/AggregationFunction.java (88%)
 copy integration-test/src/main/java/org/apache/iotdb/itbase/category/ClusterIT.java => iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/SQLFunction.java (91%)
 copy integration-test/src/main/java/org/apache/iotdb/itbase/category/AIClusterIT.java => iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/ScalarFunction.java (89%)
 copy integration-test/src/main/java/org/apache/iotdb/itbase/category/AIClusterIT.java => iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/relational/TableFunction.java (89%)
 copy iotdb-api/udf-api/src/main/java/org/apache/iotdb/udf/api/{access/Row.java => relational/access/Record.java} (95%)
 copy iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/read/function/{GetFunctionTablePlan.java => GetAllFunctionTablePlan.java} (86%)
 copy iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/write/function/{DropFunctionPlan.java => DropTableModelFunctionPlan.java} (85%)
 rename iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/write/function/{DropFunctionPlan.java => DropTreeModelFunctionPlan.java} (85%)
 copy iotdb-core/confignode/src/main/java/org/apache/iotdb/confignode/consensus/request/write/function/{CreateFunctionPlan.java => UpdateFunctionPlan.java} (65%)
 create mode 100644 iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/processor/schemachange/RenameDatabaseProcessor.java
 copy iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/{planner/plan/node/process => relational/planner/node}/ExchangeNode.java (78%)
 copy iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/node/{OffsetNode.java => ExplainAnalyzeNode.java} (51%)
 create mode 100644 iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/compaction/execute/exception/CompactionStatisticsCheckFailedException.java
 copy iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/pipe/agent/plugin/builtin/processor/{twostage/TwoStageCountProcessor.java => schemachange/RenameDatabaseProcessor.java} (76%)
 create mode 100644 iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/udf/UDFType.java
 copy iotdb-core/{datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/security/Identity.java => node-commons/src/main/java/org/apache/iotdb/commons/udf/utils/TreeUDFUtils.java} (59%)


(iotdb) 01/01: Merge branch 'master' of github.com:apache/iotdb into refector_py_table_session

Posted by ha...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

haonan pushed a commit to branch refector_py_table_session
in repository https://gitbox.apache.org/repos/asf/iotdb.git

commit b03412bd791c865b708db68d43d7b32e89902307
Merge: 44b462a5082 b34b714c80e
Author: HTHou <hh...@outlook.com>
AuthorDate: Thu Nov 28 12:05:14 2024 +0800

    Merge branch 'master' of github.com:apache/iotdb into refector_py_table_session

 .github/workflows/pipe-it-2cluster.yml             |  18 ++
 .../confignode/it/IoTDBConfigNodeSnapshotIT.java   |  14 +-
 .../it/partition/IoTDBPartitionCreationIT.java     |  59 +++-
 .../it/partition/IoTDBPartitionDurableIT.java      | 130 --------
 .../org/apache/iotdb/db/it/IoTDBRestServiceIT.java |   8 +-
 .../it/IoTDBSyntaxConventionStringLiteralIT.java   |   2 +-
 .../iotdb/db/it/udaf/IoTDBUDAFManagementIT.java    |  12 +-
 .../iotdb/db/it/udf/IoTDBUDFManagementIT.java      |  17 +-
 .../pipe/it/tablemodel/IoTDBPipeDataSinkIT.java    |  59 ++++
 .../relational/it/db/it/IoTDBDeletionTableIT.java  |  71 +++++
 .../db/it/IoTDBMultiIDsWithAttributesTableIT.java  | 148 ++++++++-
 .../it/query/old/IoTDBFilterTableIT.java           |  24 ++
 .../iotdb/relational/it/schema/IoTDBTableIT.java   |   3 +-
 .../it/session/IoTDBTableModelSessionIT.java       |  39 +++
 .../iotdb/session/it/IoTDBSessionRelationalIT.java |  32 ++
 .../udf/api/relational/AggregationFunction.java    |  11 +-
 .../iotdb/udf/api/relational/SQLFunction.java      |  11 +-
 .../iotdb/udf/api/relational/ScalarFunction.java   |  11 +-
 .../iotdb/udf/api/relational/TableFunction.java    |  11 +-
 .../iotdb/udf/api/relational/access/Record.java    | 129 ++++++++
 .../java/org/apache/iotdb/rpc/TSStatusCode.java    |   2 +
 .../heartbeat/DataNodeHeartbeatHandler.java        |  14 +-
 .../consensus/request/ConfigPhysicalPlan.java      |  10 +-
 .../consensus/request/ConfigPhysicalPlanType.java  |   5 +-
 ...TablePlan.java => GetAllFunctionTablePlan.java} |   6 +-
 .../read/function/GetFunctionTablePlan.java        |  10 +-
 ...onPlan.java => DropTableModelFunctionPlan.java} |  12 +-
 ...ionPlan.java => DropTreeModelFunctionPlan.java} |  12 +-
 ...opFunctionPlan.java => UpdateFunctionPlan.java} |  44 ++-
 .../response/function/FunctionTableResp.java       |  14 +-
 .../iotdb/confignode/manager/ConfigManager.java    |  10 +-
 .../apache/iotdb/confignode/manager/IManager.java  |   6 +-
 .../iotdb/confignode/manager/UDFManager.java       |  95 ++++--
 .../manager/load/cache/AbstractLoadCache.java      |   2 +-
 .../confignode/manager/load/cache/LoadCache.java   |   8 +
 .../load/cache/region/RegionGroupCache.java        |   4 +
 .../iotdb/confignode/manager/node/NodeManager.java |   2 +-
 .../iotdb/confignode/persistence/UDFInfo.java      |  43 ++-
 .../persistence/executor/ConfigPlanExecutor.java   |  20 +-
 .../persistence/schema/ClusterSchemaInfo.java      |   3 +
 .../confignode/persistence/schema/ConfigMTree.java |   2 +-
 .../procedure/env/RemoveDataNodeHandler.java       |  27 +-
 .../impl/pipe/AbstractOperatePipeProcedureV2.java  |  79 ++++-
 .../impl/pipe/task/CreatePipeProcedureV2.java      |   4 +-
 ...bstractOperateSubscriptionAndPipeProcedure.java |   4 +-
 .../thrift/ConfigNodeRPCServiceProcessor.java      |   7 +-
 .../request/ConfigPhysicalPlanSerDeTest.java       |  22 +-
 .../response/ConvertToThriftRespTest.java          |  19 +-
 .../iotdb/confignode/persistence/UDFInfoTest.java  |   6 +-
 .../iot/client/IoTConsensusClientPool.java         |   5 +-
 .../apache/iotdb/consensus/pipe/PipeConsensus.java |  13 +-
 .../consensus/pipe/PipeConsensusServerImpl.java    |  24 +-
 .../consensuspipe/ConsensusPipeDispatcher.java     |   3 +-
 .../pipe/consensuspipe/ConsensusPipeManager.java   |  18 +-
 .../pipe/metric/PipeConsensusSyncLagManager.java   |  88 +++---
 .../service/PipeConsensusRPCServiceProcessor.java  |   3 +-
 .../iotdb/consensus/ratis/RatisConsensus.java      |   3 +-
 .../PipeDataRegionProcessorConstructor.java        |   4 +
 .../protocol/writeback/WriteBackConnector.java     | 120 ++++----
 .../consensus/ConsensusPipeDataNodeDispatcher.java |   4 +-
 .../deletion/persist/PageCacheDeletionBuffer.java  |   4 +-
 .../db/pipe/event/common/PipeInsertionEvent.java   |   4 +
 .../dataregion/DataRegionListeningFilter.java      |  28 ++
 ...oricalDataRegionTsFileAndDeletionExtractor.java |  69 ++---
 .../schemachange/RenameDatabaseProcessor.java      | 101 ++++++
 .../pipeconsensus/PipeConsensusReceiver.java       |  27 +-
 .../iotdb/db/protocol/client/ConfigNodeClient.java |   5 +-
 .../protocol/client/DataNodeClientPoolFactory.java |   5 +-
 .../protocol/thrift/impl/ClientRPCServiceImpl.java |   4 +-
 .../impl/DataNodeInternalRPCServiceImpl.java       |   7 +-
 .../common/header/ColumnHeaderConstant.java        |   4 +-
 .../execution/aggregation/UDAFAccumulator.java     |   2 +-
 .../execution/operator/OperatorContext.java        |   4 +-
 .../TableAggregationTableScanOperator.java         |  45 +--
 .../relational/TableFullOuterJoinOperator.java     | 292 ++++--------------
 .../source/relational/TableInnerJoinOperator.java  | 182 +++++++----
 .../source/relational/TableScanOperator.java       |  44 ++-
 .../grouped/HashAggregationOperator.java           |   2 +
 .../grouped/StreamingAggregationOperator.java      |   5 +-
 .../grouped/StreamingHashAggregationOperator.java  |  18 +-
 .../builder/InMemoryHashAggregationBuilder.java    |  13 +
 .../db/queryengine/plan/analyze/AnalyzeUtils.java  |  56 +++-
 .../plan/execution/config/ConfigExecution.java     |   2 +-
 .../execution/config/TreeConfigTaskVisitor.java    |   5 +-
 .../config/executor/ClusterConfigTaskExecutor.java |  76 +++--
 .../config/executor/IConfigTaskExecutor.java       |  14 +-
 .../config/metadata/CreateFunctionTask.java        |  28 +-
 .../config/metadata/DropFunctionTask.java          |  10 +-
 .../config/metadata/ShowFunctionsTask.java         | 102 +++++--
 .../TableModelStatementMemorySourceVisitor.java    |   7 +-
 .../plan/expression/multi/FunctionExpression.java  |   4 +-
 .../db/queryengine/plan/parser/ASTVisitor.java     |   9 +-
 .../plan/planner/TableOperatorGenerator.java       |   6 +-
 .../planner/distribution/ExchangeNodeAdder.java    |   4 +
 .../plan/planner/plan/node/ExplainAnalyzeNode.java |  26 +-
 .../plan/planner/plan/node/PlanGraphPrinter.java   |  29 +-
 .../plan/planner/plan/node/PlanNodeType.java       |   9 +-
 .../plan/planner/plan/node/PlanVisitor.java        |  11 +
 .../planner/plan/node/process/ExchangeNode.java    |   9 +-
 .../planner/plan/node/write/InsertTabletNode.java  |  12 +-
 .../node/write/RelationalInsertTabletNode.java     |  88 ++++--
 .../plan/relational/analyzer/Analysis.java         |   8 +-
 .../plan/relational/analyzer/Analyzer.java         |   2 +-
 .../relational/analyzer/StatementAnalyzer.java     |   1 +
 .../function/arithmetic/AdditionResolver.java      |  14 +
 .../function/arithmetic/DivisionResolver.java      |  10 +
 .../function/arithmetic/ModulusResolver.java       |  10 +
 .../arithmetic/MultiplicationResolver.java         |  10 +
 .../function/arithmetic/SubtractionResolver.java   |  10 +
 .../relational/metadata/TableMetadataImpl.java     |  17 +-
 .../plan/relational/planner/RelationPlanner.java   |   3 +
 .../relational/planner/TableLogicalPlanner.java    |  10 +-
 .../planner/distribute/AddExchangeNodes.java       |  25 +-
 .../planner/distribute/SubPlanGenerator.java       |   2 +-
 .../distribute/TableDistributedPlanGenerator.java  |  23 +-
 .../distribute/TableDistributedPlanner.java        |   2 +-
 .../distribute/TableModelQueryFragmentPlanner.java |   6 +-
 .../TableModelTypeProviderExtractor.java           |   4 +-
 .../planner/node}/ExchangeNode.java                |  72 +++--
 .../planner/node/ExplainAnalyzeNode.java           | 115 +++++++
 .../optimizations/PushPredicateIntoTableScan.java  |  12 +-
 .../optimizations/UnaliasSymbolReferences.java     |   2 +-
 .../plan/relational/sql/ast/ExplainAnalyze.java    |   5 -
 .../plan/relational/sql/ast/InsertTablet.java      |   3 +
 .../queryengine/plan/relational/sql/ast/Query.java |   5 -
 .../plan/relational/sql/ast/Statement.java         |   4 -
 .../plan/statement/crud/InsertTabletStatement.java |  10 +
 .../metadata/CreateFunctionStatement.java          |  24 +-
 .../dag/column/leaf/NullColumnTransformer.java     |   4 +-
 .../AbstractCastFunctionColumnTransformer.java     |   4 +-
 .../dag/udf/UDAFInformationInferrer.java           |   2 +-
 .../transformation/dag/udf/UDTFExecutor.java       |   2 +-
 .../dag/udf/UDTFInformationInferrer.java           |   2 +-
 .../db/schemaengine/table/DataNodeTableCache.java  |   3 +-
 .../java/org/apache/iotdb/db/service/DataNode.java |  16 +-
 .../db/storageengine/dataregion/DataRegion.java    |  24 +-
 .../CompactionStatisticsCheckFailedException.java  |  87 ++++++
 .../task/RepairUnsortedFileCompactionTask.java     |   4 +-
 .../compaction/execute/utils/CompactionUtils.java  |   4 -
 .../compaction/repair/RepairDataFileScanUtil.java  | 338 +++++++++++++++------
 .../repair/RepairTimePartitionScanTask.java        |   4 +-
 .../dataregion/flush/MemTableFlushTask.java        |   2 +-
 .../dataregion/memtable/AbstractMemTable.java      |   7 +-
 .../memtable/AlignedWritableMemChunk.java          |  18 +-
 .../memtable/AlignedWritableMemChunkGroup.java     |  10 +-
 .../dataregion/memtable/IWritableMemChunk.java     |   5 +-
 .../dataregion/memtable/TsFileProcessor.java       |   1 +
 .../dataregion/memtable/WritableMemChunk.java      |   2 +-
 .../dataregion/modification/DeletionPredicate.java |  22 +-
 .../dataregion/modification/IDPredicate.java       |  80 ++---
 .../dataregion/modification/ModEntry.java          |  16 +-
 .../dataregion/modification/ModificationFile.java  |  31 +-
 .../modification/PartitionLevelModFileManager.java |   2 -
 .../modification/TableDeletionEntry.java           |  14 +-
 .../dataregion/modification/TreeDeletionEntry.java |  14 +-
 .../dataregion/tsfile/TsFileResource.java          |   8 +-
 .../file/UnsealedTsFileRecoverPerformer.java       |  22 ++
 .../apache/iotdb/db/utils/QueryDataSetUtils.java   |   6 +
 .../iotdb/db/utils/io/BufferSerializable.java      |   2 +-
 .../iotdb/db/utils/io/StreamSerializable.java      |   2 +-
 .../iotdb/db/metadata/path/PatternTreeMapTest.java |  27 +-
 .../plan/relational/analyzer/AnalyzerTest.java     |   2 +-
 .../plan/relational/analyzer/JoinTest.java         |   2 +-
 .../analyzer/LimitOffsetPushDownTest.java          |   2 +-
 .../plan/relational/analyzer/SortTest.java         |   2 +-
 .../plan/relational/analyzer/SubQueryTest.java     |   2 +-
 .../plan/relational/analyzer/TestUtils.java        |   2 +-
 .../planner/assertions/ExchangeNodeMatcher.java    |   2 +-
 .../planner/assertions/PlanMatchPattern.java       |   2 +-
 .../planner/assertions/PlanMatchingVisitor.java    |   4 +-
 .../storageengine/dataregion/DataRegionTest.java   |  29 +-
 .../dataregion/LastFlushTimeMapTest.java           |  13 +-
 .../inner/InnerSpaceCompactionSelectorTest.java    |   2 +-
 .../repair/RepairDataFileScanUtilTest.java         | 128 +++++++-
 .../modification/ModificationFileTest.java         |   8 +-
 .../file/UnsealedTsFileRecoverPerformerTest.java   |  47 +++
 .../apache/iotdb/commons/client/ClientManager.java |  11 +-
 .../iotdb/commons/client/ClientPoolFactory.java    |  35 +--
 .../iotdb/commons/client/IClientPoolFactory.java   |   4 +-
 .../client/factory/AsyncThriftClientFactory.java   |   6 +
 .../apache/iotdb/commons/conf/IoTDBConstant.java   |   2 +
 .../org/apache/iotdb/commons/path/PartialPath.java |   3 +-
 .../apache/iotdb/commons/path/PatternTreeMap.java  |   2 +-
 .../agent/plugin/builtin/BuiltinPipePlugin.java    |   3 +
 .../schemachange/RenameDatabaseProcessor.java}     |  18 +-
 .../config/constant/PipeProcessorConstant.java     |   2 +
 .../connector/client/IoTDBSyncClientManager.java   |  25 +-
 .../schema/table/TsTableInternalRPCUtil.java       |   6 +-
 .../apache/iotdb/commons/udf/UDFInformation.java   |  64 ++--
 .../org/apache/iotdb/commons/udf/UDFTable.java     | 105 ++++---
 .../java/org/apache/iotdb/commons/udf/UDFType.java |  71 +++++
 .../BuiltinTimeSeriesGeneratingFunction.java       |  13 +
 .../commons/udf/service/UDFExecutableManager.java  |  46 +++
 .../commons/udf/service/UDFManagementService.java  | 286 +++++++----------
 .../iotdb/commons/udf/utils/TreeUDFUtils.java}     |  32 +-
 .../iotdb/commons/client/ClientManagerTest.java    |  11 +-
 .../thrift-commons/src/main/thrift/common.thrift   |   5 +
 .../src/main/thrift/confignode.thrift              |   9 +-
 .../src/main/thrift/datanode.thrift                |   1 +
 199 files changed, 3389 insertions(+), 1770 deletions(-)