You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by li...@apache.org on 2024/02/27 14:56:33 UTC
(doris) branch high-priority-column updated (205bcecba30 -> a470cd32d18)
This is an automated email from the ASF dual-hosted git repository.
lijibing pushed a change to branch high-priority-column
in repository https://gitbox.apache.org/repos/asf/doris.git
from 205bcecba30 [refactor](stats) Remove useless async loader code. (#31380)
add 9135b676a9d [improvement](iceberg/paimon)support estimate row count (#31204)
add 49b1d372ab5 [Chore](materialized-view) forbid create mv have calculations included outside aggregate functions (#31336)
add 96496afce94 [Docs](Job)Add job Docs (#28186)
add 3919ceaf7e4 [doc](job)add job doc sidebars (#31319)
add ce6083311db [doc](community) modify the release verify doc (#31385)
add 3398044d6e4 [feature](nereids) support join with joinRelation (#30909)
add d5b760cbf14 [opt](scanner) scan enough blocks in each scan task (#31277)
add 87201c07161 [fix](function) fix unexpected be core in string search function (#31312)
add 22587ede9dc [feature](doris compose) Add create cloud cluster (#31315)
add 85f7b33a96e [feature](function) support sequence function(alias of array_range), enhance both to handle datetimev2 (#30823)
add f031b4a607b [feature](merge-cloud) Add cloud cluster auth and cloud route (#30784)
add 5d9a3cc351d [improvement](cloud)Not allowed to operate internal_schema database (#31278)
add 701202d6774 Fix cgroup cpu controller file leak (#31398)
add aeacc3a6b74 [Improment](executor)Add scanbytes/scanrows condition (#31364)
add 37d3109fcf4 [Improvement](executor)add remote scan thread pool (#31376)
add 55c5ed1b862 [fix](inverted index)Remove the strong check for `parser` when creating a table with inverted index (#31391)
add ebf474d9d89 [feature](nereids) deal the slots that appear both in agg func and grouping sets (#31318)
add f3c4b006b0d [chore](ci) fix fe conf of clout_p0 (#31388)
add 65774f81f3c [fix](scanner) allocated_bytes should be called after success (#31428)
add db0fdac568f [enhancement](pipelinex) limit add_child to and dependency to avoid error (#31394)
add 2773f7b1a78 [chore](log) change merge-on-write correctness check log to VLOG_NOTICE (#31414)
add f2aee04b1d4 Fix building issue in be on ubuntu with test enabled. (#31407)
add b2e869c7414 [feature](multi-catalog)support hms catalog create and drop table/db (#30198)
add 1b15db34464 [doc](nereids) Optimize query rewerite by materialzied view doc (#31420)
add 1261fddcd7c [Fix](be)Fix gcc compile failed #31431
add d9bfafe0b6d [Fix](test)Fix workload group query queue regression test #31419
add 124e1496dc2 [test](auto partition)increase max_auto_partition_num configure value of auto partition (#31410)
add e7d5636dac7 [docs](docs) Update docs' sidebar of Dev version (#31444)
add 608ad8e0ddd [debug](Variant) sanitize variant type and column in `find_and_set_leave_value` (#31436)
add 7d59df3f0c0 [chore](Nereids) more reasonable parse select list only query (#31346)
add 7eda188e131 [doc](colocate) Fix colocate api doc inconformity with code (#31430)
add 90fa9cf442d [Fix] Only datetime and datetimev2 types can use current_timestamp as column default value (#31395)
add 4a5f3172c91 [fix](mtmv)TVF Query JOB Concurrent Reading and Writing Causes Exception #31422
add f5734737fcf Make sure external table fetched dbId before call getRowCount. (#31379)
add 1e91a3835e2 [fix](Nereids) let with methods of plans use correct logical properties (#31447)
add 0cd0a943d06 [pipelineX](refactor) remove source state from operator functions (#31435)
add 293da238cc2 [Enhancement](group commit) optimize some group commit code (#31392)
add a54fb90f6a0 [fix](MySQL) implement `SHOW CHARSET` statement. (#31389)
add f9068a5300b [enhancement](Nereids) Support show process time and process steps by explain statement (#31339)
add 73184b94323 (merge-cloud) [Feature] Support to do schema change in cloud mode (#31055)
add 4c7d7ddb23e [chore](ci) collect core and adjust log dir (#31418)
add 10661924234 [opt](plsql) Fix procedure key compatibility (#31445)
add a470cd32d18 [fix](Nereids) let time type coercion same with legacy planner (#31472)
No new revisions were added by this update.
Summary of changes:
be/src/agent/agent_server.cpp | 5 +-
be/src/agent/cgroup_cpu_ctl.cpp | 10 +-
be/src/agent/cgroup_cpu_ctl.h | 5 +-
be/src/agent/task_worker_pool.cpp | 126 ++++-
be/src/agent/task_worker_pool.h | 2 +
be/src/agent/workload_group_listener.cpp | 2 +-
be/src/agent/workload_sched_policy_listener.cpp | 4 +-
be/src/cloud/cloud_meta_mgr.cpp | 1 -
be/src/cloud/cloud_rowset_writer.cpp | 7 +-
be/src/cloud/cloud_schema_change_job.cpp | 361 +++++++++++++++
be/src/cloud/cloud_schema_change_job.h | 55 +++
be/src/cloud/pb_convert.cpp | 8 +-
.../cloud_compaction_action.cpp} | 38 +-
.../action/cloud_compaction_action.h} | 33 +-
be/src/http/action/http_stream.cpp | 17 +-
be/src/http/action/stream_load.cpp | 17 +-
be/src/http/utils.cpp | 2 +-
be/src/http/utils.h | 2 +-
be/src/io/cache/block/block_file_cache_factory.h | 6 +
be/src/olap/base_tablet.cpp | 6 +-
be/src/olap/base_tablet.h | 4 +
be/src/olap/merger.cpp | 1 +
be/src/olap/schema_change.cpp | 511 ++++++++++-----------
be/src/olap/schema_change.h | 164 ++++---
be/src/olap/tablet.h | 3 -
be/src/olap/task/engine_alter_tablet_task.cpp | 62 ---
be/src/pipeline/exec/aggregation_sink_operator.cpp | 5 +-
be/src/pipeline/exec/aggregation_sink_operator.h | 3 +-
.../pipeline/exec/aggregation_source_operator.cpp | 70 ++-
be/src/pipeline/exec/aggregation_source_operator.h | 27 +-
be/src/pipeline/exec/analytic_sink_operator.cpp | 4 +-
be/src/pipeline/exec/analytic_sink_operator.h | 3 +-
be/src/pipeline/exec/analytic_source_operator.cpp | 6 +-
be/src/pipeline/exec/analytic_source_operator.h | 3 +-
be/src/pipeline/exec/assert_num_rows_operator.cpp | 2 +-
be/src/pipeline/exec/assert_num_rows_operator.h | 2 +-
be/src/pipeline/exec/datagen_operator.cpp | 9 +-
be/src/pipeline/exec/datagen_operator.h | 3 +-
.../distinct_streaming_aggregation_operator.cpp | 18 +-
.../exec/distinct_streaming_aggregation_operator.h | 8 +-
be/src/pipeline/exec/empty_set_operator.cpp | 4 +-
be/src/pipeline/exec/empty_set_operator.h | 3 +-
be/src/pipeline/exec/exchange_sink_operator.cpp | 28 +-
be/src/pipeline/exec/exchange_sink_operator.h | 3 +-
be/src/pipeline/exec/exchange_source_operator.cpp | 12 +-
be/src/pipeline/exec/exchange_source_operator.h | 3 +-
be/src/pipeline/exec/hashjoin_build_sink.cpp | 6 +-
be/src/pipeline/exec/hashjoin_build_sink.h | 3 +-
be/src/pipeline/exec/hashjoin_probe_operator.cpp | 28 +-
be/src/pipeline/exec/hashjoin_probe_operator.h | 7 +-
be/src/pipeline/exec/jdbc_table_sink_operator.cpp | 5 +-
be/src/pipeline/exec/jdbc_table_sink_operator.h | 3 +-
be/src/pipeline/exec/join_probe_operator.h | 6 +-
be/src/pipeline/exec/multi_cast_data_stream_sink.h | 9 +-
.../exec/multi_cast_data_stream_source.cpp | 9 +-
.../pipeline/exec/multi_cast_data_stream_source.h | 3 +-
.../exec/nested_loop_join_build_operator.cpp | 4 +-
.../exec/nested_loop_join_build_operator.h | 3 +-
.../exec/nested_loop_join_probe_operator.cpp | 25 +-
.../exec/nested_loop_join_probe_operator.h | 5 +-
be/src/pipeline/exec/olap_table_sink_operator.h | 5 +-
be/src/pipeline/exec/olap_table_sink_v2_operator.h | 5 +-
.../pipeline/exec/partition_sort_sink_operator.cpp | 7 +-
.../pipeline/exec/partition_sort_sink_operator.h | 3 +-
.../exec/partition_sort_source_operator.cpp | 9 +-
.../pipeline/exec/partition_sort_source_operator.h | 3 +-
be/src/pipeline/exec/repeat_operator.cpp | 15 +-
be/src/pipeline/exec/repeat_operator.h | 9 +-
be/src/pipeline/exec/result_file_sink_operator.cpp | 5 +-
be/src/pipeline/exec/result_file_sink_operator.h | 3 +-
be/src/pipeline/exec/result_sink_operator.cpp | 3 +-
be/src/pipeline/exec/result_sink_operator.h | 3 +-
be/src/pipeline/exec/scan_operator.cpp | 12 +-
be/src/pipeline/exec/scan_operator.h | 7 +-
be/src/pipeline/exec/schema_scan_operator.cpp | 9 +-
be/src/pipeline/exec/schema_scan_operator.h | 3 +-
be/src/pipeline/exec/select_operator.h | 4 +-
be/src/pipeline/exec/set_probe_sink_operator.cpp | 4 +-
be/src/pipeline/exec/set_probe_sink_operator.h | 3 +-
be/src/pipeline/exec/set_sink_operator.cpp | 7 +-
be/src/pipeline/exec/set_sink_operator.h | 3 +-
be/src/pipeline/exec/set_source_operator.cpp | 14 +-
be/src/pipeline/exec/set_source_operator.h | 5 +-
be/src/pipeline/exec/sort_sink_operator.cpp | 5 +-
be/src/pipeline/exec/sort_sink_operator.h | 3 +-
be/src/pipeline/exec/sort_source_operator.cpp | 11 +-
be/src/pipeline/exec/sort_source_operator.h | 3 +-
.../exec/streaming_aggregation_operator.cpp | 37 +-
.../pipeline/exec/streaming_aggregation_operator.h | 31 +-
be/src/pipeline/exec/table_function_operator.cpp | 7 +-
be/src/pipeline/exec/table_function_operator.h | 18 +-
be/src/pipeline/exec/union_sink_operator.cpp | 5 +-
be/src/pipeline/exec/union_sink_operator.h | 3 +-
be/src/pipeline/exec/union_source_operator.cpp | 19 +-
be/src/pipeline/exec/union_source_operator.h | 5 +-
be/src/pipeline/pipeline_x/dependency.h | 6 +-
.../local_exchange_sink_operator.cpp | 6 +-
.../local_exchange/local_exchange_sink_operator.h | 3 +-
.../local_exchange_source_operator.cpp | 6 +-
.../local_exchange_source_operator.h | 3 +-
.../pipeline_x/local_exchange/local_exchanger.cpp | 59 +--
.../pipeline_x/local_exchange/local_exchanger.h | 35 +-
be/src/pipeline/pipeline_x/operator.cpp | 46 +-
be/src/pipeline/pipeline_x/operator.h | 48 +-
be/src/pipeline/pipeline_x/pipeline_x_task.cpp | 16 +-
be/src/runtime/fragment_mgr.cpp | 7 -
be/src/runtime/group_commit_mgr.cpp | 3 +
be/src/runtime/query_context.cpp | 4 +-
be/src/runtime/query_context.h | 5 +
be/src/runtime/runtime_query_statistics_mgr.cpp | 19 +
be/src/runtime/runtime_query_statistics_mgr.h | 8 +
be/src/runtime/task_group/task_group.cpp | 172 ++++++-
be/src/runtime/task_group/task_group.h | 28 ++
be/src/runtime/task_group/task_group_manager.cpp | 171 +------
be/src/runtime/task_group/task_group_manager.h | 23 +-
be/src/runtime/thread_context.h | 4 +-
.../runtime/workload_management/workload_action.h | 4 +-
.../workload_management/workload_condition.h | 4 +-
.../workload_sched_policy_mgr.cpp | 5 +
be/src/service/http_service.cpp | 19 +
be/src/vec/columns/column_object.cpp | 19 +-
be/src/vec/exec/scan/scanner_context.cpp | 152 +++---
be/src/vec/exec/scan/scanner_context.h | 30 +-
be/src/vec/exec/scan/scanner_scheduler.cpp | 60 ++-
be/src/vec/exec/scan/scanner_scheduler.h | 34 +-
.../vec/functions/array/function_array_range.cpp | 184 ++++++--
.../functions/functions_multi_string_position.cpp | 15 +-
.../functions/functions_multi_string_search.cpp | 15 +-
be/test/http/stream_load_test.cpp | 2 -
build.sh | 6 +-
docker/runtime/doris-compose/Dockerfile | 19 +-
docker/runtime/doris-compose/Readme.md | 31 +-
docker/runtime/doris-compose/cluster.py | 326 ++++++++++---
docker/runtime/doris-compose/command.py | 365 ++++++++++++---
docker/runtime/doris-compose/doris-compose.py | 2 +-
.../doris-compose/resource/cloud.ini.example | 25 +
docker/runtime/doris-compose/resource/common.sh | 95 +++-
docker/runtime/doris-compose/resource/fdb.conf | 45 ++
docker/runtime/doris-compose/resource/init_be.sh | 121 ++++-
.../runtime/doris-compose/resource/init_cloud.sh | 104 +++++
.../resource/{common.sh => init_fdb.sh} | 65 +--
docker/runtime/doris-compose/resource/init_fe.sh | 104 ++++-
.../community/release-and-verify/release-verify.md | 2 +-
.../query-async-materialized-view.md | 113 +++--
.../join-optimization/colocation-join.md | 4 +-
.../sql-functions/array-functions/array-range.md | 27 +-
.../sql-functions/array-functions/sequence.md | 104 +++++
.../sql-functions/table-functions/job.md | 96 ++++
.../sql-functions/table-functions/tasks.md | 69 ++-
.../Data-Definition-Statements/Alter/PAUSE-JOB.md | 14 +-
.../Data-Definition-Statements/Alter/RESUME-JOB.md | 12 +-
.../Create/CREATE-ASYNC-MATERIALIZED-VIEW.md | 12 +
.../Create/CREATE-JOB.md | 76 ++-
.../{Alter/STOP-JOB.md => Drop/DROP-JOB.md} | 20 +-
.../sql-reference/Show-Statements/SHOW-CHARSET.md | 21 +
.../Show-Statements/SHOW-COLLATION.md | 2 +-
.../sql-reference/Show-Statements/SHOW-JOB-TASK.md | 74 ---
.../sql-reference/Show-Statements/SHOW-JOB.md | 93 ----
docs/sidebars.json | 17 +-
.../community/release-and-verify/release-verify.md | 2 +-
.../query-async-materialized-view.md | 54 ++-
.../join-optimization/colocation-join.md | 4 +-
.../sql-functions/array-functions/array-range.md | 27 +-
.../sql-functions/array-functions/sequence.md | 102 ++++
.../sql-functions/table-functions/job.md | 97 ++++
.../sql-functions/table-functions/tasks.md | 65 ++-
.../Data-Definition-Statements/Alter/PAUSE-JOB.md | 10 +-
.../Data-Definition-Statements/Alter/RESUME-JOB.md | 10 +-
.../Create/CREATE-ASYNC-MATERIALIZED-VIEW.md | 10 +
.../Create/CREATE-JOB.md | 48 +-
.../{Alter/STOP-JOB.md => Drop/DROP-JOB.md} | 14 +-
.../sql-reference/Show-Statements/SHOW-CHARSET.md | 19 +
.../Show-Statements/SHOW-COLLATION.md | 2 +-
.../sql-reference/Show-Statements/SHOW-JOB-TASK.md | 75 ---
.../sql-reference/Show-Statements/SHOW-JOB.md | 93 ----
.../org/apache/doris/catalog/PrimitiveType.java | 4 +
.../main/java/org/apache/doris/common/Config.java | 18 +-
.../org/apache/doris/common/FeMetaVersion.java | 4 +-
.../antlr4/org/apache/doris/nereids/DorisLexer.g4 | 3 +
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 20 +-
fe/fe-core/src/main/cup/sql_parser.cup | 54 ++-
.../main/java/org/apache/doris/alter/Alter.java | 20 +-
.../java/org/apache/doris/alter/AlterHandler.java | 5 +-
.../java/org/apache/doris/alter/AlterJobV2.java | 28 ++
.../org/apache/doris/alter/AlterJobV2Factory.java | 76 +++
.../org/apache/doris/alter/CloudRollupJobV2.java | 220 +++++++++
.../apache/doris/alter/CloudSchemaChangeJobV2.java | 229 +++++++++
.../doris/alter/MaterializedViewHandler.java | 26 +-
.../java/org/apache/doris/alter/RollupJobV2.java | 98 ++--
.../apache/doris/alter/SchemaChangeHandler.java | 37 +-
.../org/apache/doris/alter/SchemaChangeJobV2.java | 84 ++--
.../java/org/apache/doris/analysis/ColumnDef.java | 10 +
.../doris/analysis/CreateMaterializedViewStmt.java | 5 +
.../apache/doris/analysis/CreateResourceStmt.java | 2 +-
.../org/apache/doris/analysis/CreateTableStmt.java | 4 -
.../apache/doris/analysis/DropResourceStmt.java | 2 +-
.../org/apache/doris/analysis/DropTableStmt.java | 3 -
.../org/apache/doris/analysis/ExplainOptions.java | 8 +-
.../apache/doris/analysis/FunctionCallExpr.java | 10 +
.../java/org/apache/doris/analysis/GrantStmt.java | 32 +-
.../apache/doris/analysis/InvertedIndexUtil.java | 3 -
.../org/apache/doris/analysis/ResourcePattern.java | 39 +-
.../java/org/apache/doris/analysis/RevokeStmt.java | 16 +-
.../{ShowRolesStmt.java => ShowClusterStmt.java} | 48 +-
.../org/apache/doris/analysis/ShowRolesStmt.java | 1 +
.../org/apache/doris/analysis/UserIdentity.java | 4 +
.../java/org/apache/doris/backup/RestoreJob.java | 6 +-
.../doris/catalog/BuiltinScalarFunctions.java | 2 +-
.../apache/doris/catalog/CatalogRecycleBin.java | 4 +-
.../java/org/apache/doris/catalog/Database.java | 15 +-
.../java/org/apache/doris/catalog/DatabaseIf.java | 13 +-
.../main/java/org/apache/doris/catalog/Env.java | 28 +-
.../java/org/apache/doris/catalog/EnvFactory.java | 25 +
.../java/org/apache/doris/catalog/FunctionSet.java | 6 +
.../org/apache/doris/catalog/InfoSchemaDb.java | 4 +-
.../doris/catalog/MysqlCompatibleDatabase.java | 6 +-
.../java/org/apache/doris/catalog/MysqlDb.java | 2 +-
.../java/org/apache/doris/catalog/Replica.java | 17 +
.../ranger/doris/RangerDorisAccessController.java | 7 +
.../ranger/hive/RangerHiveAccessController.java | 7 +
.../doris/cloud/analysis/UseCloudClusterStmt.java | 113 +++++
.../org/apache/doris/cloud/catalog/CloudEnv.java | 29 ++
.../doris/cloud/catalog/CloudEnvFactory.java | 27 ++
.../apache/doris/cloud/catalog/CloudReplica.java | 10 +-
.../cloud/datasource/CloudInternalCatalog.java | 15 +-
.../apache/doris/cloud/qe/CloudCoordinator.java | 93 ++++
.../doris/cloud/system/CloudSystemInfoService.java | 80 ++++
.../transaction/CloudGlobalTransactionMgr.java | 7 +
.../java/org/apache/doris/common/FeNameFormat.java | 9 +-
.../org/apache/doris/common/proc/AuthProcDir.java | 3 +-
.../doris/common/profile/SummaryProfile.java | 29 +-
.../publish/WorkloadActionPublishThread.java | 123 -----
.../org/apache/doris/common/util/TimeUtils.java | 1 -
.../org/apache/doris/datasource/CatalogIf.java | 13 +
.../org/apache/doris/datasource/CatalogMgr.java | 40 +-
.../DatabaseMetadata.java} | 6 +-
.../apache/doris/datasource/DorisTypeVisitor.java | 79 ++++
.../apache/doris/datasource/ExternalCatalog.java | 90 +++-
.../apache/doris/datasource/ExternalDatabase.java | 33 +-
.../org/apache/doris/datasource/ExternalTable.java | 12 +
.../apache/doris/datasource/InternalCatalog.java | 10 +-
.../TableMetadata.java} | 12 +-
.../doris/datasource/es/EsExternalDatabase.java | 2 +-
.../doris/datasource/hive/HMSCachedClient.java | 12 +
.../datasource/hive/HMSCachedClientFactory.java | 42 --
.../doris/datasource/hive/HMSExternalCatalog.java | 24 +-
.../doris/datasource/hive/HMSExternalDatabase.java | 33 +-
.../doris/datasource/hive/HMSExternalTable.java | 2 +-
.../hive/HiveDatabaseMetadata.java} | 16 +-
.../datasource/hive/HiveMetaStoreClientHelper.java | 30 +-
.../doris/datasource/hive/HiveMetadataOps.java | 179 ++++++++
.../doris/datasource/hive/HiveTableMetadata.java | 102 ++++
.../hive/PostgreSQLJdbcHMSCachedClient.java | 24 +
.../datasource/hive/ThriftHMSCachedClient.java | 170 +++++++
.../datasource/hive/event/AlterDatabaseEvent.java | 4 +-
.../datasource/hive/event/AlterTableEvent.java | 8 +-
.../datasource/hive/event/CreateDatabaseEvent.java | 3 +-
.../datasource/hive/event/CreateTableEvent.java | 2 +-
.../datasource/hive/event/DropDatabaseEvent.java | 2 +-
.../datasource/hive/event/DropTableEvent.java | 2 +-
.../datasource/iceberg/DorisTypeToIcebergType.java | 140 ++++++
.../iceberg/IcebergDLFExternalCatalog.java | 2 +-
.../datasource/iceberg/IcebergExternalCatalog.java | 56 +--
.../iceberg/IcebergExternalDatabase.java | 30 +-
.../datasource/iceberg/IcebergExternalTable.java | 10 +-
.../iceberg/IcebergGlueExternalCatalog.java | 8 +-
.../iceberg/IcebergHMSExternalCatalog.java | 21 +-
.../iceberg/IcebergHadoopExternalCatalog.java | 2 +-
.../datasource/iceberg/IcebergMetadataCache.java | 8 -
.../datasource/iceberg/IcebergMetadataOps.java | 148 ++++++
.../iceberg/IcebergRestExternalCatalog.java | 2 +-
.../doris/datasource/iceberg/IcebergUtils.java | 113 ++++-
.../datasource/iceberg/source/IcebergScanNode.java | 8 +-
.../infoschema/ExternalInfoSchemaDatabase.java | 2 +-
.../datasource/jdbc/JdbcExternalDatabase.java | 2 +-
.../maxcompute/MaxComputeExternalDatabase.java | 2 +-
.../operations/ExternalMetadataOperations.java | 40 ++
.../datasource/operations/ExternalMetadataOps.java | 82 ++++
.../datasource/paimon/PaimonExternalDatabase.java | 30 +-
.../datasource/paimon/PaimonExternalTable.java | 17 +
.../datasource/property/PropertyConverter.java | 2 +-
.../datasource/test/TestExternalDatabase.java | 2 +-
.../apache/doris/job/extensions/mtmv/MTMVTask.java | 4 +-
.../main/java/org/apache/doris/load/DppConfig.java | 5 +
.../apache/doris/load/loadv2/LoadLoadingTask.java | 4 +-
.../java/org/apache/doris/mysql/MysqlProto.java | 17 +
.../mysql/privilege/AccessControllerManager.java | 12 +
.../org/apache/doris/mysql/privilege/Auth.java | 76 ++-
.../mysql/privilege/CatalogAccessController.java | 4 +
.../mysql/privilege/InternalAccessController.java | 7 +
.../apache/doris/mysql/privilege/PrivBitSet.java | 43 +-
.../doris/mysql/privilege/PrivPredicate.java | 9 +-
.../apache/doris/mysql/privilege/Privilege.java | 68 ++-
.../org/apache/doris/mysql/privilege/Role.java | 159 ++++++-
.../apache/doris/mysql/privilege/RoleManager.java | 22 +-
.../org/apache/doris/mysql/privilege/User.java | 16 +-
.../apache/doris/mysql/privilege/UserManager.java | 23 +
.../apache/doris/mysql/privilege/UserProperty.java | 68 ++-
.../doris/mysql/privilege/UserPropertyMgr.java | 50 +-
.../org/apache/doris/nereids/CascadesContext.java | 43 ++
.../org/apache/doris/nereids/NereidsPlanner.java | 94 +++-
.../PlanProcess.java} | 15 +-
.../doris/nereids/glue/LogicalPlanAdapter.java | 9 +-
.../nereids/jobs/rewrite/CustomRewriteJob.java | 19 +-
.../nereids/jobs/rewrite/PlanTreeRewriteJob.java | 27 +-
.../jobs/rewrite/RootPlanTreeRewriteJob.java | 72 +++
.../doris/nereids/parser/LogicalPlanBuilder.java | 69 ++-
.../nereids/rules/analysis/NormalizeRepeat.java | 100 +++-
.../doris/nereids/rules/rewrite/EliminateSort.java | 2 +
.../expressions/functions/scalar/ArrayRange.java | 7 +-
.../{ArrayRange.java => ArrayRangeDayUnit.java} | 57 +--
.../{ArrayRange.java => ArrayRangeHourUnit.java} | 57 +--
.../{ArrayRange.java => ArrayRangeMinuteUnit.java} | 57 +--
.../{ArrayRange.java => ArrayRangeMonthUnit.java} | 57 +--
.../{ArrayRange.java => ArrayRangeSecondUnit.java} | 57 +--
.../{ArrayRange.java => ArrayRangeWeekUnit.java} | 57 +--
.../{ArrayRange.java => ArrayRangeYearUnit.java} | 57 +--
.../expressions/visitor/ScalarFunctionVisitor.java | 35 ++
.../plans/commands/CreateProcedureCommand.java | 6 +-
.../trees/plans/commands/ExplainCommand.java | 17 +-
.../trees/plans/commands/InsertExecutor.java | 3 +-
.../trees/plans/commands/info/CreateTableInfo.java | 10 +-
.../trees/plans/commands/info/FuncNameInfo.java | 70 ++-
.../trees/plans/logical/LogicalCTEConsumer.java | 3 +-
.../trees/plans/logical/LogicalEmptyRelation.java | 2 +-
.../trees/plans/logical/LogicalGenerate.java | 8 +-
.../nereids/trees/plans/logical/LogicalJoin.java | 5 +-
.../trees/plans/logical/LogicalPartitionTopN.java | 3 +-
.../trees/plans/logical/LogicalProject.java | 3 +-
.../trees/plans/logical/LogicalResultSink.java | 2 +-
.../nereids/trees/plans/logical/LogicalTopN.java | 3 +-
.../nereids/trees/plans/logical/LogicalUnion.java | 5 -
.../nereids/trees/plans/logical/LogicalWindow.java | 3 +-
.../java/org/apache/doris/persist/PrivInfo.java | 7 +
.../org/apache/doris/persist/gson/GsonUtils.java | 15 +-
.../org/apache/doris/planner/OlapScanNode.java | 4 +
.../org/apache/doris/planner/OlapTableSink.java | 13 +-
.../src/main/java/org/apache/doris/plsql/Exec.java | 2 +-
.../plsql/functions/DorisFunctionRegistry.java | 32 +-
.../doris/plsql/functions/FunctionRegistry.java | 2 +
.../plsql/functions/InMemoryFunctionRegistry.java | 5 +
.../apache/doris/plsql/metastore/PlsqlManager.java | 16 +-
.../doris/plsql/metastore/PlsqlMetaClient.java | 59 +--
.../apache/doris/plsql/metastore/PlsqlPackage.java | 12 +-
.../doris/plsql/metastore/PlsqlProcedureKey.java | 28 +-
.../plsql/metastore/PlsqlStoredProcedure.java | 19 +-
.../doris/plsql/packages/DorisPackageRegistry.java | 12 +-
.../java/org/apache/doris/qe/ConnectContext.java | 9 +-
.../java/org/apache/doris/qe/ConnectProcessor.java | 18 +
.../main/java/org/apache/doris/qe/Coordinator.java | 6 +-
.../java/org/apache/doris/qe/SessionVariable.java | 2 +-
.../java/org/apache/doris/qe/ShowExecutor.java | 69 ++-
.../java/org/apache/doris/qe/StmtExecutor.java | 64 ++-
.../resource/workloadgroup/WorkloadGroup.java | 47 +-
.../resource/workloadgroup/WorkloadGroupMgr.java | 3 +-
.../workloadschedpolicy/WorkloadAction.java | 2 -
.../WorkloadActionMoveQueryToGroup.java | 67 ---
.../workloadschedpolicy/WorkloadCondition.java | 4 +
...tion.java => WorkloadConditionBeScanBytes.java} | 32 +-
...ition.java => WorkloadConditionBeScanRows.java} | 32 +-
.../workloadschedpolicy/WorkloadConditionMeta.java | 4 +
.../workloadschedpolicy/WorkloadMetricType.java | 2 +-
.../workloadschedpolicy/WorkloadSchedPolicy.java | 4 +-
.../WorkloadSchedPolicyMgr.java | 22 +-
.../apache/doris/service/FrontendServiceImpl.java | 3 +-
.../doris/statistics/util/StatisticsUtil.java | 26 --
.../doris/tablefunction/MetadataGenerator.java | 10 +-
.../WorkloadGroupsTableValuedFunction.java | 2 +
.../org/apache/doris/task/AlterReplicaTask.java | 7 +-
.../org/apache/doris/alter/RollupJobV2Test.java | 3 +-
.../apache/doris/alter/SchemaChangeJobV2Test.java | 3 +-
.../org/apache/doris/analysis/AccessTestUtil.java | 2 +-
.../org/apache/doris/analysis/GrantStmtTest.java | 6 +-
.../org/apache/doris/backup/CatalogMocker.java | 6 +-
.../org/apache/doris/backup/RestoreJobTest.java | 2 +-
.../org/apache/doris/catalog/CatalogTestUtil.java | 6 +-
.../org/apache/doris/catalog/DatabaseTest.java | 18 +-
.../org/apache/doris/catalog/InfoSchemaDbTest.java | 4 +-
.../java/org/apache/doris/catalog/MysqlDbTest.java | 4 +-
.../org/apache/doris/clone/DiskRebalanceTest.java | 4 +-
.../java/org/apache/doris/clone/RebalanceTest.java | 2 +-
.../org/apache/doris/common/util/UnitTestUtil.java | 2 +-
.../apache/doris/datasource/ColumnPrivTest.java | 6 +
.../org/apache/doris/http/DorisHttpTestCase.java | 6 +-
.../org/apache/doris/mysql/privilege/AuthTest.java | 25 +-
.../rules/analysis/FunctionRegistryTest.java | 27 +-
.../doris/nereids/trees/expressions/UdfTest.java | 24 +-
.../org/apache/doris/nereids/util/PlanChecker.java | 2 +-
.../org/apache/doris/persist/PrivInfoTest.java | 3 +-
.../apache/doris/planner/ResourceTagQueryTest.java | 2 +-
.../java/org/apache/doris/qe/CoordinatorTest.java | 11 +-
.../org/apache/doris/qe/OlapQueryCacheTest.java | 14 +-
gensrc/script/doris_builtins_functions.py | 8 +-
gensrc/thrift/BackendService.thrift | 14 +-
gensrc/thrift/FrontendService.thrift | 19 +-
.../nereids_function_p0/scalar_function/Array.out | 194 ++++++++
.../grouping_sets/test_grouping_sets.out | 26 ++
.../data/nereids_p0/subquery/test_subquery.out | 3 +
...ot_both_appear_in_agg_fun_and_grouping_sets.out | 66 +++
.../query_p0/grouping_sets/test_grouping_sets.out | 5 +
.../array_functions/test_array_functions.out | 131 ++++++
.../data/show_p0/test_show_char_set.out | 4 +
.../test_workload_sched_policy.out | 1 -
.../org/apache/doris/regression/suite/Suite.groovy | 17 +-
.../doris/regression/suite/SuiteCluster.groovy | 16 +-
.../pipeline/cloud_p0/conf/fe_custom.conf | 2 +
.../cloud_p0/conf/regression-conf-custom.groovy | 2 +-
regression-test/pipeline/cloud_p0/deploy.sh | 1 +
regression-test/pipeline/cloud_p0/prepare.sh | 1 +
regression-test/pipeline/cloud_p0/run.sh | 6 +-
regression-test/pipeline/common/doris-utils.sh | 131 ++++--
regression-test/pipeline/common/oss-utils.sh | 28 +-
.../account_p0/test_auth_compatibility.groovy | 91 ++++
.../auth/test_grant_revoke_cluster_role.groovy | 265 +++++++++++
.../auth/test_grant_revoke_cluster_to_user.groovy | 308 +++++++++++++
...urrent_timestamp_as_column_default_value.groovy | 47 ++
.../suites/demo_p0/docker_action.groovy | 15 +
.../inverted_index_p0/test_properties.groovy | 4 +-
regression-test/suites/mv_p0/k1s2m3/k1s2m3.groovy | 15 +
.../scalar_function/Array.groovy | 14 +
.../grouping_sets/test_grouping_sets.groovy | 26 +-
.../nereids_p0/subquery/test_subquery.groovy | 6 +
...both_appear_in_agg_fun_and_grouping_sets.groovy | 63 +++
.../suites/nereids_syntax_p0/grouping_sets.groovy | 16 -
.../suites/nereids_syntax_p0/type_cast.groovy | 12 +
.../diff_data/stress_test_diff_date_list.groovy | 4 +-
.../stress_test_high_concurrency_load.groovy | 4 +-
.../same_data/stress_test_same_date_range.groovy | 4 +-
.../stress_test_two_stream_load.groovy | 4 +-
.../grouping_sets/test_grouping_sets.groovy | 27 +-
.../array_functions/test_array_functions.groovy | 84 ++++
.../test_multi_string_position.groovy | 24 +
.../test_multi_string_search.groovy | 26 +-
.../suites/show_p0/test_show_char_set.groovy | 8 +-
.../suites/variant_github_events_p0/load.groovy | 21 +-
.../variant_github_events_p0_new/load.groovy | 12 +-
.../workload_manager_p0/test_curd_wlg.groovy | 4 +-
.../test_workload_sched_policy.groovy | 21 +-
438 files changed, 10024 insertions(+), 3696 deletions(-)
create mode 100644 be/src/cloud/cloud_schema_change_job.cpp
create mode 100644 be/src/cloud/cloud_schema_change_job.h
copy be/src/http/{utils.h => action/cloud_compaction_action.cpp} (50%)
rename be/src/{olap/task/engine_alter_tablet_task.h => http/action/cloud_compaction_action.h} (55%)
delete mode 100644 be/src/olap/task/engine_alter_tablet_task.cpp
create mode 100644 docker/runtime/doris-compose/resource/cloud.ini.example
create mode 100644 docker/runtime/doris-compose/resource/fdb.conf
create mode 100644 docker/runtime/doris-compose/resource/init_cloud.sh
copy docker/runtime/doris-compose/resource/{common.sh => init_fdb.sh} (54%)
create mode 100644 docs/en/docs/sql-manual/sql-functions/array-functions/sequence.md
create mode 100644 docs/en/docs/sql-manual/sql-functions/table-functions/job.md
rename docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/{Alter/STOP-JOB.md => Drop/DROP-JOB.md} (79%)
delete mode 100644 docs/en/docs/sql-manual/sql-reference/Show-Statements/SHOW-JOB-TASK.md
delete mode 100644 docs/en/docs/sql-manual/sql-reference/Show-Statements/SHOW-JOB.md
create mode 100644 docs/zh-CN/docs/sql-manual/sql-functions/array-functions/sequence.md
create mode 100644 docs/zh-CN/docs/sql-manual/sql-functions/table-functions/job.md
rename docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/{Alter/STOP-JOB.md => Drop/DROP-JOB.md} (80%)
delete mode 100644 docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-JOB-TASK.md
delete mode 100644 docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-JOB.md
create mode 100644 fe/fe-core/src/main/java/org/apache/doris/alter/AlterJobV2Factory.java
create mode 100644 fe/fe-core/src/main/java/org/apache/doris/alter/CloudRollupJobV2.java
create mode 100644 fe/fe-core/src/main/java/org/apache/doris/alter/CloudSchemaChangeJobV2.java
copy fe/fe-core/src/main/java/org/apache/doris/analysis/{ShowRolesStmt.java => ShowClusterStmt.java} (52%)
create mode 100644 fe/fe-core/src/main/java/org/apache/doris/cloud/analysis/UseCloudClusterStmt.java
create mode 100644 fe/fe-core/src/main/java/org/apache/doris/cloud/qe/CloudCoordinator.java
delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/common/publish/WorkloadActionPublishThread.java
copy fe/fe-core/src/main/java/org/apache/doris/{resource/workloadschedpolicy/WorkloadMetricType.java => datasource/DatabaseMetadata.java} (85%)
create mode 100644 fe/fe-core/src/main/java/org/apache/doris/datasource/DorisTypeVisitor.java
copy fe/fe-core/src/main/java/org/apache/doris/{resource/workloadschedpolicy/WorkloadMetricType.java => datasource/TableMetadata.java} (81%)
delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HMSCachedClientFactory.java
copy fe/fe-core/src/main/java/org/apache/doris/{resource/workloadschedpolicy/WorkloadMetricType.java => datasource/hive/HiveDatabaseMetadata.java} (70%)
create mode 100644 fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveMetadataOps.java
create mode 100644 fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HiveTableMetadata.java
create mode 100644 fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/DorisTypeToIcebergType.java
create mode 100644 fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/IcebergMetadataOps.java
create mode 100644 fe/fe-core/src/main/java/org/apache/doris/datasource/operations/ExternalMetadataOperations.java
create mode 100644 fe/fe-core/src/main/java/org/apache/doris/datasource/operations/ExternalMetadataOps.java
copy fe/fe-core/src/main/java/org/apache/doris/{resource/workloadschedpolicy/WorkloadMetricType.java => nereids/PlanProcess.java} (67%)
copy fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java => ArrayRangeDayUnit.java} (52%)
copy fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java => ArrayRangeHourUnit.java} (52%)
copy fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java => ArrayRangeMinuteUnit.java} (52%)
copy fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java => ArrayRangeMonthUnit.java} (52%)
copy fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java => ArrayRangeSecondUnit.java} (52%)
copy fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java => ArrayRangeWeekUnit.java} (52%)
copy fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/{ArrayRange.java => ArrayRangeYearUnit.java} (52%)
delete mode 100644 fe/fe-core/src/main/java/org/apache/doris/resource/workloadschedpolicy/WorkloadActionMoveQueryToGroup.java
copy fe/fe-core/src/main/java/org/apache/doris/resource/workloadschedpolicy/{WorkloadCondition.java => WorkloadConditionBeScanBytes.java} (51%)
copy fe/fe-core/src/main/java/org/apache/doris/resource/workloadschedpolicy/{WorkloadCondition.java => WorkloadConditionBeScanRows.java} (51%)
create mode 100644 regression-test/data/nereids_rules_p0/grouping_sets/slot_both_appear_in_agg_fun_and_grouping_sets.out
create mode 100644 regression-test/data/show_p0/test_show_char_set.out
create mode 100644 regression-test/suites/account_p0/test_auth_compatibility.groovy
create mode 100644 regression-test/suites/cloud/auth/test_grant_revoke_cluster_role.groovy
create mode 100644 regression-test/suites/cloud/auth/test_grant_revoke_cluster_to_user.groovy
create mode 100644 regression-test/suites/ddl_p0/test_current_timestamp_as_column_default_value.groovy
create mode 100644 regression-test/suites/nereids_rules_p0/grouping_sets/slot_both_appear_in_agg_fun_and_grouping_sets.groovy
copy fe/fe-core/src/main/java/org/apache/doris/resource/workloadschedpolicy/WorkloadMetricType.java => regression-test/suites/show_p0/test_show_char_set.groovy (85%)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org