You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by mo...@apache.org on 2023/04/28 08:19:39 UTC
[doris] branch branch-2.0-alpha updated (107bd7d08e -> 363bb21cbd)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch branch-2.0-alpha
in repository https://gitbox.apache.org/repos/asf/doris.git
from 107bd7d08e [chore](third-party) Fix the checksums of mysql (#19047) (#19162)
new 5eecf99c1f [Fix](HttpServer) Chinese garbled characters appear when obtaining query plan (#18820)
new 4d8d278a1f [fix](test) fix result of CHARACTER_OCTET_LENGTH in . (#18896)
new bd4346fb3e [fix](session-var) ignore exception when setting global only var in non master FE (#18949)
new 2c900e569b [Enhancement](data-type) add FE config to prohibit create date and decimalv2 type (#19077)
new 9a4b71b06f [fix](fixed_hashtable) The incorrect implementation of copy constructor (#18921)
new ac38c2da01 [fix](Nereids) should not fallback to legacy planner when execution failed (#18847)
new a8876cf9ce [fix](nereids) remove unnecessary project above scan node (#18920)
new 7c00eacb2e [Compile](vec) Fix compile by BHREAD_SCANNER (#18979)
new c7c10180c1 [fix](Nereids): move SimplifyAggGroupBy before NormalizeAggregate. (#18918)
new 5dc00b473e [fix](schema) fix show create table get wrong random distribution info (#18895)
new c5cfe16aec [bugfix](exception) remove fmt code to test if there still exist core (#19009)
new 7269bd5d00 [fix](regression) fix show create table in_memory = false test result error #19022
new 9423716041 [minor](regression) fix unstable test case (#19018)
new c6f3c5d1c5 [Fix](auth) fix some problem of skip_localhost_auth_check in FE config #18996
new 22cf215ae3 [bugfix](MOW) fix core in set_txn_related_delete_bitmap (#18956)
new 5935ccda52 [Bug](serde) fix serialize column to jsonb when meet boolean and decimal_v3 (#19011)
new d164b373dd [fix](parquet) hasn't initialize select vector when number of nested values equals zero (#18953)
new 2740e08ae3 [fix](planner) vdatetime_value.cpp:1585 Array access may overflow. (#18872)
new a839feba4b [fix](function) Array_map compared offset rows one by one (#18406)
new 569b3f9a9a [fix](session-var) ignore exception when setting global only var in non master FE (#18949)
new 76271fdbfd [Fix](schema scanner) Fixed the problem of overflow when multiplying two INT
new 1c8ab9b96c [Bug](delete) fix the delete ignore char case (#18714)
new 8139c38efb [vectorzied](function) fix year_floor get result is incorrectly (#19006)
new b8eff6b7f5 [fix](Jdbc Catalog) fix Druid Pool parameter and set `testWhileIdle = true` (#19049)
new 703dd7e3a1 [fix](Nereids): UT shouldn't contains slotId (#19082)
new 41b30bd7b1 [fix](Nereids) dead loop in FillUpMissingSlots (#18902)
new 5512deadfc [fix](planner) NPE when use ctas to create table (#18973)
new de9ed9e5f2 [fix](stats) Fix potential NPE when loading Histogram (#19078)
new c738828d87 [Fix](Nereids) fix nereids fold failed by be return null exception (#19013)
new fd03015b4b [fix](doc) fix jsonb_extract doc (#19059)
new 3fa6a5cc16 [fix](regression) mv segcompaction_p1 to segcompaction_p2 (#18806)
new ce717c3177 [bugfix](aggregate_function) Fix wrong registration for percentile_approx #19070
new 39d7f7d857 [Fix](multi catalog)Fix Hive partition path doesn't contain partition value case bug (#19053)
new 6aeb01cbc9 [Fix](Planner) fix throw NPE when parse error (#19097)
new 3c0eb5cc69 [fix](colocate) prohibit dead backend as destination backend in colocate balance (#18970)
new 437eeae852 [fix](nereids) column prune should use slots from children (#19112)
new e802f8a0c2 [fix](planner) SetOperationNode's slots' nullability calculation is wrong (#19108)
new 935329a990 [bug](map-type)fix some bugs in map and map element function (#18935)
new 6c61c59d36 [fix](segment_iter) do not init segment_iterator twice (#18337)
new ac95e1ee3a [fix](heartbeat) fix update BE last start time (#18962)
new 69cee2fce6 [fix](load-profile) fix show load profile return emoty result (#19125)
new 47ba42e9e6 [Bug](mysql proto) fix binary proto with dynamic mode (#19055)
new 070c21b9f0 [fix](test) change to strong pwd to avoid test failure (#19042)
new 3c54fd4934 [fix](Nereids) topn two phase read do not process child correct (#19136)
new 600329d2e3 [doc](fix)load json doc add through S3 table function load #19141
new def203f0d8 [fix](Hive bitmap udf) NPE error when mapreduce task only has map task (#19140)
new 9c20fa88ea [doc](fix)Unified JSON format (#19147)
new 957fca8eab [fix](planner) AnalyticEvalNode should call child's getOutputTupleIds method to get the correct output tuple id (#19163)
new 8b1a9e1f7a [fix](planner) ctas should not clone queryStmt after parse (#19114)
new 25b71b0441 [bugfix](txn_manager) use write lock to protect txn_tablet_map (#19161)
new c7dcf754dd doc style fix (#19158)
new c74b09d04f [clone](download) fix be clone action download tablet content length overflow (#18851)
new 50109d765b Revert "[bugfix](jsonb) fix jsonb parser crash on noavx2 host (#18977)"
new 9d8debfddf Revert "[fix](Nereids): move SimplifyAggGroupBy before NormalizeAggregate. (#18918)"
new 3e59a81c2e fix merge error
new 363bb21cbd Revert "[chore](third-party) Fix the checksums of mysql (#19047) (#19162)"
The 56 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
be/src/agent/heartbeat_server.cpp | 2 +-
be/src/common/exception.h | 27 ++---
be/src/common/status.h | 3 +
.../exec/schema_scanner/schema_columns_scanner.cpp | 2 +-
be/src/http/action/download_action.cpp | 53 +++++++--
be/src/http/http_client.h | 6 +
be/src/olap/rowset/beta_rowset_reader.cpp | 5 -
.../segment_v2/inverted_index_compound_reader.cpp | 2 +-
be/src/olap/rowset/segment_v2/segment.cpp | 1 +
be/src/olap/task/engine_clone_task.cpp | 12 +-
be/src/olap/txn_manager.cpp | 7 +-
be/src/util/blocking_priority_queue.hpp | 2 +-
be/src/util/mysql_row_buffer.cpp | 26 ++---
.../aggregate_function_percentile_approx.cpp | 8 +-
be/src/vec/common/hash_table/fixed_hash_table.h | 3 +-
be/src/vec/exec/format/parquet/schema_desc.cpp | 6 +-
.../parquet/vparquet_column_chunk_reader.cpp | 3 +
.../exec/format/parquet/vparquet_column_reader.cpp | 2 +-
be/src/vec/exec/scan/pip_scanner_context.h | 2 +-
be/src/vec/exec/scan/scanner_context.cpp | 8 +-
be/src/vec/exec/scan/scanner_scheduler.cpp | 11 +-
.../exprs/lambda_function/varray_map_function.cpp | 15 ++-
.../vec/functions/array/function_array_element.h | 3 +-
be/src/vec/runtime/vdatetime_value.cpp | 15 ++-
be/test/olap/delta_writer_test.cpp | 3 +-
conf/be.conf | 2 +-
.../admin-manual/privilege-ldap/user-privilege.md | 2 +-
.../import/import-way/load-json-format.md | 93 +++++++--------
docs/en/docs/lakehouse/multi-catalog/faq.md | 6 +
.../json-functions/jsonb_exists_path.md} | 25 +++--
.../json-functions/jsonb_type.md} | 34 ++++--
.../sql-manual/sql-reference/Data-Types/DATE.md | 2 +
.../sql-manual/sql-reference/Data-Types/DECIMAL.md | 3 +
docs/sidebars.json | 2 +
.../admin-manual/privilege-ldap/user-privilege.md | 2 +-
.../import/import-way/load-json-format.md | 101 ++++++++---------
docs/zh-CN/docs/lakehouse/multi-catalog/faq.md | 6 +
.../json-functions/jsonb_exists_path.md} | 25 +++--
.../sql-functions/json-functions/jsonb_extract.md | 22 +---
.../json-functions/jsonb_type.md} | 34 ++++--
.../sql-manual/sql-reference/Data-Types/DATE.md | 1 +
.../sql-manual/sql-reference/Data-Types/DECIMAL.md | 3 +
.../main/java/org/apache/doris/common/Config.java | 32 ++++++
.../antlr4/org/apache/doris/nereids/DorisParser.g4 | 2 +-
fe/fe-core/src/main/cup/sql_parser.cup | 15 ++-
.../doris/analysis/CreateTableAsSelectStmt.java | 2 +-
.../org/apache/doris/analysis/CreateTableStmt.java | 7 ++
.../apache/doris/analysis/FunctionCallExpr.java | 12 +-
.../apache/doris/analysis/ShowLoadProfileStmt.java | 5 +
.../doris/catalog/RandomDistributionInfo.java | 4 +-
.../clone/ColocateTableCheckerAndBalancer.java | 11 ++
.../doris/datasource/hive/HiveMetaStoreCache.java | 54 ++++++---
.../doris/httpv2/rest/TableQueryPlanAction.java | 4 +-
.../doris/httpv2/rest/manager/HttpUtils.java | 7 ++
.../java/org/apache/doris/load/DeleteHandler.java | 3 +-
.../glue/translator/PhysicalPlanTranslator.java | 52 ++++++---
.../nereids/processor/post/TwoPhaseReadOpt.java | 1 +
.../nereids/rules/analysis/FillUpMissingSlots.java | 92 +++++++++------
.../expression/rules/FoldConstantRuleOnBE.java | 4 +-
.../rules/rewrite/logical/ColumnPruning.java | 7 +-
.../rewrite/logical/SemiJoinAggTranspose.java | 2 +
.../logical/SemiJoinAggTransposeProject.java | 2 +
.../rules/rewrite/logical/SemiJoinCommute.java | 2 +
.../logical/SemiJoinLogicalJoinTranspose.java | 2 +
.../SemiJoinLogicalJoinTransposeProject.java | 2 +
.../doris/nereids/stats/FilterEstimation.java | 6 +-
.../expressions/functions/agg/QuantileUnion.java | 4 +-
.../functions/scalar/QuantilePercent.java | 4 +-
.../functions/scalar/ToQuantileState.java | 4 +-
.../nereids/trees/plans/logical/LogicalHaving.java | 2 +-
.../org/apache/doris/planner/AnalyticEvalNode.java | 2 +-
.../org/apache/doris/planner/SetOperationNode.java | 3 +-
.../apache/doris/planner/external/FileSplit.java | 10 +-
.../doris/planner/external/HiveSplitter.java | 17 +--
.../doris/planner/external/QueryScanProvider.java | 10 +-
.../apache/doris/planner/external/TVFSplitter.java | 6 +-
.../planner/external/iceberg/IcebergSplit.java | 2 +-
.../java/org/apache/doris/qe/ShowExecutor.java | 2 +-
.../java/org/apache/doris/qe/StmtExecutor.java | 125 +++++++++++----------
.../main/java/org/apache/doris/qe/VariableMgr.java | 33 +++++-
.../apache/doris/statistics/ColumnStatistic.java | 4 +
.../org/apache/doris/statistics/Histogram.java | 4 +-
.../apache/doris/statistics/StatisticsCache.java | 4 +-
.../main/java/org/apache/doris/system/Backend.java | 11 +-
.../org/apache/doris/system/BackendHbResponse.java | 13 ---
.../java/org/apache/doris/system/HeartbeatMgr.java | 4 +-
.../ExternalFileTableValuedFunction.java | 53 +++++++--
fe/fe-core/src/main/jflex/sql_scanner.flex | 3 +
.../java/org/apache/doris/catalog/BackendTest.java | 6 +-
.../clone/ColocateTableCheckerAndBalancerTest.java | 10 ++
.../rules/analysis/FillUpMissingSlotsTest.java | 16 +--
.../rules/rewrite/logical/MergeProjectsTest.java | 4 +-
.../apache/doris/utframe/TestWithFeService.java | 2 +
.../org/apache/doris/utframe/UtFrameUtils.java | 2 +
.../java/org/apache/doris/udf/BitmapUnionUDAF.java | 2 +-
.../java/org/apache/doris/udf/ToBitmapUDAF.java | 2 +-
.../java/org/apache/doris/udf/JdbcExecutor.java | 19 +++-
regression-test/conf/regression-conf.groovy | 2 +-
.../correctness_p0/test_union_subquery_groupby.out | 3 +
.../scalar_types/sql/infomation_schema.out | 2 +-
.../hive/test_hive_partition_location.out | 41 +++++++
.../external_table_emr_p2/hive/test_tvf_p2.out | 9 ++
.../stream_load/test_map_load_and_function.out | 27 +++++
.../data/nereids_p0/aggregate/aggregate.out | 3 +
.../data/point_query_p0/test_point_query.out | 47 ++++----
.../data/query_p0/show/test_show_create_table.out | 3 +
.../array_functions/test_array_sortby.out | 32 +++---
.../datetime_functions/test_date_function.out | 3 +
.../test_segcompaction_agg_keys.out | 0
.../test_segcompaction_agg_keys_index.out | 0
.../test_segcompaction_dup_keys.out | 0
.../test_segcompaction_dup_keys_index.out | 0
.../test_segcompaction_unique_keys.out | 0
.../test_segcompaction_unique_keys_mow.out | 0
.../test_segcompaction_unique_keys_mow_index.out | 0
.../pipeline/common/check-pr-if-need-run-build.sh | 2 +
regression-test/pipeline/p0/conf/fe.conf | 2 +
regression-test/pipeline/p1/conf/fe.conf | 2 +
.../account_p0/test_nereids_row_policy.groovy | 8 +-
.../test_union_subquery_groupby.groovy | 48 ++++++++
.../test_window_function_with_project.groovy | 109 ++++++++++++++++++
regression-test/suites/ddl_p0/test_ctas.groovy | 34 ++++++
.../suites/delete_p0/test_delete.groovy | 2 +-
.../hive/test_hive_partition_location.groovy | 65 +++++++++++
.../external_table_emr_p2/hive/test_tvf_p2.groovy | 20 ++++
.../unique_with_mow/test_pk_uk_case.groovy | 8 +-
.../stream_load/test_map_load_and_function.groovy | 9 ++
.../suites/nereids_p0/aggregate/aggregate.groovy | 72 +++++++++++-
.../fold_constant/fold_constant_by_be.groovy | 33 +++---
.../fold_constant/fold_constant_by_fe.groovy | 29 +----
.../suites/nereids_p0/fold_constant/load.groovy | 28 +----
.../suites/point_query_p0/test_point_query.groovy | 37 +++---
.../query_p0/show/test_show_create_table.groovy | 18 +++
.../array_functions/test_array_map_function.groovy | 17 ++-
.../array_functions/test_array_sortby.groovy | 4 +-
.../datetime_functions/test_date_function.groovy | 1 +
.../test_segcompaction_agg_keys.groovy | 2 +-
.../test_segcompaction_agg_keys_index.groovy | 2 +-
.../test_segcompaction_dup_keys.groovy | 2 +-
.../test_segcompaction_dup_keys_index.groovy | 2 +-
.../test_segcompaction_unique_keys.groovy | 2 +-
.../test_segcompaction_unique_keys_index.groovy | 0
.../test_segcompaction_unique_keys_mow.groovy | 2 +-
...test_segcompaction_unique_keys_mow_index.groovy | 2 +-
.../unique_with_mow_p2/test_pk_uk_case.groovy | 8 +-
thirdparty/vars.sh | 2 +-
146 files changed, 1441 insertions(+), 606 deletions(-)
copy docs/en/docs/sql-manual/{sql-reference/Data-Types/DECIMAL.md => sql-functions/json-functions/jsonb_exists_path.md} (66%)
copy docs/en/docs/sql-manual/{sql-reference/Data-Types/DECIMAL.md => sql-functions/json-functions/jsonb_type.md} (64%)
copy docs/zh-CN/docs/sql-manual/{sql-reference/Data-Types/DECIMAL.md => sql-functions/json-functions/jsonb_exists_path.md} (67%)
copy docs/zh-CN/docs/sql-manual/{sql-reference/Data-Types/DECIMAL.md => sql-functions/json-functions/jsonb_type.md} (66%)
create mode 100644 regression-test/data/external_table_emr_p2/hive/test_hive_partition_location.out
rename regression-test/data/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_agg_keys.out (100%)
rename regression-test/data/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_agg_keys_index.out (100%)
rename regression-test/data/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_dup_keys.out (100%)
rename regression-test/data/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_dup_keys_index.out (100%)
rename regression-test/data/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_unique_keys.out (100%)
rename regression-test/data/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_unique_keys_mow.out (100%)
rename regression-test/data/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_unique_keys_mow_index.out (100%)
create mode 100644 regression-test/suites/correctness_p0/test_window_function_with_project.groovy
create mode 100644 regression-test/suites/external_table_emr_p2/hive/test_hive_partition_location.groovy
rename fe/fe-core/src/main/java/org/apache/doris/httpv2/util/HttpUtil.java => regression-test/suites/nereids_p0/fold_constant/fold_constant_by_be.groovy (55%)
copy fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergSplit.java => regression-test/suites/nereids_p0/fold_constant/fold_constant_by_fe.groovy (57%)
copy fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergSplit.java => regression-test/suites/nereids_p0/fold_constant/load.groovy (57%)
rename regression-test/suites/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_agg_keys.groovy (99%)
rename regression-test/suites/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_agg_keys_index.groovy (99%)
rename regression-test/suites/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_dup_keys.groovy (99%)
rename regression-test/suites/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_dup_keys_index.groovy (99%)
rename regression-test/suites/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_unique_keys.groovy (99%)
rename regression-test/suites/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_unique_keys_index.groovy (100%)
rename regression-test/suites/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_unique_keys_mow.groovy (99%)
rename regression-test/suites/{segcompaction_p1 => segcompaction_p2}/test_segcompaction_unique_keys_mow_index.groovy (99%)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org