You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by mo...@apache.org on 2023/07/03 07:25:10 UTC
[doris] branch runtimefilter_multi_send updated (02685fdfcd -> 7a6532ece9)
This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a change to branch runtimefilter_multi_send
in repository https://gitbox.apache.org/repos/asf/doris.git
omit 02685fdfcd [fix]input/output block were confused (#21439)
omit 4debe00586 [fix](chore) revert idea/vcs.xml change
omit 5d3228a10e [fix](Nereids) use cte producer consumer map to generate new slot in translator
omit 25fbfdb2a2 [fix]Do filter before projection on multi cast sender (#21411)
omit 0424b19fb6 [opt](Nereids) optimize explain stream of DataStreamSink
omit 68af15bbb0 [fix] slot id not found with projection in multi cast sender (#21407)
omit 1c342c0064 [opt](Nereids) optimize explain stream of DataStreamSink
omit de38454222 [fix] conjuncts not initialized (#21376)
omit 1539dc96b9 [opt](Nereids) support set cte shuffle type for each consumer
omit 6fc0667e8e [feature](runtime_filter) MultiCastSender supporting runtime filter on be (#21304)
add f5af735fa6 [fix](multi-catalog)fix obj file cache and dlf iceberg catalog (#21238)
add ca0953ea51 [improvement](join) Serialize build keys in a vectorized (columnar) way (#21361)
add 124516c1ea [Fix](orc-reader) Fix `Wrong data type for column` error when column order in hive table is not same in orc file schema. (#21306)
add f90e8fcb26 [Chore](storage) add debug info for TabletColumn::get_aggregate_function (#21408)
add 59c1bbd163 [Feature](materialized view) support query match mv with agg_state on nereids planner (#21067)
add 2827bc1a39 [Fix](nereids) fix a bug in ColumnStatistics.numNulls update #21220
add a3d34e1e08 [decimalv2](compatibility) add config to allow invalid decimalv2 literal (#21327)
add 7e02566333 [fix](pipeline) fix coredump caused by uncaught exception (#21387)
add 8e8a8da2e7 [Improve](regresstest) update collect distinct regress test for array hash (#21417)
add 17af099dc3 [fix](nereids)miss group id in explain plan #21402
add 9fa2dac352 [fix](Nereids): DefaultPlanRewriter visit plan children. (#21395)
add 88b2d81873 [Fix](multi-catalog) Add hadoop system classpath to CLASSPATH to resolve can not enable hadoop short circuit reading in some environments. (#21430)
add f80df20b6f [Fix](multi-catalog) Fix read error in mixed partition locations. (#21399)
add 50bf006931 [feature](runtime_filter) MultiCastSender supporting runtime filter on be (#21304)
add 98e29e376d [opt](Nereids) support set cte shuffle type for each consumer
add d56d3a25a7 [fix] conjuncts not initialized (#21376)
add 50cd62d1c7 [opt](Nereids) optimize explain stream of DataStreamSink
add 66663e8df0 [fix] slot id not found with projection in multi cast sender (#21407)
add 5a259e75ca [opt](Nereids) optimize explain stream of DataStreamSink
add 9cd1ddceb8 [fix]Do filter before projection on multi cast sender (#21411)
add 7d5b9a92e3 [fix](Nereids) use cte producer consumer map to generate new slot in translator
add 2bb94b6344 [fix](chore) revert idea/vcs.xml change
add 7a6532ece9 [fix]input/output block were confused (#21439)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (02685fdfcd)
\
N -- N -- N refs/heads/runtimefilter_multi_send (7a6532ece9)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
be/src/common/config.cpp | 2 +
be/src/common/config.h | 3 +
be/src/olap/memtable.cpp | 4 +
be/src/olap/tablet_schema.cpp | 7 +-
be/src/pipeline/pipeline_fragment_context.cpp | 5 +-
be/src/runtime/decimalv2_value.cpp | 4 +-
be/src/util/timezone_utils.cpp | 2 +-
be/src/vec/exec/join/process_hash_table_probe.h | 7 ++
.../vec/exec/join/process_hash_table_probe_impl.h | 102 +++++++++++-------
be/src/vec/exec/join/vhash_join_node.cpp | 12 +++
bin/start_be.sh | 5 +-
.../hive/scripts/create_preinstalled_table.hql | 20 ++++
.../orc/test_different_column_orders.orc | Bin 0 -> 671 bytes
.../parquet/test_different_column_orders.parquet | Bin 0 -> 925 bytes
docs/en/docs/lakehouse/faq.md | 11 +-
docs/zh-CN/docs/lakehouse/faq.md | 8 ++
.../apache/doris/analysis/NativeInsertStmt.java | 4 +-
.../doris/catalog/external/HMSExternalTable.java | 4 +
.../java/org/apache/doris/common/util/S3Util.java | 97 +++++++++++++++--
.../doris/datasource/HMSExternalCatalog.java | 4 +
.../doris/datasource/hive/HiveMetaStoreCache.java | 12 ++-
.../doris/datasource/hive/HiveVersionUtil.java | 22 ++++
.../datasource/iceberg/HiveCompatibleCatalog.java | 8 +-
.../doris/datasource/iceberg/dlf/DLFCatalog.java | 31 ++++++
.../datasource/property/PropertyConverter.java | 25 +++--
.../property/constants/S3Properties.java | 4 +
.../org/apache/doris/fs/FileSystemFactory.java | 5 +-
.../java/org/apache/doris/fs/obj/S3ObjStorage.java | 65 ++----------
.../org/apache/doris/fs/remote/RemoteFile.java | 4 +
.../org/apache/doris/nereids/memo/GroupId.java | 2 +-
.../java/org/apache/doris/nereids/memo/Memo.java | 2 +-
.../rewrite/AggScalarSubQueryToWindowFunction.java | 3 +-
.../rules/rewrite/EnsureProjectOnTopJoin.java | 1 -
.../nereids/rules/rewrite/InferPredicates.java | 4 +-
.../mv/SelectMaterializedIndexWithAggregate.java | 55 ++++++++++
.../functions/combinator/StateCombinator.java | 4 +
.../doris/nereids/trees/plans/AbstractPlan.java | 11 +-
.../trees/plans/physical/PhysicalHashJoin.java | 7 +-
.../doris/planner/external/FileQueryScanNode.java | 82 +++++++--------
.../doris/planner/external/HiveScanNode.java | 11 +-
.../doris/planner/external/MaxComputeScanNode.java | 5 +
.../apache/doris/planner/external/TVFScanNode.java | 5 +
.../planner/external/iceberg/IcebergScanNode.java | 13 ++-
.../planner/external/paimon/PaimonScanNode.java | 6 +-
.../org/apache/doris/statistics/Statistics.java | 4 +-
.../hive/test_different_column_orders.out | 29 ++++++
.../hive/test_mixed_par_locations.out | 37 +++++++
.../data/mv_p0/agg_state/test_agg_state_max_by.out | 22 +++-
.../nereids_tpcds_shape_sf100_p0/shape/query21.out | 3 +-
.../nereids_tpcds_shape_sf100_p0/shape/query95.out | 54 +++++-----
.../query_p0/join/test_collect_list_distinct.out | 16 +++
....groovy => test_different_column_orders.groovy} | 25 +++--
.../hive/test_mixed_par_locations.groovy | 63 +++++++++++
.../mv_p0/agg_state/test_agg_state_max_by.groovy | 41 ++++++--
.../join/test_collect_list_distinct.groovy | 116 +++++++++++++++++++++
55 files changed, 854 insertions(+), 244 deletions(-)
create mode 100644 docker/thirdparties/docker-compose/hive/scripts/preinstalled_data/test_different_column_orders/orc/test_different_column_orders.orc
create mode 100644 docker/thirdparties/docker-compose/hive/scripts/preinstalled_data/test_different_column_orders/parquet/test_different_column_orders.parquet
create mode 100644 regression-test/data/external_catalog_p0/hive/test_different_column_orders.out
create mode 100644 regression-test/data/external_table_emr_p2/hive/test_mixed_par_locations.out
create mode 100644 regression-test/data/query_p0/join/test_collect_list_distinct.out
copy regression-test/suites/external_catalog_p0/hive/{test_transactional_hive.groovy => test_different_column_orders.groovy} (70%)
create mode 100644 regression-test/suites/external_table_emr_p2/hive/test_mixed_par_locations.groovy
create mode 100644 regression-test/suites/query_p0/join/test_collect_list_distinct.groovy
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org