You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by al...@apache.org on 2021/04/09 10:28:26 UTC
[arrow] branch master updated (24da9fa -> 6bace6e)
This is an automated email from the ASF dual-hosted git repository.
alamb pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git.
from 24da9fa MINOR: [JS] Correct build commands (#9960)
add 6bace6e ARROW-12170: [Rust][DataFusion] Introduce repartition optimization
No new revisions were added by this update.
Summary of changes:
rust/datafusion/src/datasource/parquet.rs | 5 +-
rust/datafusion/src/execution/context.rs | 70 +++-----
rust/datafusion/src/lib.rs | 1 +
rust/datafusion/src/optimizer/mod.rs | 1 -
.../src/physical_optimizer/coalesce_batches.rs | 88 ++++++++++
.../src/physical_optimizer/merge_exec.rs | 74 ++++++++
.../src/{datasource => physical_optimizer}/mod.rs | 16 +-
.../{optimizer => physical_optimizer}/optimizer.rs | 19 ++-
.../src/physical_optimizer/repartition.rs | 186 +++++++++++++++++++++
rust/datafusion/src/physical_plan/mod.rs | 2 +-
rust/datafusion/src/physical_plan/parquet.rs | 4 +-
rust/datafusion/src/physical_plan/planner.rs | 90 +++-------
rust/datafusion/tests/user_defined_plan.rs | 9 +-
13 files changed, 425 insertions(+), 140 deletions(-)
create mode 100644 rust/datafusion/src/physical_optimizer/coalesce_batches.rs
create mode 100644 rust/datafusion/src/physical_optimizer/merge_exec.rs
copy rust/datafusion/src/{datasource => physical_optimizer}/mod.rs (77%)
copy rust/datafusion/src/{optimizer => physical_optimizer}/optimizer.rs (71%)
create mode 100644 rust/datafusion/src/physical_optimizer/repartition.rs