You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by ji...@apache.org on 2022/02/09 15:36:57 UTC
[arrow-datafusion] branch logical-plan updated (b6d0cb7 -> f9ed67a)
This is an automated email from the ASF dual-hosted git repository.
jiayuliu pushed a change to branch logical-plan
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git.
discard b6d0cb7 remove reference to physical plan in logical plan
discard 93c1bf1 add module level comments
discard 040c6d5 move expr functions to datafusion-expr expr_fn
add f6df759 Less verbose plans in debug logging (#1787)
add 071f14a Update `ExecutionPlan` to know about sortedness and repartitioning optimizer pass respect the invariants (#1776)
add ecd0081 [split/13] move rest of expr to expr_fn in datafusion-expr module (#1794)
new b2231d8 move expr functions to datafusion-expr expr_fn
new 7ba97e9 add module level comments
new f9ed67a remove reference to physical plan in logical plan
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b6d0cb7)
\
N -- N -- N refs/heads/logical-plan (f9ed67a)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../core/src/execution_plans/distributed_query.rs | 9 +
.../core/src/execution_plans/shuffle_reader.rs | 9 +
.../core/src/execution_plans/shuffle_writer.rs | 9 +
.../core/src/execution_plans/unresolved_shuffle.rs | 9 +
ballista/rust/executor/src/collect.rs | 5 +
datafusion/src/execution/context.rs | 8 +-
datafusion/src/physical_optimizer/repartition.rs | 396 ++++++++++++++++++---
datafusion/src/physical_plan/analyze.rs | 9 +
datafusion/src/physical_plan/coalesce_batches.rs | 9 +
.../src/physical_plan/coalesce_partitions.rs | 9 +
datafusion/src/physical_plan/cross_join.rs | 9 +
datafusion/src/physical_plan/empty.rs | 5 +
datafusion/src/physical_plan/explain.rs | 10 +-
datafusion/src/physical_plan/file_format/avro.rs | 9 +
datafusion/src/physical_plan/file_format/csv.rs | 9 +
datafusion/src/physical_plan/file_format/json.rs | 9 +
.../src/physical_plan/file_format/parquet.rs | 9 +
datafusion/src/physical_plan/filter.rs | 14 +
datafusion/src/physical_plan/hash_aggregate.rs | 13 +
datafusion/src/physical_plan/hash_join.rs | 9 +
datafusion/src/physical_plan/limit.rs | 40 ++-
datafusion/src/physical_plan/memory.rs | 9 +
datafusion/src/physical_plan/mod.rs | 67 +++-
datafusion/src/physical_plan/planner.rs | 20 +-
datafusion/src/physical_plan/projection.rs | 15 +-
datafusion/src/physical_plan/repartition.rs | 9 +
datafusion/src/physical_plan/sorts/sort.rs | 13 +
.../physical_plan/sorts/sort_preserving_merge.rs | 8 +
datafusion/src/physical_plan/union.rs | 11 +-
datafusion/src/physical_plan/values.rs | 9 +
.../src/physical_plan/windows/window_agg_exec.rs | 13 +
datafusion/src/test/exec.rs | 24 +-
datafusion/tests/custom_sources.rs | 4 +
datafusion/tests/provider_filter_pushdown.rs | 5 +
datafusion/tests/statistics.rs | 9 +-
datafusion/tests/user_defined_plan.rs | 9 +
36 files changed, 755 insertions(+), 78 deletions(-)
[arrow-datafusion] 02/03: add module level comments
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jiayuliu pushed a commit to branch logical-plan
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git
commit 7ba97e98f9128884ed2f84a168bfac20c93a79d3
Author: Jiayu Liu <ji...@hey.com>
AuthorDate: Wed Feb 9 13:14:43 2022 +0800
add module level comments
---
datafusion-expr/src/lib.rs | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/datafusion-expr/src/lib.rs b/datafusion-expr/src/lib.rs
index 1d0837f..709fa63 100644
--- a/datafusion-expr/src/lib.rs
+++ b/datafusion-expr/src/lib.rs
@@ -37,8 +37,8 @@ pub use columnar_value::{ColumnarValue, NullColumnarValue};
pub use expr::Expr;
pub use expr_fn::col;
pub use function::{
- AccumulatorFunctionImplementation, ReturnTypeFunction, ScalarFunctionImplementation,
- StateTypeFunction,
+ AccumulatorFunctionImplementation, ReturnTypeFunction, ScalarFunctionImplementation,
+ StateTypeFunction,
};
pub use literal::{lit, lit_timestamp_nano, Literal, TimestampLiteral};
pub use operator::Operator;
[arrow-datafusion] 03/03: remove reference to physical plan in logical plan
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jiayuliu pushed a commit to branch logical-plan
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git
commit f9ed67aec59d0a9339ec210c508f862456f98089
Author: Jiayu Liu <ji...@hey.com>
AuthorDate: Wed Feb 9 18:09:29 2022 +0800
remove reference to physical plan in logical plan
---
datafusion/src/logical_plan/expr.rs | 3 +--
datafusion/src/logical_plan/registry.rs | 6 ++----
2 files changed, 3 insertions(+), 6 deletions(-)
diff --git a/datafusion/src/logical_plan/expr.rs b/datafusion/src/logical_plan/expr.rs
index ba853ee..f075aaa 100644
--- a/datafusion/src/logical_plan/expr.rs
+++ b/datafusion/src/logical_plan/expr.rs
@@ -22,8 +22,6 @@ pub use super::Operator;
use crate::error::Result;
use crate::logical_plan::ExprSchemable;
use crate::logical_plan::{DFField, DFSchema};
-use crate::physical_plan::udaf::AggregateUDF;
-use crate::physical_plan::udf::ScalarUDF;
use arrow::datatypes::DataType;
use datafusion_common::DataFusionError;
pub use datafusion_common::{Column, ExprSchema};
@@ -33,6 +31,7 @@ use datafusion_expr::BuiltinScalarFunction;
pub use datafusion_expr::Expr;
use datafusion_expr::StateTypeFunction;
pub use datafusion_expr::{lit, lit_timestamp_nano, Literal};
+use datafusion_expr::{AggregateUDF, ScalarUDF};
use datafusion_expr::{
ReturnTypeFunction, ScalarFunctionImplementation, Signature, Volatility,
};
diff --git a/datafusion/src/logical_plan/registry.rs b/datafusion/src/logical_plan/registry.rs
index d9b1839..f439a12 100644
--- a/datafusion/src/logical_plan/registry.rs
+++ b/datafusion/src/logical_plan/registry.rs
@@ -15,11 +15,9 @@
// specific language governing permissions and limitations
// under the License.
-use std::{collections::HashSet, sync::Arc};
-
use crate::error::Result;
-use crate::physical_plan::udaf::AggregateUDF;
-use crate::physical_plan::udf::ScalarUDF;
+use datafusion_expr::{AggregateUDF, ScalarUDF};
+use std::{collections::HashSet, sync::Arc};
/// A registry knows how to build logical expressions out of user-defined function' names
pub trait FunctionRegistry {
[arrow-datafusion] 01/03: move expr functions to datafusion-expr expr_fn
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
jiayuliu pushed a commit to branch logical-plan
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git
commit b2231d8a2d4120e49bcf527f477f63d5e5fb863f
Author: Jiayu Liu <ji...@hey.com>
AuthorDate: Wed Feb 9 13:08:39 2022 +0800
move expr functions to datafusion-expr expr_fn
---
datafusion-expr/src/lib.rs | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/datafusion-expr/src/lib.rs b/datafusion-expr/src/lib.rs
index 709fa63..1d0837f 100644
--- a/datafusion-expr/src/lib.rs
+++ b/datafusion-expr/src/lib.rs
@@ -37,8 +37,8 @@ pub use columnar_value::{ColumnarValue, NullColumnarValue};
pub use expr::Expr;
pub use expr_fn::col;
pub use function::{
- AccumulatorFunctionImplementation, ReturnTypeFunction, ScalarFunctionImplementation,
- StateTypeFunction,
+ AccumulatorFunctionImplementation, ReturnTypeFunction, ScalarFunctionImplementation,
+ StateTypeFunction,
};
pub use literal::{lit, lit_timestamp_nano, Literal, TimestampLiteral};
pub use operator::Operator;