You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by ne...@apache.org on 2022/04/02 02:33:40 UTC
[arrow-datafusion] branch rdbms-changes updated (307abcc -> e6614aa)
This is an automated email from the ASF dual-hosted git repository.
nevime pushed a change to branch rdbms-changes
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git.
discard 307abcc add a Tablesource
add 12996ce revise document of installing ballista pinned to specified version (#2034)
add 503618f chore: rearrange the code and add comment (#2037)
add 74bf7ab fix bug the optimizer rule filter push down (#2039)
add c1f6269 Use SessionContext to parse Expr protobuf (#2024)
add 7ed3be6 I think using info in formal code is better than using println. (#2020)
add d02d969 use cargo-tomlfmt to check Cargo.toml formatting in CI (#2033)
add 2dcdb1f Minor: tune log level, lint (#2046)
add 8de2a76 minor: format the annotation (#2047)
add 5936edc Refactor SessionContext, SessionState and SessionConfig to support multi-tenancy configurations - Part 2 (#2029)
add f5c0cea fix panic in register_catalog if default catalog not named "datafusion" and information schema enabled (#2050)
add 2e6833c Update to arrow/parquet 11.0 (#2048)
add 59c6d93 Add `write_json`, `read_json`, `register_json`, and `JsonFormat` to `CREATE EXTERNAL TABLE` functionality (#2023)
add afbeaa6 Allow `CatalogProvider::register_catalog` to return an error (#2052)
add 29d0a65 [Ballista][Scheduler] Change log level for noisy logs (#2060)
add 634252b Qualified wildcard (#2012)
add 257d030 Change the DataFusion explain plans to make it clearer in the predicate/filter (#2063)
add 0194a27 Split datafusion-object-store module (#2065)
add d3c45c2 [MINOR] fix doc in `EXTRACT(field FROM source) (#2074)
add e8ed603 #2004 approx percentile with weight (#2031)
add 04da6a6 [Bug][Datafusion] fix TaskContext session_config bug (#2070)
add 122837d *: fix #1727 (#2085)
add 3d31915 Fix lost filters and projections in ParquetExec, CSVExec etc (#2077)
add d644fae Remove dependency of common for the storage crate (#2076)
add 703c789 *: remove duplicate test (#2089)
add 8159294 fix issue#2058 file_format/json.rs attempt to subtract with overflow (#2066)
add ff110d6 Short-circuit evaluation for `CaseWhen` (#2068)
add 73ea6e1 [Ballista] Support Union in ballista. (#2098)
add a09e1ae add docs for approx functions (#2082)
add 2598893 doc: separate and fix link for `extract` and `date_part` (#2104)
add 2d6addd Refactor SessionContext, BallistaContext to support multi-tenancy configurations - Part 3 (#2091)
add 22fdca3 update zlib version to 1.2.12 (#2106)
add 41b4e49 Reorganize the project folders (#2081)
add b7d3bb1 Create jit-expression from datafusion expression (#2103)
add 86df7ee minor: replace array_equals in case evaluation with eq_dyn from arrow-rs (#2121)
add 91673b3 Serialize timezone in timestamp scalar values (#2120)
add 57a3a6a minor: fix clippy on nightly rust (#2119)
add f313e43 doc: update release schedule (#2110)
add 9e3bec8 Fix case evaluation with NULLs (#2118)
add 3063105 Minor: make disk_manager pub (#2126)
add c43b9ab issue#1967 ignore channel close (#2113)
add f619d43 Minor add clarifying comment in parquet (#2127)
add 4c2320e JIT-compille DataFusion expression with column name (#2124)
new e6614aa add a Tablesource
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (307abcc)
\
N -- N -- N refs/heads/rdbms-changes (e6614aa)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/rust.yml | 57 +-
Cargo.toml | 15 +-
ballista-examples/Cargo.toml | 16 +-
.../bin/ballista-sql.rs => examples/test_sql.rs} | 24 +-
ballista-examples/src/bin/ballista-dataframe.rs | 2 +-
ballista-examples/src/bin/ballista-sql.rs | 2 +-
ballista/rust/client/Cargo.toml | 10 +-
ballista/rust/client/README.md | 6 +-
ballista/rust/client/src/context.rs | 275 +++--
ballista/rust/client/src/prelude.rs | 1 +
ballista/rust/core/Cargo.toml | 22 +-
ballista/rust/core/proto/ballista.proto | 18 +-
ballista/rust/core/src/config.rs | 40 +
ballista/rust/core/src/error.rs | 2 +-
.../core/src/execution_plans/distributed_query.rs | 33 +-
.../core/src/execution_plans/shuffle_writer.rs | 8 +-
.../core/src/execution_plans/unresolved_shuffle.rs | 2 +-
ballista/rust/core/src/serde/logical_plan/mod.rs | 139 ++-
ballista/rust/core/src/serde/mod.rs | 37 +-
.../core/src/serde/physical_plan/from_proto.rs | 15 +-
ballista/rust/core/src/serde/physical_plan/mod.rs | 181 +++-
.../rust/core/src/serde/physical_plan/to_proto.rs | 2 +-
ballista/rust/core/src/utils.rs | 24 +-
ballista/rust/executor/Cargo.toml | 20 +-
ballista/rust/executor/src/cpu_bound_executor.rs | 4 +-
ballista/rust/executor/src/execution_loop.rs | 29 +-
ballista/rust/executor/src/executor.rs | 22 +-
ballista/rust/executor/src/executor_server.rs | 31 +-
ballista/rust/executor/src/flight_service.rs | 3 +-
ballista/rust/executor/src/main.rs | 15 +-
ballista/rust/executor/src/standalone.rs | 12 +-
ballista/rust/scheduler/Cargo.toml | 16 +-
ballista/rust/scheduler/src/main.rs | 8 +-
ballista/rust/scheduler/src/planner.rs | 17 +-
.../rust/scheduler/src/scheduler_server/grpc.rs | 135 ++-
.../rust/scheduler/src/scheduler_server/mod.rs | 137 ++-
.../src/scheduler_server/query_stage_scheduler.rs | 2 +-
ballista/rust/scheduler/src/standalone.rs | 3 -
ballista/rust/scheduler/src/state/mod.rs | 20 +-
.../rust/scheduler/src/state/persistent_state.rs | 52 +-
ballista/rust/scheduler/src/state/stage_manager.rs | 5 +-
.../rust/scheduler/src/state/task_scheduler.rs | 23 +-
ballista/rust/scheduler/src/test_utils.rs | 2 +-
benchmarks/Cargo.toml | 12 +-
benchmarks/src/bin/tpch.rs | 67 +-
{datafusion-examples => data-access}/Cargo.toml | 29 +-
{ballista/rust/scheduler => data-access}/README.md | 5 +-
data-access/src/lib.rs | 77 ++
.../src}/object_store/local.rs | 33 +-
data-access/src/object_store/mod.rs | 104 ++
datafusion-cli/Cargo.toml | 12 +-
datafusion-cli/src/context.rs | 13 +-
datafusion-cli/src/main.rs | 2 +-
datafusion-examples/Cargo.toml | 12 +-
datafusion-examples/examples/avro_sql.rs | 2 +-
datafusion-examples/examples/csv_sql.rs | 2 +-
datafusion-examples/examples/dataframe.rs | 2 +-
.../examples/dataframe_in_memory.rs | 2 +-
datafusion-examples/examples/flight_server.rs | 4 +-
datafusion-examples/examples/memtable.rs | 2 +-
datafusion-examples/examples/parquet_sql.rs | 2 +-
.../examples/parquet_sql_multiple_files.rs | 2 +-
datafusion-examples/examples/simple_udaf.rs | 2 +-
datafusion-examples/examples/simple_udf.rs | 2 +-
datafusion-physical-expr/src/physical_expr.rs | 41 -
.../common}/Cargo.toml | 10 +-
{datafusion-common => datafusion/common}/README.md | 0
.../common}/src/column.rs | 0
.../common}/src/dfschema.rs | 8 +
.../common}/src/error.rs | 0
.../common}/src/lib.rs | 0
.../common}/src/pyarrow.rs | 0
.../common}/src/scalar.rs | 0
datafusion/{ => core}/Cargo.toml | 77 +-
.../{ => core}/benches/aggregate_query_sql.rs | 2 +-
datafusion/{ => core}/benches/data_utils/mod.rs | 0
datafusion/{ => core}/benches/filter_query_sql.rs | 2 +-
datafusion/{ => core}/benches/jit.rs | 0
datafusion/{ => core}/benches/math_query_sql.rs | 2 +-
datafusion/{ => core}/benches/parquet_query_sql.rs | 4 +-
.../{ => core}/benches/parquet_query_sql.sql | 0
datafusion/{ => core}/benches/physical_plan.rs | 0
datafusion/{ => core}/benches/scalar.rs | 0
.../{ => core}/benches/sort_limit_query_sql.rs | 6 +-
datafusion/{ => core}/benches/window_query_sql.rs | 2 +-
datafusion/{ => core}/fuzz-utils/Cargo.toml | 4 +-
datafusion/{ => core}/fuzz-utils/src/lib.rs | 0
.../src/avro_to_arrow/arrow_array_reader.rs | 0
datafusion/{ => core}/src/avro_to_arrow/mod.rs | 0
datafusion/{ => core}/src/avro_to_arrow/reader.rs | 0
datafusion/{ => core}/src/avro_to_arrow/schema.rs | 0
datafusion/{ => core}/src/catalog/catalog.rs | 54 +-
.../{ => core}/src/catalog/information_schema.rs | 3 +-
datafusion/{ => core}/src/catalog/mod.rs | 0
datafusion/{ => core}/src/catalog/schema.rs | 9 +-
datafusion/{ => core}/src/dataframe.rs | 96 +-
datafusion/{ => core}/src/datasource/datasource.rs | 0
datafusion/{ => core}/src/datasource/empty.rs | 0
.../{ => core}/src/datasource/file_format/avro.rs | 10 +-
.../{ => core}/src/datasource/file_format/csv.rs | 14 +-
.../{ => core}/src/datasource/file_format/json.rs | 33 +-
.../{ => core}/src/datasource/file_format/mod.rs | 4 +-
.../src/datasource/file_format/parquet.rs | 14 +-
.../{ => core}/src/datasource/listing/helpers.rs | 19 +-
datafusion/core/src/datasource/listing/mod.rs | 71 ++
.../{ => core}/src/datasource/listing/table.rs | 21 +-
datafusion/{ => core}/src/datasource/memory.rs | 0
datafusion/{ => core}/src/datasource/mod.rs | 40 +-
.../core/src/datasource/object_store_registry.rs | 106 ++
datafusion/{ => core}/src/error.rs | 0
datafusion/{ => core}/src/execution/context.rs | 1104 +++++++++++---------
.../{ => core}/src/execution/disk_manager.rs | 0
.../{ => core}/src/execution/memory_manager.rs | 0
datafusion/{ => core}/src/execution/mod.rs | 2 +-
datafusion/{ => core}/src/execution/options.rs | 16 +-
datafusion/{ => core}/src/execution/runtime_env.rs | 82 +-
datafusion/{ => core}/src/from_slice.rs | 0
datafusion/{ => core}/src/lib.rs | 11 +-
datafusion/{ => core}/src/logical_plan/builder.rs | 74 +-
datafusion/{ => core}/src/logical_plan/dfschema.rs | 0
datafusion/{ => core}/src/logical_plan/display.rs | 0
datafusion/{ => core}/src/logical_plan/expr.rs | 0
.../{ => core}/src/logical_plan/expr_rewriter.rs | 5 +-
.../{ => core}/src/logical_plan/expr_schema.rs | 8 +
.../{ => core}/src/logical_plan/expr_simplier.rs | 0
.../{ => core}/src/logical_plan/expr_visitor.rs | 3 +-
.../{ => core}/src/logical_plan/extension.rs | 0
datafusion/{ => core}/src/logical_plan/mod.rs | 0
.../{ => core}/src/logical_plan/operators.rs | 0
datafusion/{ => core}/src/logical_plan/plan.rs | 46 +-
datafusion/{ => core}/src/logical_plan/registry.rs | 0
.../{ => core}/src/logical_plan/window_frames.rs | 0
.../src/optimizer/common_subexpr_eliminate.rs | 4 +
.../{ => core}/src/optimizer/eliminate_filter.rs | 0
.../{ => core}/src/optimizer/eliminate_limit.rs | 0
.../{ => core}/src/optimizer/filter_push_down.rs | 44 +-
.../{ => core}/src/optimizer/limit_push_down.rs | 0
datafusion/{ => core}/src/optimizer/mod.rs | 0
datafusion/{ => core}/src/optimizer/optimizer.rs | 0
.../src/optimizer/projection_push_down.rs | 0
.../src/optimizer/simplify_expressions.rs | 4 +-
.../src/optimizer/single_distinct_to_groupby.rs | 2 +-
.../{ => core}/src/optimizer/to_approx_perc.rs | 0
datafusion/{ => core}/src/optimizer/utils.rs | 11 +-
.../src/physical_optimizer/aggregate_statistics.rs | 2 +-
.../src/physical_optimizer/coalesce_batches.rs | 8 +-
.../physical_optimizer/hash_build_probe_order.rs | 0
.../src/physical_optimizer/merge_exec.rs | 0
.../{ => core}/src/physical_optimizer/mod.rs | 0
.../{ => core}/src/physical_optimizer/optimizer.rs | 0
.../{ => core}/src/physical_optimizer/pruning.rs | 5 +
.../src/physical_optimizer/repartition.rs | 34 +-
.../{ => core}/src/physical_optimizer/utils.rs | 0
.../{ => core}/src/physical_plan/aggregate_rule.rs | 0
.../{ => core}/src/physical_plan/aggregates.rs | 27 +
datafusion/{ => core}/src/physical_plan/analyze.rs | 0
.../src/physical_plan/coalesce_batches.rs | 0
.../src/physical_plan/coalesce_partitions.rs | 2 +-
datafusion/{ => core}/src/physical_plan/common.rs | 0
.../{ => core}/src/physical_plan/cross_join.rs | 0
datafusion/{ => core}/src/physical_plan/display.rs | 0
datafusion/{ => core}/src/physical_plan/empty.rs | 0
datafusion/{ => core}/src/physical_plan/explain.rs | 0
.../src/physical_plan/file_format/avro.rs | 9 +-
.../src/physical_plan/file_format/csv.rs | 23 +-
.../src/physical_plan/file_format/file_stream.rs | 8 +-
.../src/physical_plan/file_format/json.rs | 104 +-
.../src/physical_plan/file_format/mod.rs | 23 +-
.../src/physical_plan/file_format/parquet.rs | 243 ++---
datafusion/{ => core}/src/physical_plan/filter.rs | 2 +-
.../{ => core}/src/physical_plan/functions.rs | 0
.../{ => core}/src/physical_plan/hash_aggregate.rs | 0
.../{ => core}/src/physical_plan/hash_join.rs | 0
.../{ => core}/src/physical_plan/hash_utils.rs | 0
.../{ => core}/src/physical_plan/join_utils.rs | 0
datafusion/{ => core}/src/physical_plan/limit.rs | 2 +-
datafusion/{ => core}/src/physical_plan/memory.rs | 0
.../src/physical_plan/metrics/baseline.rs | 0
.../src/physical_plan/metrics/builder.rs | 0
.../src/physical_plan/metrics/composite.rs | 0
.../{ => core}/src/physical_plan/metrics/mod.rs | 0
.../src/physical_plan/metrics/tracker.rs | 0
.../{ => core}/src/physical_plan/metrics/value.rs | 0
datafusion/{ => core}/src/physical_plan/mod.rs | 2 +-
datafusion/{ => core}/src/physical_plan/planner.rs | 12 +-
.../{ => core}/src/physical_plan/projection.rs | 2 +-
.../{ => core}/src/physical_plan/repartition.rs | 0
.../{ => core}/src/physical_plan/sorts/mod.rs | 0
.../{ => core}/src/physical_plan/sorts/sort.rs | 21 +-
.../physical_plan/sorts/sort_preserving_merge.rs | 15 +-
datafusion/{ => core}/src/physical_plan/stream.rs | 0
.../{ => core}/src/physical_plan/type_coercion.rs | 0
datafusion/{ => core}/src/physical_plan/udaf.rs | 0
datafusion/{ => core}/src/physical_plan/udf.rs | 0
datafusion/{ => core}/src/physical_plan/union.rs | 9 +-
datafusion/{ => core}/src/physical_plan/values.rs | 0
.../src/physical_plan/window_functions.rs | 0
.../{ => core}/src/physical_plan/windows/mod.rs | 2 +-
.../src/physical_plan/windows/window_agg_exec.rs | 0
datafusion/{ => core}/src/prelude.rs | 0
datafusion/{ => core}/src/row/mod.rs | 0
datafusion/{ => core}/src/row/reader.rs | 0
datafusion/{ => core}/src/row/writer.rs | 0
datafusion/{ => core}/src/scalar.rs | 0
datafusion/{ => core}/src/sql/mod.rs | 0
datafusion/{ => core}/src/sql/parser.rs | 0
datafusion/{ => core}/src/sql/planner.rs | 34 +-
datafusion/{ => core}/src/sql/utils.rs | 1 +
datafusion/{ => core}/src/test/exec.rs | 0
datafusion/{ => core}/src/test/mod.rs | 6 +-
datafusion/{ => core}/src/test/object_store.rs | 16 +-
datafusion/{ => core}/src/test/user_defined.rs | 0
datafusion/{ => core}/src/test/variable.rs | 0
datafusion/{ => core}/src/test_util.rs | 4 +-
datafusion/{ => core}/src/variable/mod.rs | 0
datafusion/{ => core}/tests/aggregate_simple.csv | 0
datafusion/{ => core}/tests/custom_sources.rs | 4 +-
datafusion/{ => core}/tests/customer.csv | 0
datafusion/{ => core}/tests/dataframe.rs | 6 +-
datafusion/{ => core}/tests/dataframe_functions.rs | 2 +-
datafusion/{ => core}/tests/example.csv | 0
datafusion/{ => core}/tests/jsons/1.json | 0
datafusion/{ => core}/tests/jsons/2.json | 0
datafusion/core/tests/jsons/3.json | 1 +
.../core/tests/jsons/schema_infer_limit.json | 4 +
datafusion/{ => core}/tests/merge_fuzz.rs | 0
datafusion/{ => core}/tests/order_spill_fuzz.rs | 6 +-
datafusion/{ => core}/tests/parquet_pruning.rs | 14 +-
datafusion/{ => core}/tests/path_partition.rs | 57 +-
.../{ => core}/tests/provider_filter_pushdown.rs | 2 +-
datafusion/{ => core}/tests/simplification.rs | 0
datafusion/{ => core}/tests/sql/aggregates.rs | 249 +++--
datafusion/{ => core}/tests/sql/avro.rs | 16 +-
datafusion/{ => core}/tests/sql/create_drop.rs | 18 +-
datafusion/{ => core}/tests/sql/errors.rs | 18 +-
datafusion/{ => core}/tests/sql/explain.rs | 0
datafusion/{ => core}/tests/sql/explain_analyze.rs | 86 +-
datafusion/{ => core}/tests/sql/expr.rs | 62 +-
datafusion/{ => core}/tests/sql/functions.rs | 24 +-
datafusion/{ => core}/tests/sql/group_by.rs | 54 +-
.../{ => core}/tests/sql/information_schema.rs | 99 +-
datafusion/{ => core}/tests/sql/intersection.rs | 8 +-
datafusion/{ => core}/tests/sql/joins.rs | 22 +-
datafusion/core/tests/sql/json.rs | 103 ++
datafusion/{ => core}/tests/sql/limit.rs | 16 +-
datafusion/{ => core}/tests/sql/mod.rs | 41 +-
datafusion/{ => core}/tests/sql/order.rs | 8 +-
datafusion/{ => core}/tests/sql/parquet.rs | 10 +-
datafusion/{ => core}/tests/sql/partitioned_csv.rs | 3 +-
datafusion/{ => core}/tests/sql/predicates.rs | 50 +-
datafusion/{ => core}/tests/sql/projection.rs | 8 +-
datafusion/{ => core}/tests/sql/references.rs | 6 +-
datafusion/{ => core}/tests/sql/select.rs | 52 +-
datafusion/{ => core}/tests/sql/timestamp.rs | 54 +-
datafusion/{ => core}/tests/sql/udf.rs | 8 +-
datafusion/{ => core}/tests/sql/unicode.rs | 4 +-
datafusion/{ => core}/tests/sql/union.rs | 6 +-
datafusion/core/tests/sql/wildcard.rs | 149 +++
datafusion/{ => core}/tests/sql/window.rs | 16 +-
datafusion/{ => core}/tests/sql_integration.rs | 0
datafusion/{ => core}/tests/statistics.rs | 10 +-
datafusion/{ => core}/tests/tpch-csv/customer.csv | 0
datafusion/{ => core}/tests/tpch-csv/lineitem.csv | 0
datafusion/{ => core}/tests/tpch-csv/nation.csv | 0
datafusion/{ => core}/tests/tpch-csv/orders.csv | 0
datafusion/{ => core}/tests/user_defined_plan.rs | 15 +-
{datafusion-expr => datafusion/expr}/Cargo.toml | 6 +-
{datafusion-expr => datafusion/expr}/README.md | 0
.../expr}/src/accumulator.rs | 0
.../expr}/src/aggregate_function.rs | 5 +
.../expr}/src/built_in_function.rs | 6 +-
.../expr}/src/columnar_value.rs | 0
{datafusion-expr => datafusion/expr}/src/expr.rs | 6 +
.../expr}/src/expr_fn.rs | 13 +
.../expr}/src/function.rs | 0
{datafusion-expr => datafusion/expr}/src/lib.rs | 0
.../expr}/src/literal.rs | 0
.../expr}/src/operator.rs | 0
.../expr}/src/signature.rs | 0
{datafusion-expr => datafusion/expr}/src/udaf.rs | 0
{datafusion-expr => datafusion/expr}/src/udf.rs | 0
.../expr}/src/window_frame.rs | 0
.../expr}/src/window_function.rs | 0
{datafusion-jit => datafusion/jit}/Cargo.toml | 7 +-
{datafusion-jit => datafusion/jit}/src/api.rs | 0
{datafusion-jit => datafusion/jit}/src/ast.rs | 84 +-
{datafusion-jit => datafusion/jit}/src/jit.rs | 0
{datafusion-jit => datafusion/jit}/src/lib.rs | 46 +-
.../physical-expr}/Cargo.toml | 22 +-
.../physical-expr}/README.md | 0
.../physical-expr}/src/aggregate_expr.rs | 0
.../physical-expr}/src/array_expressions.rs | 0
.../src/coercion_rule/aggregate_rule.rs | 21 +
.../src/coercion_rule/binary_rule.rs | 0
.../physical-expr}/src/coercion_rule/mod.rs | 0
.../physical-expr}/src/crypto_expressions.rs | 0
.../physical-expr}/src/datetime_expressions.rs | 0
.../src/expressions/approx_distinct.rs | 0
.../src/expressions/approx_median.rs | 0
.../src/expressions/approx_percentile_cont.rs | 156 ++-
.../approx_percentile_cont_with_weight.rs | 154 +++
.../physical-expr}/src/expressions/array_agg.rs | 0
.../physical-expr}/src/expressions/average.rs | 0
.../physical-expr}/src/expressions/binary.rs | 0
.../physical-expr}/src/expressions/case.rs | 342 +++---
.../physical-expr}/src/expressions/cast.rs | 0
.../physical-expr}/src/expressions/column.rs | 0
.../physical-expr}/src/expressions/correlation.rs | 0
.../physical-expr}/src/expressions/count.rs | 0
.../physical-expr}/src/expressions/covariance.rs | 0
.../physical-expr}/src/expressions/cume_dist.rs | 0
.../src/expressions/distinct_expressions.rs | 2 +-
.../src/expressions/get_indexed_field.rs | 0
.../physical-expr}/src/expressions/in_list.rs | 0
.../physical-expr}/src/expressions/is_not_null.rs | 0
.../physical-expr}/src/expressions/is_null.rs | 0
.../physical-expr}/src/expressions/lead_lag.rs | 0
.../physical-expr}/src/expressions/literal.rs | 0
.../physical-expr}/src/expressions/min_max.rs | 0
.../physical-expr}/src/expressions/mod.rs | 2 +
.../physical-expr}/src/expressions/negative.rs | 0
.../physical-expr}/src/expressions/not.rs | 0
.../physical-expr}/src/expressions/nth_value.rs | 0
.../physical-expr}/src/expressions/nullif.rs | 0
.../physical-expr}/src/expressions/rank.rs | 0
.../physical-expr}/src/expressions/row_number.rs | 0
.../physical-expr}/src/expressions/stats.rs | 0
.../physical-expr}/src/expressions/stddev.rs | 0
.../physical-expr}/src/expressions/sum.rs | 0
.../physical-expr}/src/expressions/try_cast.rs | 0
.../physical-expr}/src/expressions/variance.rs | 0
.../physical-expr}/src/field_util.rs | 0
.../physical-expr}/src/functions.rs | 0
.../physical-expr}/src/hyperloglog/mod.rs | 0
.../physical-expr}/src/lib.rs | 0
.../physical-expr}/src/math_expressions.rs | 0
datafusion/physical-expr/src/physical_expr.rs | 195 ++++
.../physical-expr}/src/regex_expressions.rs | 0
.../physical-expr}/src/sort_expr.rs | 0
.../physical-expr}/src/string_expressions.rs | 0
.../physical-expr}/src/tdigest/mod.rs | 75 +-
.../physical-expr}/src/unicode_expressions.rs | 0
.../physical-expr}/src/window/aggregate.rs | 0
.../physical-expr}/src/window/built_in.rs | 0
.../src/window/built_in_window_function_expr.rs | 0
.../physical-expr}/src/window/mod.rs | 0
.../src/window/partition_evaluator.rs | 0
.../physical-expr}/src/window/window_expr.rs | 0
{datafusion-proto => datafusion/proto}/Cargo.toml | 2 +-
{datafusion-proto => datafusion/proto}/build.rs | 0
.../proto}/proto/datafusion.proto | 32 +-
.../proto}/src/from_proto.rs | 811 ++++++++------
{datafusion-proto => datafusion/proto}/src/lib.rs | 196 +++-
.../proto}/src/to_proto.rs | 72 +-
datafusion/src/datasource/listing/mod.rs | 24 -
datafusion/src/datasource/object_store/mod.rs | 244 -----
dev/docker/ballista-base.dockerfile | 4 +-
dev/release/README.md | 15 +-
.../distributed/deployment/cargo-install.md | 4 +-
docs/source/user-guide/example-usage.md | 4 +-
docs/source/user-guide/library.md | 2 +-
docs/source/user-guide/sql/aggregate_functions.md | 62 ++
docs/source/user-guide/sql/datafusion-functions.md | 20 +-
docs/source/user-guide/sql/index.rst | 1 +
docs/source/user-guide/sql/sql_status.md | 6 +-
365 files changed, 5594 insertions(+), 2917 deletions(-)
copy ballista-examples/{src/bin/ballista-sql.rs => examples/test_sql.rs} (71%)
copy {datafusion-examples => data-access}/Cargo.toml (75%)
copy {ballista/rust/scheduler => data-access}/README.md (76%)
create mode 100644 data-access/src/lib.rs
rename {datafusion/src/datasource => data-access/src}/object_store/local.rs (89%)
create mode 100644 data-access/src/object_store/mod.rs
delete mode 100644 datafusion-physical-expr/src/physical_expr.rs
rename {datafusion-common => datafusion/common}/Cargo.toml (92%)
rename {datafusion-common => datafusion/common}/README.md (100%)
rename {datafusion-common => datafusion/common}/src/column.rs (100%)
rename {datafusion-common => datafusion/common}/src/dfschema.rs (98%)
rename {datafusion-common => datafusion/common}/src/error.rs (100%)
rename {datafusion-common => datafusion/common}/src/lib.rs (100%)
rename {datafusion-common => datafusion/common}/src/pyarrow.rs (100%)
rename {datafusion-common => datafusion/common}/src/scalar.rs (100%)
rename datafusion/{ => core}/Cargo.toml (86%)
rename datafusion/{ => core}/benches/aggregate_query_sql.rs (99%)
rename datafusion/{ => core}/benches/data_utils/mod.rs (100%)
rename datafusion/{ => core}/benches/filter_query_sql.rs (98%)
rename datafusion/{ => core}/benches/jit.rs (100%)
rename datafusion/{ => core}/benches/math_query_sql.rs (98%)
rename datafusion/{ => core}/benches/parquet_query_sql.rs (98%)
rename datafusion/{ => core}/benches/parquet_query_sql.sql (100%)
rename datafusion/{ => core}/benches/physical_plan.rs (100%)
rename datafusion/{ => core}/benches/scalar.rs (100%)
rename datafusion/{ => core}/benches/sort_limit_query_sql.rs (96%)
rename datafusion/{ => core}/benches/window_query_sql.rs (99%)
rename datafusion/{ => core}/fuzz-utils/Cargo.toml (94%)
rename datafusion/{ => core}/fuzz-utils/src/lib.rs (100%)
rename datafusion/{ => core}/src/avro_to_arrow/arrow_array_reader.rs (100%)
rename datafusion/{ => core}/src/avro_to_arrow/mod.rs (100%)
rename datafusion/{ => core}/src/avro_to_arrow/reader.rs (100%)
rename datafusion/{ => core}/src/avro_to_arrow/schema.rs (100%)
rename datafusion/{ => core}/src/catalog/catalog.rs (75%)
rename datafusion/{ => core}/src/catalog/information_schema.rs (99%)
rename datafusion/{ => core}/src/catalog/mod.rs (100%)
rename datafusion/{ => core}/src/catalog/schema.rs (97%)
rename datafusion/{ => core}/src/dataframe.rs (93%)
rename datafusion/{ => core}/src/datasource/datasource.rs (100%)
rename datafusion/{ => core}/src/datasource/empty.rs (100%)
rename datafusion/{ => core}/src/datasource/file_format/avro.rs (97%)
rename datafusion/{ => core}/src/datasource/file_format/csv.rs (95%)
rename datafusion/{ => core}/src/datasource/file_format/json.rs (87%)
rename datafusion/{ => core}/src/datasource/file_format/mod.rs (94%)
rename datafusion/{ => core}/src/datasource/file_format/parquet.rs (98%)
rename datafusion/{ => core}/src/datasource/listing/helpers.rs (98%)
create mode 100644 datafusion/core/src/datasource/listing/mod.rs
rename datafusion/{ => core}/src/datasource/listing/table.rs (97%)
rename datafusion/{ => core}/src/datasource/memory.rs (100%)
rename datafusion/{ => core}/src/datasource/mod.rs (82%)
create mode 100644 datafusion/core/src/datasource/object_store_registry.rs
rename datafusion/{ => core}/src/error.rs (100%)
rename datafusion/{ => core}/src/execution/context.rs (84%)
rename datafusion/{ => core}/src/execution/disk_manager.rs (100%)
rename datafusion/{ => core}/src/execution/memory_manager.rs (100%)
rename datafusion/{ => core}/src/execution/mod.rs (97%)
rename datafusion/{ => core}/src/execution/options.rs (90%)
rename datafusion/{ => core}/src/execution/runtime_env.rs (61%)
rename datafusion/{ => core}/src/from_slice.rs (100%)
rename datafusion/{ => core}/src/lib.rs (97%)
rename datafusion/{ => core}/src/logical_plan/builder.rs (95%)
rename datafusion/{ => core}/src/logical_plan/dfschema.rs (100%)
rename datafusion/{ => core}/src/logical_plan/display.rs (100%)
rename datafusion/{ => core}/src/logical_plan/expr.rs (100%)
rename datafusion/{ => core}/src/logical_plan/expr_rewriter.rs (99%)
rename datafusion/{ => core}/src/logical_plan/expr_schema.rs (95%)
rename datafusion/{ => core}/src/logical_plan/expr_simplier.rs (100%)
rename datafusion/{ => core}/src/logical_plan/expr_visitor.rs (98%)
rename datafusion/{ => core}/src/logical_plan/extension.rs (100%)
rename datafusion/{ => core}/src/logical_plan/mod.rs (100%)
rename datafusion/{ => core}/src/logical_plan/operators.rs (100%)
rename datafusion/{ => core}/src/logical_plan/plan.rs (96%)
rename datafusion/{ => core}/src/logical_plan/registry.rs (100%)
rename datafusion/{ => core}/src/logical_plan/window_frames.rs (100%)
rename datafusion/{ => core}/src/optimizer/common_subexpr_eliminate.rs (99%)
rename datafusion/{ => core}/src/optimizer/eliminate_filter.rs (100%)
rename datafusion/{ => core}/src/optimizer/eliminate_limit.rs (100%)
rename datafusion/{ => core}/src/optimizer/filter_push_down.rs (97%)
rename datafusion/{ => core}/src/optimizer/limit_push_down.rs (100%)
rename datafusion/{ => core}/src/optimizer/mod.rs (100%)
rename datafusion/{ => core}/src/optimizer/optimizer.rs (100%)
rename datafusion/{ => core}/src/optimizer/projection_push_down.rs (100%)
rename datafusion/{ => core}/src/optimizer/simplify_expressions.rs (99%)
rename datafusion/{ => core}/src/optimizer/single_distinct_to_groupby.rs (99%)
rename datafusion/{ => core}/src/optimizer/to_approx_perc.rs (100%)
rename datafusion/{ => core}/src/optimizer/utils.rs (97%)
rename datafusion/{ => core}/src/physical_optimizer/aggregate_statistics.rs (99%)
rename datafusion/{ => core}/src/physical_optimizer/coalesce_batches.rs (90%)
rename datafusion/{ => core}/src/physical_optimizer/hash_build_probe_order.rs (100%)
rename datafusion/{ => core}/src/physical_optimizer/merge_exec.rs (100%)
rename datafusion/{ => core}/src/physical_optimizer/mod.rs (100%)
rename datafusion/{ => core}/src/physical_optimizer/optimizer.rs (100%)
rename datafusion/{ => core}/src/physical_optimizer/pruning.rs (99%)
rename datafusion/{ => core}/src/physical_optimizer/repartition.rs (94%)
rename datafusion/{ => core}/src/physical_optimizer/utils.rs (100%)
rename datafusion/{ => core}/src/physical_plan/aggregate_rule.rs (100%)
rename datafusion/{ => core}/src/physical_plan/aggregates.rs (97%)
rename datafusion/{ => core}/src/physical_plan/analyze.rs (100%)
rename datafusion/{ => core}/src/physical_plan/coalesce_batches.rs (100%)
rename datafusion/{ => core}/src/physical_plan/coalesce_partitions.rs (99%)
rename datafusion/{ => core}/src/physical_plan/common.rs (100%)
rename datafusion/{ => core}/src/physical_plan/cross_join.rs (100%)
rename datafusion/{ => core}/src/physical_plan/display.rs (100%)
rename datafusion/{ => core}/src/physical_plan/empty.rs (100%)
rename datafusion/{ => core}/src/physical_plan/explain.rs (100%)
rename datafusion/{ => core}/src/physical_plan/file_format/avro.rs (98%)
rename datafusion/{ => core}/src/physical_plan/file_format/csv.rs (97%)
rename datafusion/{ => core}/src/physical_plan/file_format/file_stream.rs (98%)
rename datafusion/{ => core}/src/physical_plan/file_format/json.rs (72%)
rename datafusion/{ => core}/src/physical_plan/file_format/mod.rs (97%)
rename datafusion/{ => core}/src/physical_plan/file_format/parquet.rs (89%)
rename datafusion/{ => core}/src/physical_plan/filter.rs (99%)
rename datafusion/{ => core}/src/physical_plan/functions.rs (100%)
rename datafusion/{ => core}/src/physical_plan/hash_aggregate.rs (100%)
rename datafusion/{ => core}/src/physical_plan/hash_join.rs (100%)
rename datafusion/{ => core}/src/physical_plan/hash_utils.rs (100%)
rename datafusion/{ => core}/src/physical_plan/join_utils.rs (100%)
rename datafusion/{ => core}/src/physical_plan/limit.rs (99%)
rename datafusion/{ => core}/src/physical_plan/memory.rs (100%)
rename datafusion/{ => core}/src/physical_plan/metrics/baseline.rs (100%)
rename datafusion/{ => core}/src/physical_plan/metrics/builder.rs (100%)
rename datafusion/{ => core}/src/physical_plan/metrics/composite.rs (100%)
rename datafusion/{ => core}/src/physical_plan/metrics/mod.rs (100%)
rename datafusion/{ => core}/src/physical_plan/metrics/tracker.rs (100%)
rename datafusion/{ => core}/src/physical_plan/metrics/value.rs (100%)
rename datafusion/{ => core}/src/physical_plan/mod.rs (99%)
rename datafusion/{ => core}/src/physical_plan/planner.rs (99%)
rename datafusion/{ => core}/src/physical_plan/projection.rs (99%)
rename datafusion/{ => core}/src/physical_plan/repartition.rs (100%)
rename datafusion/{ => core}/src/physical_plan/sorts/mod.rs (100%)
rename datafusion/{ => core}/src/physical_plan/sorts/sort.rs (97%)
rename datafusion/{ => core}/src/physical_plan/sorts/sort_preserving_merge.rs (99%)
rename datafusion/{ => core}/src/physical_plan/stream.rs (100%)
rename datafusion/{ => core}/src/physical_plan/type_coercion.rs (100%)
rename datafusion/{ => core}/src/physical_plan/udaf.rs (100%)
rename datafusion/{ => core}/src/physical_plan/udf.rs (100%)
rename datafusion/{ => core}/src/physical_plan/union.rs (98%)
rename datafusion/{ => core}/src/physical_plan/values.rs (100%)
rename datafusion/{ => core}/src/physical_plan/window_functions.rs (100%)
rename datafusion/{ => core}/src/physical_plan/windows/mod.rs (99%)
rename datafusion/{ => core}/src/physical_plan/windows/window_agg_exec.rs (100%)
rename datafusion/{ => core}/src/prelude.rs (100%)
rename datafusion/{ => core}/src/row/mod.rs (100%)
rename datafusion/{ => core}/src/row/reader.rs (100%)
rename datafusion/{ => core}/src/row/writer.rs (100%)
rename datafusion/{ => core}/src/scalar.rs (100%)
rename datafusion/{ => core}/src/sql/mod.rs (100%)
rename datafusion/{ => core}/src/sql/parser.rs (100%)
rename datafusion/{ => core}/src/sql/planner.rs (99%)
rename datafusion/{ => core}/src/sql/utils.rs (99%)
rename datafusion/{ => core}/src/test/exec.rs (100%)
rename datafusion/{ => core}/src/test/mod.rs (98%)
rename datafusion/{ => core}/src/test/object_store.rs (90%)
rename datafusion/{ => core}/src/test/user_defined.rs (100%)
rename datafusion/{ => core}/src/test/variable.rs (100%)
rename datafusion/{ => core}/src/test_util.rs (98%)
rename datafusion/{ => core}/src/variable/mod.rs (100%)
rename datafusion/{ => core}/tests/aggregate_simple.csv (100%)
rename datafusion/{ => core}/tests/custom_sources.rs (99%)
rename datafusion/{ => core}/tests/customer.csv (100%)
rename datafusion/{ => core}/tests/dataframe.rs (97%)
rename datafusion/{ => core}/tests/dataframe_functions.rs (99%)
rename datafusion/{ => core}/tests/example.csv (100%)
rename datafusion/{ => core}/tests/jsons/1.json (100%)
rename datafusion/{ => core}/tests/jsons/2.json (100%)
create mode 100644 datafusion/core/tests/jsons/3.json
create mode 100644 datafusion/core/tests/jsons/schema_infer_limit.json
rename datafusion/{ => core}/tests/merge_fuzz.rs (100%)
rename datafusion/{ => core}/tests/order_spill_fuzz.rs (94%)
rename datafusion/{ => core}/tests/parquet_pruning.rs (98%)
rename datafusion/{ => core}/tests/path_partition.rs (92%)
rename datafusion/{ => core}/tests/provider_filter_pushdown.rs (99%)
rename datafusion/{ => core}/tests/simplification.rs (100%)
rename datafusion/{ => core}/tests/sql/aggregates.rs (79%)
rename datafusion/{ => core}/tests/sql/avro.rs (94%)
rename datafusion/{ => core}/tests/sql/create_drop.rs (91%)
rename datafusion/{ => core}/tests/sql/errors.rs (91%)
rename datafusion/{ => core}/tests/sql/explain.rs (100%)
rename datafusion/{ => core}/tests/sql/explain_analyze.rs (93%)
rename datafusion/{ => core}/tests/sql/expr.rs (96%)
rename datafusion/{ => core}/tests/sql/functions.rs (91%)
rename datafusion/{ => core}/tests/sql/group_by.rs (92%)
rename datafusion/{ => core}/tests/sql/information_schema.rs (88%)
rename datafusion/{ => core}/tests/sql/intersection.rs (94%)
rename datafusion/{ => core}/tests/sql/joins.rs (98%)
create mode 100644 datafusion/core/tests/sql/json.rs
rename datafusion/{ => core}/tests/sql/limit.rs (92%)
rename datafusion/{ => core}/tests/sql/mod.rs (96%)
rename datafusion/{ => core}/tests/sql/order.rs (97%)
rename datafusion/{ => core}/tests/sql/parquet.rs (97%)
rename datafusion/{ => core}/tests/sql/partitioned_csv.rs (96%)
rename datafusion/{ => core}/tests/sql/predicates.rs (90%)
rename datafusion/{ => core}/tests/sql/projection.rs (97%)
rename datafusion/{ => core}/tests/sql/references.rs (97%)
rename datafusion/{ => core}/tests/sql/select.rs (96%)
rename datafusion/{ => core}/tests/sql/timestamp.rs (96%)
rename datafusion/{ => core}/tests/sql/udf.rs (96%)
rename datafusion/{ => core}/tests/sql/unicode.rs (98%)
rename datafusion/{ => core}/tests/sql/union.rs (94%)
create mode 100644 datafusion/core/tests/sql/wildcard.rs
rename datafusion/{ => core}/tests/sql/window.rs (97%)
rename datafusion/{ => core}/tests/sql_integration.rs (100%)
rename datafusion/{ => core}/tests/statistics.rs (97%)
rename datafusion/{ => core}/tests/tpch-csv/customer.csv (100%)
rename datafusion/{ => core}/tests/tpch-csv/lineitem.csv (100%)
rename datafusion/{ => core}/tests/tpch-csv/nation.csv (100%)
rename datafusion/{ => core}/tests/tpch-csv/orders.csv (100%)
rename datafusion/{ => core}/tests/user_defined_plan.rs (97%)
rename {datafusion-expr => datafusion/expr}/Cargo.toml (91%)
rename {datafusion-expr => datafusion/expr}/README.md (100%)
rename {datafusion-expr => datafusion/expr}/src/accumulator.rs (100%)
rename {datafusion-expr => datafusion/expr}/src/aggregate_function.rs (93%)
rename {datafusion-expr => datafusion/expr}/src/built_in_function.rs (98%)
rename {datafusion-expr => datafusion/expr}/src/columnar_value.rs (100%)
rename {datafusion-expr => datafusion/expr}/src/expr.rs (98%)
rename {datafusion-expr => datafusion/expr}/src/expr_fn.rs (97%)
rename {datafusion-expr => datafusion/expr}/src/function.rs (100%)
rename {datafusion-expr => datafusion/expr}/src/lib.rs (100%)
rename {datafusion-expr => datafusion/expr}/src/literal.rs (100%)
rename {datafusion-expr => datafusion/expr}/src/operator.rs (100%)
rename {datafusion-expr => datafusion/expr}/src/signature.rs (100%)
rename {datafusion-expr => datafusion/expr}/src/udaf.rs (100%)
rename {datafusion-expr => datafusion/expr}/src/udf.rs (100%)
rename {datafusion-expr => datafusion/expr}/src/window_frame.rs (100%)
rename {datafusion-expr => datafusion/expr}/src/window_function.rs (100%)
rename {datafusion-jit => datafusion/jit}/Cargo.toml (89%)
rename {datafusion-jit => datafusion/jit}/src/api.rs (100%)
rename {datafusion-jit => datafusion/jit}/src/ast.rs (73%)
rename {datafusion-jit => datafusion/jit}/src/jit.rs (100%)
rename {datafusion-jit => datafusion/jit}/src/lib.rs (71%)
rename {datafusion-physical-expr => datafusion/physical-expr}/Cargo.toml (91%)
rename {datafusion-physical-expr => datafusion/physical-expr}/README.md (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/aggregate_expr.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/array_expressions.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/coercion_rule/aggregate_rule.rs (91%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/coercion_rule/binary_rule.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/coercion_rule/mod.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/crypto_expressions.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/datetime_expressions.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/approx_distinct.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/approx_median.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/approx_percentile_cont.rs (76%)
create mode 100644 datafusion/physical-expr/src/expressions/approx_percentile_cont_with_weight.rs
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/array_agg.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/average.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/binary.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/case.rs (67%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/cast.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/column.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/correlation.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/count.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/covariance.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/cume_dist.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/distinct_expressions.rs (99%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/get_indexed_field.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/in_list.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/is_not_null.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/is_null.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/lead_lag.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/literal.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/min_max.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/mod.rs (98%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/negative.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/not.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/nth_value.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/nullif.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/rank.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/row_number.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/stats.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/stddev.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/sum.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/try_cast.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/expressions/variance.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/field_util.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/functions.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/hyperloglog/mod.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/lib.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/math_expressions.rs (100%)
create mode 100644 datafusion/physical-expr/src/physical_expr.rs
rename {datafusion-physical-expr => datafusion/physical-expr}/src/regex_expressions.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/sort_expr.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/string_expressions.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/tdigest/mod.rs (93%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/unicode_expressions.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/window/aggregate.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/window/built_in.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/window/built_in_window_function_expr.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/window/mod.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/window/partition_evaluator.rs (100%)
rename {datafusion-physical-expr => datafusion/physical-expr}/src/window/window_expr.rs (100%)
rename {datafusion-proto => datafusion/proto}/Cargo.toml (95%)
rename {datafusion-proto => datafusion/proto}/build.rs (100%)
rename {datafusion-proto => datafusion/proto}/proto/datafusion.proto (94%)
rename {datafusion-proto => datafusion/proto}/src/from_proto.rs (68%)
rename {datafusion-proto => datafusion/proto}/src/lib.rs (79%)
rename {datafusion-proto => datafusion/proto}/src/to_proto.rs (94%)
delete mode 100644 datafusion/src/datasource/listing/mod.rs
delete mode 100644 datafusion/src/datasource/object_store/mod.rs
create mode 100644 docs/source/user-guide/sql/aggregate_functions.md
[arrow-datafusion] 01/01: add a Tablesource
Posted by ne...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nevime pushed a commit to branch rdbms-changes
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git
commit e6614aa8ff84ffc6d36d19ae5eaa3e71602df949
Author: Wakahisa <ne...@gmail.com>
AuthorDate: Mon Feb 14 22:50:05 2022 +0200
add a Tablesource
Tablesource contains more information about the source of the table.
It can be a relational table, file(s), in-memory or unspecified.
---
datafusion/core/src/datasource/datasource.rs | 34 ++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/datafusion/core/src/datasource/datasource.rs b/datafusion/core/src/datasource/datasource.rs
index 1b59c85..48a2dc0 100644
--- a/datafusion/core/src/datasource/datasource.rs
+++ b/datafusion/core/src/datasource/datasource.rs
@@ -55,6 +55,35 @@ pub enum TableType {
Temporary,
}
+/// Indicates the source of this table for metadata/catalog purposes.
+#[derive(Debug, Clone, PartialEq)]
+pub enum TableSource {
+ /// An ordinary physical table.
+ Relational {
+ ///
+ server: Option<String>,
+ ///
+ database: Option<String>,
+ ///
+ schema: Option<String>,
+ ///
+ table: String
+ },
+ /// A file on some file system
+ File {
+ ///
+ protocol: String,
+ ///
+ path: String,
+ ///
+ format: String,
+ },
+ /// A transient table.
+ InMemory,
+ /// An unspecified source, used as the default
+ Unspecified,
+}
+
/// Source table
#[async_trait]
pub trait TableProvider: Sync + Send {
@@ -70,6 +99,11 @@ pub trait TableProvider: Sync + Send {
TableType::Base
}
+ /// The source of this table
+ fn table_source(&self) -> TableSource {
+ TableSource::Unspecified
+ }
+
/// Create an ExecutionPlan that will scan the table.
/// The table provider will be usually responsible of grouping
/// the source data into partitions that can be efficiently