You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "adriangb (via GitHub)" <gi...@apache.org> on 2026/07/12 18:06:40 UTC
[PR] [EPIC/reference] Port ExecutionPlan serialization to try_to_proto / try_from_proto hooks (full) [datafusion]
adriangb opened a new pull request, #23496:
URL: https://github.com/apache/datafusion/pull/23496
## Which issue does this PR close?
- Part of #23494 (reference / end-state, not intended to merge as-is).
## Rationale for this change
This **draft** shows the complete end state of #23494 in one place: every built-in `ExecutionPlan`, all five `FileSource` scans, all three file sinks, and `MemorySourceConfig` moved onto the `try_to_proto` / `try_from_proto` hooks, with the central `downcast_ref` chain deleted. It is a reference for the incremental PRs — those land one plan/family at a time off #23495; this is not meant to be merged directly.
## What changes are included in this PR?
- The ctx + hooks foundation (== #23495) plus the typed function-codec ctx methods and a scalar-subquery-results scope.
- **~35 plans migrated:** projection, filter, all five joins, unions, sorts (incl. `SortPreservingMergeExec`), limits, repartition, both coalesces, unnest, explain, analyze, cooperative, buffer, placeholder, empty, coalesce-batches, `AggregateExec` (UDAF), both window execs (UDWF), `AsyncFuncExec`, `ScalarSubqueryExec`.
- **DataSource family:** a `try_to_proto` hook on the `DataSource`/`FileSource` traits + the `FileScanConfig` serde ported into `datafusion-datasource`; csv/json/arrow/avro/parquet sources + `MemorySourceConfig`. `DataSourceExec` now serializes purely by delegating `DataSource → FileSource` — no downcast in the central encode path.
- **DataSink family:** the mirror — a `DataSink` hook + `FileSinkConfig` serde port + the three file sinks.
- The 58 now-dead per-plan `PhysicalPlanNodeExt` methods and the `try_from_data_source_exec` / `try_from_data_sink_exec` dispatchers are deleted; `mod.rs` shrinks by ~1900 lines.
**Left as typed dispatch (by design):** `GenerateSeries`/`LazyMemoryExec` (their generator types live above physical-plan — a hook can't name them without a crate cycle) and `Extension` (the terminal third-party fallback).
Notable findings baked in: enum conversions are inlined as **by-name** matches (proto vs. `datafusion_common` numbering differs); the encode dispatch resolves `downcast_delegate()` before the hook (a wrapper-plan bug this caught); `Partitioning`/`PhysicalSortExpr`/`Schema`/`ScalarValue` all serialize from `datafusion-proto-models`/`-common` with no `datafusion-proto`-only helper; no migrated plan needs the raw `PhysicalExtensionCodec`.
## Are these changes tested?
Yes — the full `proto_integration` roundtrip suite passes (193 tests incl. TPC-H, with `avro`/`parquet`), plus new scan/sink roundtrip tests (`roundtrip_{csv,json,avro}_scan`) that exercise the new hook paths. `clippy -D warnings` + `fmt` clean; the workspace builds.
## Are there any user-facing changes?
Yes — a documented breaking `api change`: the per-plan `PhysicalPlanNodeExt` serialization methods and the `TryFromProto<&protobuf::{Json,Csv,Parquet}Sink>` impls are removed (pub-for-proto scaffolding from #21929, released in 54.0.0). The load-bearing public API (`AsExecutionPlan`, `PhysicalExtensionCodec`, `PhysicalProtoConverterExtension`) is unchanged, and the wire format is byte-for-byte identical. An Upgrade Guide note will accompany the incremental removals.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [EPIC/reference] Port ExecutionPlan serialization to try_to_proto / try_from_proto hooks (full) [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #23496:
URL: https://github.com/apache/datafusion/pull/23496#issuecomment-5005357731
<!-- semver-check-comment -->
Thank you for opening this pull request!
Reviewer note: [cargo-semver-checks](https://github.com/obi1kenobi/cargo-semver-checks) reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).
<details>
<summary>Details</summary>
```
Cloning apache/main
Building datafusion-datasource v54.0.0 (current)
Built [ 43.060s] (current)
Parsing datafusion-datasource v54.0.0 (current)
Parsed [ 0.033s] (current)
Building datafusion-datasource v54.0.0 (baseline)
Built [ 38.889s] (baseline)
Parsing datafusion-datasource v54.0.0 (baseline)
Parsed [ 0.032s] (baseline)
Checking datafusion-datasource v54.0.0 -> v54.0.0 (no change; assume patch)
Checked [ 0.256s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 83.698s] datafusion-datasource
Building datafusion-datasource-arrow v54.0.0 (current)
Built [ 43.979s] (current)
Parsing datafusion-datasource-arrow v54.0.0 (current)
Parsed [ 0.012s] (current)
Building datafusion-datasource-arrow v54.0.0 (baseline)
Built [ 38.404s] (baseline)
Parsing datafusion-datasource-arrow v54.0.0 (baseline)
Parsed [ 0.011s] (baseline)
Checking datafusion-datasource-arrow v54.0.0 -> v54.0.0 (no change; assume patch)
Checked [ 0.079s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 83.462s] datafusion-datasource-arrow
Building datafusion-datasource-avro v54.0.0 (current)
Built [ 41.895s] (current)
Parsing datafusion-datasource-avro v54.0.0 (current)
Parsed [ 0.010s] (current)
Building datafusion-datasource-avro v54.0.0 (baseline)
Built [ 38.473s] (baseline)
Parsing datafusion-datasource-avro v54.0.0 (baseline)
Parsed [ 0.010s] (baseline)
Checking datafusion-datasource-avro v54.0.0 -> v54.0.0 (no change; assume patch)
Checked [ 0.073s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 81.508s] datafusion-datasource-avro
Building datafusion-datasource-csv v54.0.0 (current)
Built [ 41.056s] (current)
Parsing datafusion-datasource-csv v54.0.0 (current)
Parsed [ 0.011s] (current)
Building datafusion-datasource-csv v54.0.0 (baseline)
Built [ 37.834s] (baseline)
Parsing datafusion-datasource-csv v54.0.0 (baseline)
Parsed [ 0.012s] (baseline)
Checking datafusion-datasource-csv v54.0.0 -> v54.0.0 (no change; assume patch)
Checked [ 0.105s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 79.941s] datafusion-datasource-csv
Building datafusion-datasource-json v54.0.0 (current)
Built [ 41.077s] (current)
Parsing datafusion-datasource-json v54.0.0 (current)
Parsed [ 0.012s] (current)
Building datafusion-datasource-json v54.0.0 (baseline)
Built [ 37.182s] (baseline)
Parsing datafusion-datasource-json v54.0.0 (baseline)
Parsed [ 0.013s] (baseline)
Checking datafusion-datasource-json v54.0.0 -> v54.0.0 (no change; assume patch)
Checked [ 0.099s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 79.325s] datafusion-datasource-json
Building datafusion-datasource-parquet v54.0.0 (current)
Built [ 46.805s] (current)
Parsing datafusion-datasource-parquet v54.0.0 (current)
Parsed [ 0.030s] (current)
Building datafusion-datasource-parquet v54.0.0 (baseline)
Built [ 42.284s] (baseline)
Parsing datafusion-datasource-parquet v54.0.0 (baseline)
Parsed [ 0.031s] (baseline)
Checking datafusion-datasource-parquet v54.0.0 -> v54.0.0 (no change; assume patch)
Checked [ 0.165s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 90.404s] datafusion-datasource-parquet
Building datafusion-physical-plan v54.0.0 (current)
Built [ 35.695s] (current)
Parsing datafusion-physical-plan v54.0.0 (current)
Parsed [ 0.139s] (current)
Building datafusion-physical-plan v54.0.0 (baseline)
Built [ 35.856s] (baseline)
Parsing datafusion-physical-plan v54.0.0 (baseline)
Parsed [ 0.140s] (baseline)
Checking datafusion-physical-plan v54.0.0 -> v54.0.0 (no change; assume patch)
Checked [ 0.712s] 223 checks: 222 pass, 1 fail, 0 warn, 30 skip
--- failure trait_method_added: pub trait method added ---
Description:
A non-sealed public trait added a new method without a default implementation, which breaks downstream implementations of the trait
ref: https://doc.rust-lang.org/cargo/reference/semver.html#trait-new-item-no-default
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/trait_method_added.ron
Failed in:
trait method datafusion_physical_plan::proto::ExecutionPlanDecode::decode_plan_with_scalar_subquery_results in file /home/runner/work/datafusion/datafusion/datafusion/physical-plan/src/proto.rs:116
Summary semver requires new major version: 1 major and 0 minor checks failed
Finished [ 73.597s] datafusion-physical-plan
Building datafusion-proto v54.0.0 (current)
Built [ 58.487s] (current)
Parsing datafusion-proto v54.0.0 (current)
Parsed [ 0.018s] (current)
Building datafusion-proto v54.0.0 (baseline)
Built [ 59.320s] (baseline)
Parsing datafusion-proto v54.0.0 (baseline)
Parsed [ 0.020s] (baseline)
Checking datafusion-proto v54.0.0 -> v54.0.0 (no change; assume patch)
Checked [ 0.262s] 223 checks: 221 pass, 2 fail, 0 warn, 30 skip
--- failure trait_method_marked_deprecated: trait method #[deprecated] added ---
Description:
A trait method is now #[deprecated]. Downstream crates will get a compiler warning when using this method.
ref: https://doc.rust-lang.org/reference/attributes/diagnostics.html#the-deprecated-attribute
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/trait_method_marked_deprecated.ron
Failed in:
method try_into_explain_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_filter_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_csv_scan_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_json_scan_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_arrow_scan_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_parquet_scan_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_avro_scan_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_memory_scan_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_coalesce_batches_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_merge_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_repartition_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_global_limit_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_local_limit_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_window_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_aggregate_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_hash_join_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_symmetric_hash_join_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_union_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_interleave_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_cross_join_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_empty_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_placeholder_row_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_sort_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_sort_preserving_merge_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_nested_loop_join_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_analyze_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_unnest_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_sort_join in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_cooperative_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_async_func_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_buffer_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_into_scalar_subquery_physical_plan in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_explain_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_analyze_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_filter_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_global_limit_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_local_limit_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_hash_join_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_symmetric_hash_join_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_sort_merge_join_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_cross_join_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_aggregate_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_empty_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_placeholder_row_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_coalesce_batches_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_coalesce_partitions_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_repartition_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_sort_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_union_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_interleave_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_sort_preserving_merge_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_nested_loop_join_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_window_agg_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_bounded_window_agg_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_unnest_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_cooperative_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_async_func_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_buffer_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
method try_from_scalar_subquery_exec in trait datafusion_proto::physical_plan::PhysicalPlanNodeExt in /home/runner/work/datafusion/datafusion/datafusion/proto/src/physical_plan/mod.rs:713
--- failure trait_method_missing: pub trait method removed or renamed ---
Description:
A trait method is no longer callable, and may have been renamed or removed entirely.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#trait-item-signature
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.48.0/src/lints/trait_method_missing.ron
Failed in:
method try_into_json_sink_physical_plan of trait PhysicalPlanNodeExt, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/60ca49fd5eb155b3d67dd28dddcfcec0441bfc35/datafusion/proto/src/physical_plan/mod.rs:2448
method try_into_csv_sink_physical_plan of trait PhysicalPlanNodeExt, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/60ca49fd5eb155b3d67dd28dddcfcec0441bfc35/datafusion/proto/src/physical_plan/mod.rs:2485
method try_into_parquet_sink_physical_plan of trait PhysicalPlanNodeExt, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/60ca49fd5eb155b3d67dd28dddcfcec0441bfc35/datafusion/proto/src/physical_plan/mod.rs:2523
method try_from_data_source_exec of trait PhysicalPlanNodeExt, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/60ca49fd5eb155b3d67dd28dddcfcec0441bfc35/datafusion/proto/src/physical_plan/mod.rs:3525
method try_from_data_sink_exec of trait PhysicalPlanNodeExt, previously in file /home/runner/work/datafusion/datafusion/target/semver-checks/git-apache_main/60ca49fd5eb155b3d67dd28dddcfcec0441bfc35/datafusion/proto/src/physical_plan/mod.rs:4019
Summary semver requires new major version: 1 major and 1 minor checks failed
Finished [ 119.182s] datafusion-proto
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [EPIC/reference] Port ExecutionPlan serialization to try_to_proto / try_from_proto hooks (full) [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb closed pull request #23496: [EPIC/reference] Port ExecutionPlan serialization to try_to_proto / try_from_proto hooks (full)
URL: https://github.com/apache/datafusion/pull/23496
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [EPIC/reference] Port ExecutionPlan serialization to try_to_proto / try_from_proto hooks (full) [datafusion]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #23496:
URL: https://github.com/apache/datafusion/pull/23496#issuecomment-5005460393
## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/23496?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `68.45186%` with `1078 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 80.01%. Comparing base ([`b16bd3d`](https://app.codecov.io/gh/apache/datafusion/commit/b16bd3d874a9e11877d7aa4e59e8845ff576c430?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`4be7866`](https://app.codecov.io/gh/apache/datafusion/commit/4be7866574514c13599abb75b2edd2502b1c83e6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on main.
| [Files with missing lines](https://app.codecov.io/gh/apache/datafusion/pull/23496?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [datafusion/physical-plan/src/explain.rs](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&filepath=datafusion%2Fphysical-plan%2Fsrc%2Fexplain.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1wbGFuL3NyYy9leHBsYWluLnJz) | 0.00% | [125 Missing :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...atafusion/datasource/src/file\_scan\_config/proto.rs](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&filepath=datafusion%2Fdatasource%2Fsrc%2Ffile_scan_config%2Fproto.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9kYXRhc291cmNlL3NyYy9maWxlX3NjYW5fY29uZmlnL3Byb3RvLnJz) | 69.50% | [59 Missing and 52 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...ion/physical-plan/src/joins/symmetric\_hash\_join.rs](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&filepath=datafusion%2Fphysical-plan%2Fsrc%2Fjoins%2Fsymmetric_hash_join.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1wbGFuL3NyYy9qb2lucy9zeW1tZXRyaWNfaGFzaF9qb2luLnJz) | 58.89% | [76 Missing and 21 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...fusion/physical-plan/src/joins/nested\_loop\_join.rs](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&filepath=datafusion%2Fphysical-plan%2Fsrc%2Fjoins%2Fnested_loop_join.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1wbGFuL3NyYy9qb2lucy9uZXN0ZWRfbG9vcF9qb2luLnJz) | 43.51% | [67 Missing and 7 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...usion/physical-plan/src/windows/window\_agg\_exec.rs](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&filepath=datafusion%2Fphysical-plan%2Fsrc%2Fwindows%2Fwindow_agg_exec.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1wbGFuL3NyYy93aW5kb3dzL3dpbmRvd19hZ2dfZXhlYy5ycw==) | 78.16% | [23 Missing and 34 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/datasource/src/memory.rs](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&filepath=datafusion%2Fdatasource%2Fsrc%2Fmemory.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9kYXRhc291cmNlL3NyYy9tZW1vcnkucnM=) | 46.07% | [38 Missing and 17 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/physical-plan/src/aggregates/mod.rs](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&filepath=datafusion%2Fphysical-plan%2Fsrc%2Faggregates%2Fmod.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1wbGFuL3NyYy9hZ2dyZWdhdGVzL21vZC5ycw==) | 79.61% | [22 Missing and 31 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...tafusion/physical-plan/src/joins/hash\_join/exec.rs](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&filepath=datafusion%2Fphysical-plan%2Fsrc%2Fjoins%2Fhash_join%2Fexec.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1wbGFuL3NyYy9qb2lucy9oYXNoX2pvaW4vZXhlYy5ycw==) | 76.66% | [27 Missing and 22 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...on/physical-plan/src/joins/sort\_merge\_join/exec.rs](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&filepath=datafusion%2Fphysical-plan%2Fsrc%2Fjoins%2Fsort_merge_join%2Fexec.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1wbGFuL3NyYy9qb2lucy9zb3J0X21lcmdlX2pvaW4vZXhlYy5ycw==) | 80.61% | [22 Missing and 16 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/physical-plan/src/joins/cross\_join.rs](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&filepath=datafusion%2Fphysical-plan%2Fsrc%2Fjoins%2Fcross_join.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1wbGFuL3NyYy9qb2lucy9jcm9zc19qb2luLnJz) | 0.00% | [34 Missing :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| ... and [31 more](https://app.codecov.io/gh/apache/datafusion/pull/23496?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #23496 +/- ##
==========================================
- Coverage 80.66% 80.01% -0.65%
==========================================
Files 1087 1089 +2
Lines 367414 370322 +2908
Branches 367414 370322 +2908
==========================================
- Hits 296373 296324 -49
- Misses 53388 56234 +2846
- Partials 17653 17764 +111
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/23496?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [EPIC/reference] Port ExecutionPlan serialization to try_to_proto / try_from_proto hooks (full) [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #23496:
URL: https://github.com/apache/datafusion/pull/23496#issuecomment-5195930834
Closing this since most of the work has been subsumed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org