You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "mbutrovich (via GitHub)" <gi...@apache.org> on 2026/08/04 16:28:22 UTC

[PR] feat: prune unread Parquet leaves when a nested column is cast to a narrower type [datafusion]

mbutrovich opened a new pull request, #24090:
URL: https://github.com/apache/datafusion/pull/24090

   ## Which issue does this PR close?
   
   - Related to https://github.com/apache/datafusion-comet/issues/4859. No DataFusion issue is filed for this one.
   
   ## Rationale for this change
   
   When a table declares a nested column narrower than the Parquet file's physical type, DataFusion reads every leaf of the column and drops the extra subfields in memory instead of skipping them at read time.
   
   - DefaultPhysicalExprAdapter rewrites the projected column into CAST(col AS narrow_type).
   - Projection mask derivation only understands literal get_field chains, so the cast's inner column expands to every physical leaf via ProjectionMask::roots.
   - Comet reported a production query reading 1.35 TB where plain Spark read 30.9 GB for the same pruned ReadSchema (datafusion-comet#4859).
   - Any embedder that hands DataFusion a pre-pruned schema (Comet, delta-rs, Iceberg integrations) hits the same gap.
   
   This is a port of #23398 onto current main. #23398 (stacked on the merged #23396 and #23397, superseding an earlier attempt at #23392) implements the fix and was reviewed favorably, but has merge conflicts against main since a follow-up refactor moved PushdownChecker and PushdownColumns into projection_read_plan.rs, and has four unanswered review comments. This PR reimplements the same approach against current main and resolves those four comments by construction:
   
   - Drops the nested_projection_pruning config flag; the clip's total fallback design makes a kill switch unnecessary.
   - Drops an unreachable zero-name-overlap code path; validate_struct_compatibility already rejects that case during physical planning.
   - Drops the untested union of a cast clip with a sibling get_field access on the same root, in favor of a full read fallback for that case.
   - Notes that ListView, LargeListView, and Dictionary wrappers are conservatively left unclipped, a candidate follow up.
   
   ## What changes are included in this PR?
   
   - New module datafusion/datasource-parquet/src/nested_schema_pruning.rs. clip_for_cast walks the physical and cast target type trees together, matches struct fields by name, recurses through List and LargeList, and returns the kept leaf offsets plus the pruned Arrow type in one pass. The clip is total: maps, dictionaries, wrapper-kind mismatches, and any shape it does not understand keep every leaf, so the worst case is today's full read.
   - PushdownChecker in projection_read_plan.rs now also collects CastColumnAccess entries for a CastExpr over a plain Column where the cast requires nested struct handling. Collection is opt in and only enabled for projection analysis, so filter pushdown is unchanged.
   - build_projection_read_plan routes to a new build_read_plan_with_cast_clipping when any cast access survives. It partitions referenced roots into whole column reads, cast clipped reads, get_field only reads, and full read fallbacks, including a fallback for a root reached by both a cast and a separate get_field access.
   - Fixed a pre-existing bug in the has_struct_columns fast path: it only tested the top level field type against Struct, so a LIST STRUCT root such as events was misclassified as containing no struct and skipped PushdownChecker entirely, meaning cast based clipping never fired for that shape. Replaced with a recursive contains_struct check through List, LargeList, ListView, LargeListView, FixedSizeList, Map, Dictionary, and RunEndEncoded.
   - No new configuration option.
   
   ## Are these changes tested?
   
   - 15 unit tests in nested_schema_pruning.rs: struct subset, reordering, leaf promotion, missing field null fill, nested struct in struct, list of struct, two levels of list of struct nesting, maps, dictionaries, wrapper mismatches, and an arrow-rs roundtrip test pinning that ProjectionMask::leaves over a subset of List Struct leaves emits exactly the predicted type.
   - 2 new unit tests in projection_read_plan.rs, plus the pre-existing struct preservation test, unchanged.
   - 8 integration tests in datafusion/core/tests/parquet/expr_adapter.rs, asserting identical results and a bytes_scanned drop of more than 2x: list of struct narrowing, top level struct, struct level nullability, get_field on a narrowed struct, mixed whole column and subfield access, filter pushdown enabled, a scan mixing a physically narrow and a wide file, and a regression test built from the ReadSchema and InputSchema shapes reported in datafusion-comet#4859 (a two level list STRUCT column with a dropped struct sibling, a dropped map sibling, and dropped top level columns).
   - New datafusion/sqllogictest/test_files/parquet_nested_schema_pruning.slt covering the end to end SQL path.
   - The existing benchmark (already merged as part of #23397) had an assertion documenting the pre-fix baseline, narrow equals full. That assertion now fails as expected and has been flipped to assert narrow reads less than half of full. Measured for the top level struct case: narrow_schema=1081 bytes vs full_schema=8398289 bytes, matching the physically_narrow floor of 1081 bytes.
   
   ## Are there any user-facing changes?
   
   No API changes and no new configuration option. Behavior is IO reduction only, results are unchanged.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: prune unread Parquet leaves when a nested column is cast to a narrower type [datafusion]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on PR #24090:
URL: https://github.com/apache/datafusion/pull/24090#issuecomment-5194726771

   > I spent some time stress-testing this (mutation testing, randomised differential testing against the arrow-rs reader, and benchmarks on wide schemas) and found one correctness bug plus a few smaller things. I've opened [mbutrovich#1](https://github.com/mbutrovich/datafusion/pull/1) against your branch with fixes and tests β€” fold it in however you like.
   
   Does it address the earlier review comments?
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: prune unread Parquet leaves when a nested column is cast to a narrower type [datafusion]

Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb merged PR #24090:
URL: https://github.com/apache/datafusion/pull/24090


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: prune unread Parquet leaves when a nested column is cast to a narrower type [datafusion]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #24090:
URL: https://github.com/apache/datafusion/pull/24090#issuecomment-5183499872

   ## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/24090?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `96.18474%` with `19 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 80.91%. Comparing base ([`597170e`](https://app.codecov.io/gh/apache/datafusion/commit/597170e925f174040f0c95ecb8d673a6642796df?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`9f7fb5a`](https://app.codecov.io/gh/apache/datafusion/commit/9f7fb5a2cf86027d0ed7fb8c43a3b576c56e8fd1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 2 commits behind head on main.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/datafusion/pull/24090?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [...ion/datasource-parquet/src/projection\_read\_plan.rs](https://app.codecov.io/gh/apache/datafusion/pull/24090?src=pr&el=tree&filepath=datafusion%2Fdatasource-parquet%2Fsrc%2Fprojection_read_plan.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9kYXRhc291cmNlLXBhcnF1ZXQvc3JjL3Byb2plY3Rpb25fcmVhZF9wbGFuLnJz) | 92.82% | [16 Missing :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/24090?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...on/datasource-parquet/src/nested\_schema\_pruning.rs](https://app.codecov.io/gh/apache/datafusion/pull/24090?src=pr&el=tree&filepath=datafusion%2Fdatasource-parquet%2Fsrc%2Fnested_schema_pruning.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9kYXRhc291cmNlLXBhcnF1ZXQvc3JjL25lc3RlZF9zY2hlbWFfcHJ1bmluZy5ycw==) | 98.90% | [3 Missing :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/24090?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@            Coverage Diff             @@
   ##             main   #24090      +/-   ##
   ==========================================
   + Coverage   80.89%   80.91%   +0.02%     
   ==========================================
     Files        1102     1103       +1     
     Lines      376111   376607     +496     
     Branches   376111   376607     +496     
   ==========================================
   + Hits       304248   304725     +477     
   - Misses      53756    53773      +17     
   - Partials    18107    18109       +2     
   ```
   </details>
   
   [:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/24090?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: prune unread Parquet leaves when a nested column is cast to a narrower type [datafusion]

Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on code in PR #24090:
URL: https://github.com/apache/datafusion/pull/24090#discussion_r3718149700


##########
datafusion/datasource-parquet/src/projection_read_plan.rs:
##########
@@ -94,6 +99,13 @@ pub(crate) struct PushdownChecker<'schema> {
     required_columns: Vec<usize>,
     /// Struct field accesses via `get_field`.
     struct_field_accesses: Vec<StructFieldAccess>,
+    /// Whole-column casts to a narrower nested type
+    /// (`CAST(col AS narrower_struct)`). Only collected when
+    /// [`Self::with_cast_collection`] enables it (projection analysis);
+    /// filter pushdown leaves this off.

Review Comment:
   Is it intentional that this not work for filter push down? I don't immediately see any reason why it couldn't work.



##########
datafusion/core/tests/parquet/expr_adapter.rs:
##########
@@ -1204,3 +1204,671 @@ async fn test_physical_expr_adapter_factory_reuse_across_tables() {
     ];
     assert_batches_eq!(expected, &batches);
 }
+
+// ---------------------------------------------------------------------------
+// Nested projection pruning: when the table schema declares a nested column
+// narrower than the physical Parquet file, the scan should only read the
+// leaves the declared schema names, instead of reading the whole column and
+// discarding the extra subfields in the adapter-inserted cast.
+//
+// Each test registers two tables against the *same* physical file: `t_narrow`
+// (the declared schema under test) and `t_full` (the file's own physical
+// schema, so no cast is inserted and the scan always reads every leaf). That
+// gives a same-context upper bound to compare `bytes_scanned` against,
+// without needing a config flag to disable pruning.

Review Comment:
   Good idea πŸ‘πŸ» 



##########
datafusion/datasource-parquet/src/projection_read_plan.rs:
##########
@@ -419,6 +484,160 @@ pub(crate) fn build_projection_read_plan(
     read_plan
 }
 
+/// Builds a [`ParquetReadPlan`] when at least one projected root column is
+/// consumed through a cast to a narrower nested type.
+///
+/// Per root, in ascending root-index order:
+/// - roots referenced as whole columns keep every leaf and their full
+///   physical field (whole-column reads take precedence; cast accesses on
+///   such roots were already dropped by the caller);
+/// - roots consumed through a cast, and not also through a `get_field`
+///   access on the same root, keep only the leaves the cast target names
+///   (see `crate::nested_schema_pruning`);
+/// - roots consumed only through `get_field` accesses keep the union of the
+///   leaves those accesses reach, as before;
+/// - any other referenced root, a cast that can't be safely clipped (see
+///   `nested_schema_pruning::clip_for_cast`), or a root reached by both a
+///   cast and a `get_field` access (not produced by
+///   `DefaultPhysicalExprAdapter`, which always routes a `get_field` over a
+///   narrowed column through the same cast rather than a separate access,
+///   but a custom `PhysicalExprAdapter` could in principle inject both),
+///   falls back to a full read of that root.
+fn build_read_plan_with_cast_clipping(

Review Comment:
   This seems like a possible source of fragility, e.g. because of the indirect coupling to `DefaultPhysicalExprAdapter`. I think the failure modes are okay (loss of optimization) and we can fix bugs / edge cases if they are added in the future. For now though it's worth measuring coverage on this function and making sure it's well tested.



##########
datafusion/sqllogictest/test_files/parquet_nested_schema_pruning.slt:
##########
@@ -0,0 +1,83 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+##########
+# Nested projection pruning: a table whose declared nested type is narrower
+# than the Parquet file's physical type reads only the declared leaves.
+# The bytes-scanned assertions live in the Rust tests
+# (datafusion/core/tests/parquet/expr_adapter.rs); this file covers the
+# end-to-end SQL correctness path.

Review Comment:
   Thank you. I'll try to check that we have full coverage via SLT tests and if there is any low hanging fruit etsting we can add as SLTs.



##########
datafusion/datasource-parquet/src/projection_read_plan.rs:
##########
@@ -419,6 +484,160 @@ pub(crate) fn build_projection_read_plan(
     read_plan
 }
 
+/// Builds a [`ParquetReadPlan`] when at least one projected root column is
+/// consumed through a cast to a narrower nested type.
+///
+/// Per root, in ascending root-index order:
+/// - roots referenced as whole columns keep every leaf and their full
+///   physical field (whole-column reads take precedence; cast accesses on
+///   such roots were already dropped by the caller);
+/// - roots consumed through a cast, and not also through a `get_field`
+///   access on the same root, keep only the leaves the cast target names
+///   (see `crate::nested_schema_pruning`);
+/// - roots consumed only through `get_field` accesses keep the union of the
+///   leaves those accesses reach, as before;
+/// - any other referenced root, a cast that can't be safely clipped (see
+///   `nested_schema_pruning::clip_for_cast`), or a root reached by both a
+///   cast and a `get_field` access (not produced by
+///   `DefaultPhysicalExprAdapter`, which always routes a `get_field` over a
+///   narrowed column through the same cast rather than a separate access,
+///   but a custom `PhysicalExprAdapter` could in principle inject both),
+///   falls back to a full read of that root.
+fn build_read_plan_with_cast_clipping(

Review Comment:
   I also want to check that all operations are O(projected_columns) or O(logical_plan_column) and ideally not O(schema) in some way, or at least not e.g. cuadratic with schema or struct column size.



##########
datafusion/core/tests/parquet/expr_adapter.rs:
##########
@@ -1204,3 +1204,671 @@ async fn test_physical_expr_adapter_factory_reuse_across_tables() {
     ];
     assert_batches_eq!(expected, &batches);
 }
+
+// ---------------------------------------------------------------------------
+// Nested projection pruning: when the table schema declares a nested column
+// narrower than the physical Parquet file, the scan should only read the
+// leaves the declared schema names, instead of reading the whole column and
+// discarding the extra subfields in the adapter-inserted cast.
+//
+// Each test registers two tables against the *same* physical file: `t_narrow`
+// (the declared schema under test) and `t_full` (the file's own physical
+// schema, so no cast is inserted and the scan always reads every leaf). That
+// gives a same-context upper bound to compare `bytes_scanned` against,
+// without needing a config flag to disable pruning.
+// ---------------------------------------------------------------------------
+
+mod nested_projection_pruning {
+    use super::*;
+    use arrow::buffer::NullBuffer;
+    use datafusion::physical_plan::collect;
+    use datafusion_physical_plan::metrics::MetricsSet;
+
+    use crate::parquet::utils::MetricsFinder;
+
+    const NUM_ELEMENTS: usize = 64;
+    const PAD_LEN: usize = 2048;
+
+    /// Physical item struct written to the file: the narrow fields plus fat
+    /// pads the narrow table schema will not mention. `x` is Int32 in the
+    /// file (the narrow schema declares Int64 to also exercise leaf
+    /// promotion).
+    fn wide_item_fields() -> Fields {
+        Fields::from(vec![
+            Field::new("x", DataType::Int32, false),
+            Field::new("y", DataType::Utf8, true),
+            Field::new("pad_a", DataType::Utf8, false),
+            Field::new("pad_b", DataType::Utf8, false),
+            Field::new("pad_c", DataType::Utf8, false),
+        ])
+    }
+
+    /// The narrow item struct one table declares: a subset of the physical
+    /// fields in a different order, a promoted leaf type for `x`, plus `z`
+    /// which does not exist in the file (null-filled by the cast).
+    fn narrow_item_fields() -> Fields {
+        Fields::from(vec![
+            Field::new("y", DataType::Utf8, true),
+            Field::new("x", DataType::Int64, true),
+            Field::new("z", DataType::Int64, true),
+        ])
+    }
+
+    fn wide_struct_values(validity: Option<NullBuffer>) -> StructArray {
+        let pad = |seed: usize| {
+            let base = "x".repeat(PAD_LEN);
+            Arc::new(StringArray::from_iter_values(
+                (0..NUM_ELEMENTS).map(|i| format!("{}{base}", seed + i)),
+            )) as ArrayRef
+        };
+        StructArray::new(
+            wide_item_fields(),
+            vec![
+                Arc::new(Int32Array::from_iter_values(0..NUM_ELEMENTS as i32)),
+                Arc::new(StringArray::from_iter_values(
+                    (0..NUM_ELEMENTS).map(|i| format!("y-{i}")),
+                )),
+                pad(1000),
+                pad(2000),
+                pad(3000),
+            ],
+            validity,
+        )
+    }
+
+    fn wide_list_schema() -> SchemaRef {
+        let item = Arc::new(Field::new(
+            "item",
+            DataType::Struct(wide_item_fields()),
+            true,
+        ));
+        Arc::new(Schema::new(vec![
+            Field::new("id", DataType::Int32, false),
+            Field::new("events", DataType::List(item), true),
+        ]))
+    }
+
+    /// File batch: `id Int32`, `events List<wide struct>` (one element per row).
+    fn wide_list_batch() -> RecordBatch {
+        let schema = wide_list_schema();
+        let item = match schema.field(1).data_type() {
+            DataType::List(item) => Arc::clone(item),
+            other => unreachable!("expected List, got {other:?}"),
+        };
+        let events = ListArray::new(
+            item,
+            OffsetBuffer::from_lengths(std::iter::repeat_n(1, NUM_ELEMENTS)),
+            Arc::new(wide_struct_values(None)),
+            None,
+        );
+        RecordBatch::try_new(
+            Arc::clone(&schema),
+            vec![
+                Arc::new(Int32Array::from_iter_values(0..NUM_ELEMENTS as i32)),
+                Arc::new(events),
+            ],
+        )
+        .unwrap()
+    }
+
+    fn narrow_list_table_schema() -> SchemaRef {
+        let item = Arc::new(Field::new(
+            "item",
+            DataType::Struct(narrow_item_fields()),
+            true,
+        ));
+        Arc::new(Schema::new(vec![
+            Field::new("id", DataType::Int32, false),
+            Field::new("events", DataType::List(item), true),
+        ]))
+    }
+
+    fn wide_struct_schema() -> SchemaRef {
+        Arc::new(Schema::new(vec![
+            Field::new("id", DataType::Int32, false),
+            Field::new("s", DataType::Struct(wide_item_fields()), true),
+        ]))
+    }
+
+    /// File batch: `id Int32`, `s <wide struct>`, with per-row struct
+    /// validity so struct-level nullability can be asserted.
+    fn wide_struct_batch() -> RecordBatch {
+        // rows 0, 10, 20, ... have a NULL struct
+        let validity =
+            NullBuffer::from((0..NUM_ELEMENTS).map(|i| i % 10 != 0).collect::<Vec<_>>());
+        RecordBatch::try_new(
+            wide_struct_schema(),
+            vec![
+                Arc::new(Int32Array::from_iter_values(0..NUM_ELEMENTS as i32)),
+                Arc::new(wide_struct_values(Some(validity))),
+            ],
+        )
+        .unwrap()
+    }
+
+    fn narrow_struct_table_schema() -> SchemaRef {
+        Arc::new(Schema::new(vec![
+            Field::new("id", DataType::Int32, false),
+            Field::new("s", DataType::Struct(narrow_item_fields()), true),
+        ]))
+    }
+
+    /// Registers `t_narrow` (the schema under test) and `t_full` (the file's
+    /// own physical schema, so no cast is inserted) against the same store.
+    async fn register_narrow_and_full(
+        ctx: &SessionContext,
+        store: Arc<dyn ObjectStore>,
+        narrow_schema: SchemaRef,
+        full_schema: SchemaRef,
+    ) {
+        let store_url = ObjectStoreUrl::parse("memory://").unwrap();
+        ctx.register_object_store(store_url.as_ref(), store);
+
+        for (name, schema) in [("t_narrow", narrow_schema), ("t_full", full_schema)] {
+            let config = ListingTableConfig::new(
+                ListingTableUrl::parse("memory:///data/").unwrap(),
+            )
+            .infer_options(&ctx.state())
+            .await
+            .unwrap()
+            .with_schema(schema)
+            .with_expr_adapter_factory(Arc::new(DefaultPhysicalExprAdapterFactory));
+            let table = ListingTable::try_new(config).unwrap();
+            ctx.register_table(name, Arc::new(table)).unwrap();
+        }
+    }
+
+    async fn setup_with_config(
+        batches: Vec<(&str, RecordBatch)>,
+        narrow_schema: SchemaRef,
+        full_schema: SchemaRef,
+        cfg: SessionConfig,
+    ) -> SessionContext {
+        let store = Arc::new(InMemory::new()) as Arc<dyn ObjectStore>;
+        for (name, batch) in batches {
+            write_parquet(batch, Arc::clone(&store), &format!("data/{name}")).await;
+        }
+        let ctx = SessionContext::new_with_config(cfg);
+        register_narrow_and_full(&ctx, store, narrow_schema, full_schema).await;
+        ctx
+    }
+
+    async fn setup(
+        batches: Vec<(&str, RecordBatch)>,
+        narrow_schema: SchemaRef,
+        full_schema: SchemaRef,
+    ) -> SessionContext {
+        setup_with_config(
+            batches,
+            narrow_schema,
+            full_schema,
+            SessionConfig::new().with_collect_statistics(false),
+        )
+        .await
+    }
+
+    async fn run(ctx: &SessionContext, sql: &str) -> (Vec<RecordBatch>, MetricsSet) {
+        let df = ctx.sql(sql).await.unwrap();
+        let (state, logical) = df.into_parts();
+        let plan = state.create_physical_plan(&logical).await.unwrap();
+        let batches = collect(Arc::clone(&plan), state.task_ctx()).await.unwrap();
+        let metrics = MetricsFinder::find_metrics(plan.as_ref()).unwrap();
+        (batches, metrics)
+    }
+
+    fn bytes_scanned(metrics: &MetricsSet) -> usize {
+        metrics
+            .sum(|m| m.value().name() == "bytes_scanned")
+            .map(|v| v.as_usize())
+            .expect("bytes_scanned metric")
+    }
+
+    /// Run `narrow_sql` against `t_narrow` and `full_sql` against `t_full`;
+    /// assert the narrow scan read strictly less than half of the full
+    /// scan's bytes (the pads dominate the file), and return the narrow
+    /// scan's results for correctness assertions.
+    ///
+    /// The two SQL strings need not have the same shape: a `get_field` over
+    /// a narrowed struct clips to the *cast target*, not further down to the
+    /// specific field accessed (see `prunes_get_field_on_narrowed_struct`),
+    /// so comparing against the same `get_field` query on `t_full` would
+    /// unfairly compare this clip against `get_field`'s own, more precise,
+    /// single-leaf pruning (which only applies when there is no cast in the
+    /// way). Callers that aren't in that situation can just pass the same
+    /// query shape with the table name substituted.
+    async fn assert_prunes(
+        batches: Vec<(&str, RecordBatch)>,
+        narrow_schema: SchemaRef,
+        full_schema: SchemaRef,
+        narrow_sql: &str,
+        full_sql: &str,
+    ) -> Vec<RecordBatch> {
+        let ctx = setup(batches, narrow_schema, full_schema).await;
+
+        let (result_narrow, metrics_narrow) = run(&ctx, narrow_sql).await;
+        let (_result_full, metrics_full) = run(&ctx, full_sql).await;
+
+        let (narrow_bytes, full_bytes) =
+            (bytes_scanned(&metrics_narrow), bytes_scanned(&metrics_full));
+        assert!(
+            narrow_bytes * 2 < full_bytes,
+            "expected pruned scan to read less than half of {full_bytes} bytes, \
+             read {narrow_bytes}: {narrow_sql}"
+        );
+        result_narrow
+    }
+
+    #[tokio::test]
+    async fn prunes_list_of_struct() {
+        // Narrow schema over the wide file: subset of fields, reordered,
+        // promoted leaf (x: Int32 -> Int64), missing subfield z null-filled.
+        let batches = assert_prunes(
+            vec![("wide.parquet", wide_list_batch())],
+            narrow_list_table_schema(),
+            wide_list_schema(),
+            "SELECT events FROM t_narrow ORDER BY id",
+            "SELECT events FROM t_full ORDER BY id",
+        )
+        .await;
+
+        let events = batches[0].column(0);
+        let list = events.as_any().downcast_ref::<ListArray>().unwrap();
+        let items = list
+            .values()
+            .as_any()
+            .downcast_ref::<StructArray>()
+            .unwrap();
+        assert_eq!(items.fields().len(), 3);
+        let x = items
+            .column_by_name("x")
+            .unwrap()
+            .as_any()
+            .downcast_ref::<Int64Array>()
+            .unwrap();
+        assert_eq!(x.value(5), 5);
+        let z = items.column_by_name("z").unwrap();
+        assert_eq!(z.null_count(), z.len(), "z is not in the file");
+    }
+
+    #[tokio::test]
+    async fn prunes_top_level_struct() {
+        assert_prunes(
+            vec![("wide.parquet", wide_struct_batch())],
+            narrow_struct_table_schema(),
+            wide_struct_schema(),
+            "SELECT s FROM t_narrow ORDER BY id",
+            "SELECT s FROM t_full ORDER BY id",
+        )
+        .await;
+    }
+
+    /// Struct-level nullability must survive the clip: rows where the struct
+    /// itself is NULL stay NULL (not `{y: NULL, x: NULL, z: NULL}`).
+    #[tokio::test]
+    async fn preserves_struct_nullability() {
+        let batches = assert_prunes(
+            vec![("wide.parquet", wide_struct_batch())],
+            narrow_struct_table_schema(),
+            wide_struct_schema(),
+            "SELECT id, s IS NULL AS s_null, s FROM t_narrow ORDER BY id",
+            "SELECT id, s IS NULL AS s_null, s FROM t_full ORDER BY id",
+        )
+        .await;
+
+        let combined = concat_batches(&batches[0].schema(), &batches).unwrap();
+        let s_null = combined
+            .column(1)
+            .as_any()
+            .downcast_ref::<BooleanArray>()
+            .unwrap();
+        for i in 0..NUM_ELEMENTS {
+            assert_eq!(s_null.value(i), i % 10 == 0, "row {i}");
+        }
+    }
+
+    /// `get_field` on a schema-narrowed struct becomes
+    /// `get_field(CAST(s), 'x')`; the read clips to the cast target (every
+    /// field the *narrow* schema declares), not further down to just `x`.
+    /// The fair "no clipping happened" baseline is therefore reading every
+    /// physical leaf of `s` (`SELECT s FROM t_full`), not the same
+    /// `get_field` query against `t_full`. That query needs no cast at all
+    /// and takes `get_field`'s own, more precise, single-leaf pushdown path.
+    #[tokio::test]
+    async fn prunes_get_field_on_narrowed_struct() {
+        let batches = assert_prunes(
+            vec![("wide.parquet", wide_struct_batch())],
+            narrow_struct_table_schema(),
+            wide_struct_schema(),
+            "SELECT s['x'] AS x FROM t_narrow ORDER BY id",
+            "SELECT s FROM t_full ORDER BY id",
+        )
+        .await;
+        let combined = concat_batches(&batches[0].schema(), &batches).unwrap();
+        let x = combined
+            .column(0)
+            .as_any()
+            .downcast_ref::<Int64Array>()
+            .unwrap();
+        assert_eq!(x.value(5), 5);
+        assert_eq!(x.value(NUM_ELEMENTS - 1), NUM_ELEMENTS as i64 - 1);
+    }
+
+    /// Mixed access: the whole (narrowed) column and a subfield of it.
+    #[tokio::test]
+    async fn prunes_mixed_struct_and_subfield_access() {
+        assert_prunes(
+            vec![("wide.parquet", wide_struct_batch())],
+            narrow_struct_table_schema(),
+            wide_struct_schema(),
+            "SELECT s, s['y'] AS y FROM t_narrow ORDER BY id",
+            "SELECT s, s['y'] AS y FROM t_full ORDER BY id",
+        )
+        .await;
+    }
+
+    /// Predicate on a primitive column with filter pushdown enabled while
+    /// the projected nested column is clipped.
+    #[tokio::test]
+    async fn prunes_with_filter_pushdown() {
+        let mut cfg = SessionConfig::new().with_collect_statistics(false);
+        cfg.options_mut().execution.parquet.pushdown_filters = true;
+        let ctx = setup_with_config(
+            vec![("wide.parquet", wide_list_batch())],
+            narrow_list_table_schema(),
+            wide_list_schema(),
+            cfg,
+        )
+        .await;
+
+        let filter = "WHERE id >= 32 ORDER BY id";
+        let (result_narrow, metrics_narrow) =
+            run(&ctx, &format!("SELECT events FROM t_narrow {filter}")).await;
+        let (_result_full, metrics_full) =
+            run(&ctx, &format!("SELECT events FROM t_full {filter}")).await;
+
+        let combined =
+            concat_batches(&result_narrow[0].schema(), &result_narrow).unwrap();
+        assert_eq!(combined.num_rows(), NUM_ELEMENTS / 2);
+        assert!(bytes_scanned(&metrics_narrow) * 2 < bytes_scanned(&metrics_full));
+    }
+
+    /// A scan over two files where one matches the table schema exactly (no
+    /// cast is inserted) and one is wider (clipped): both must be read
+    /// correctly in the same scan.
+    #[tokio::test]
+    async fn mixed_files_narrow_and_wide() {
+        // The physically-narrow file has exactly the table's item struct.
+        let narrow_item = narrow_item_fields();
+        let item = Arc::new(Field::new(
+            "item",
+            DataType::Struct(narrow_item.clone()),
+            true,
+        ));
+        let events = ListArray::new(
+            Arc::clone(&item),
+            OffsetBuffer::from_lengths([1]),
+            Arc::new(StructArray::new(
+                narrow_item,
+                vec![
+                    Arc::new(StringArray::from(vec![Some("y-narrow")])) as ArrayRef,
+                    Arc::new(Int64Array::from(vec![Some(4242)])) as ArrayRef,
+                    Arc::new(Int64Array::from(vec![Some(7)])) as ArrayRef,
+                ],
+                None,
+            )),
+            None,
+        );
+        let narrow_batch = RecordBatch::try_new(
+            narrow_list_table_schema(),
+            vec![
+                Arc::new(Int32Array::from(vec![NUM_ELEMENTS as i32])),
+                Arc::new(events),
+            ],
+        )
+        .unwrap();
+
+        let store = Arc::new(InMemory::new()) as Arc<dyn ObjectStore>;
+        write_parquet(wide_list_batch(), Arc::clone(&store), "data/wide.parquet").await;
+        write_parquet(narrow_batch, Arc::clone(&store), "data/narrow.parquet").await;
+
+        let ctx = test_context();
+        register_memory_listing_table(
+            &ctx,
+            store,
+            "memory:///data/",
+            narrow_list_table_schema(),
+        )
+        .await;
+
+        let (batches, _) = run(
+            &ctx,
+            "SELECT id, e['x'] AS x, e['z'] AS z \
+             FROM (SELECT id, unnest(events) AS e FROM t) ORDER BY id",
+        )
+        .await;
+        let combined = concat_batches(&batches[0].schema(), &batches).unwrap();
+        assert_eq!(combined.num_rows(), NUM_ELEMENTS + 1);
+        let x = combined
+            .column(1)
+            .as_any()
+            .downcast_ref::<Int64Array>()
+            .unwrap();
+        assert_eq!(x.value(NUM_ELEMENTS), 4242, "row from the narrow file");
+        let z = combined.column(2);
+        // z is null-filled for the wide file, present in the narrow file
+        assert_eq!(z.null_count(), NUM_ELEMENTS);
+    }
+
+    /// Regression test for the exact shape reported in
+    /// `datafusion-comet#4859`: a two-level `array<struct<...,

Review Comment:
   Can we make this a real link?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: prune unread Parquet leaves when a nested column is cast to a narrower type [datafusion]

Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on code in PR #24090:
URL: https://github.com/apache/datafusion/pull/24090#discussion_r3714396359


##########
datafusion/sqllogictest/test_files/parquet_nested_schema_pruning.slt:
##########
@@ -0,0 +1,83 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+##########
+# Nested projection pruning: a table whose declared nested type is narrower
+# than the Parquet file's physical type reads only the declared leaves.
+# The bytes-scanned assertions live in the Rust tests
+# (datafusion/core/tests/parquet/expr_adapter.rs); this file covers the
+# end-to-end SQL correctness path.

Review Comment:
   Can we somehow assert that the pruning worked / the pad columns where not read?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: prune unread Parquet leaves when a nested column is cast to a narrower type [datafusion]

Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #24090:
URL: https://github.com/apache/datafusion/pull/24090#issuecomment-5195626057

   > > I spent some time stress-testing this (mutation testing, randomised differential testing against the arrow-rs reader, and benchmarks on wide schemas) and found one correctness bug plus a few smaller things. I've opened [mbutrovich#1](https://github.com/mbutrovich/datafusion/pull/1) against your branch with fixes and tests β€” fold it in however you like.
   > 
   > Does it address the earlier review comments?
   
   It addresses the coverage aspect, it doesn't implement this for filter pushdown, I think we should defer that to followups. I think if we merge that into here we can then merge this into main and track a couple of followups, namely:
   - Getting this optimization to work for filter pushdown
   - Taking the union of fields from a cast instead of reading the whole column


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: prune unread Parquet leaves when a nested column is cast to a narrower type [datafusion]

Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #24090:
URL: https://github.com/apache/datafusion/pull/24090#issuecomment-5194655874

   I spent some time stress-testing this (mutation testing, randomised differential testing against the arrow-rs reader, and benchmarks on wide schemas) and found one correctness bug plus a few smaller things. I've opened mbutrovich/datafusion#1 against your branch with fixes and tests β€” fold it in however you like.
   
   ### Two cast targets on one column read too few leaves
   
   `build_read_plan_with_cast_clipping` skips every cast access on a root it has already clipped:
   
   ```rust
   if whole_roots.contains(&root)
       || fallback_roots.contains(&root)
       || clipped_by_root.contains_key(&root)   // <-- second cast on this root: dropped
   { continue; }
   ```
   
   If a projection consumes one column through two *different* narrowing casts, only the first target's leaves reach the mask, and the second cast then evaluates against a struct missing the children it names:
   
   * overlapping targets β†’ `cast_column` null-fills them β†’ **silently wrong results**
   * disjoint targets β†’ `validate_struct_compatibility` rejects it β†’ the query **fails** where it succeeds without pruning
   
   I originally assumed this needed a custom adapter, but it's reachable from plain SQL: `ProjectionExec` is merged into the scan via `ParquetSource::try_pushdown_projection`, so a query-level `CAST(col AS STRUCT<...>)` reaches the same analysis as an adapter-inserted one. Against a table whose schema is inferred from the file (so no adapter cast is interposed):
   
   ```sql
   -- file: s Struct<x Int64, y Utf8, pad Utf8>
   SELECT CAST(s AS STRUCT<x BIGINT>), CAST(s AS STRUCT<x BIGINT, y VARCHAR>) FROM t;
   -- {x: 100} {x: 100, y: NULL}   <- y should be 's1'
   
   SELECT CAST(s AS STRUCT<x BIGINT>), CAST(s AS STRUCT<pad VARCHAR>) FROM t;
   -- Error: Cannot cast struct with 1 fields to 1 fields because there is no field name overlap
   ```
   
   Both return correct results with `with_cast_collection` disabled, so they're regressions rather than pre-existing behaviour. The fix in the linked PR demotes the root to a full read when a second cast has a different target; identical repeated targets (what the expression adapter actually produces) still clip.
   
   ### Smaller things
   
   * **A zero-overlap nested struct level predicts a type the reader never emits.** `clip_type` can return `Struct([])` for a name-matched child whose own children share no name with their target, while keeping that child in the emitted type β€” but arrow-rs drops a struct child whose leaves are all masked out (`complex.rs`, `if children.is_empty() { return Ok(None) }`). I couldn't reach it through the default stack (`validate_struct_compatibility` and the logical planner both reject the cast first), so it isn't live today. But the module doc's safety argument rests on a *caller* invariant and the motivating use case is a custom `PhysicalExprAdapter`, so I'd rather `clip_for_cast` enforce it itself. Test added pinning the arrow-rs behaviour the argument depends on.
   * **`leaves_by_root[root]` is a panicking `BTreeMap` index**, on a path where the same function already guards the "root with no parquet leaves" case twenty lines earlier. Latent β€” I couldn't build a file that reaches it β€” but a one-line fix.
   
   ### Performance
   
   Three things weren't `O(projected_columns)`:
   
   * `clip_type` matches struct fields with a linear `find` per physical child β€” Θ(NΒ·M) string comparisons per struct level, ~375k for a 1000β†’500 subfield struct. Spark's `ParquetReadSupport.clipParquetGroupFields` uses a nameβ†’field map for exactly this; the PR does the same above a small width.
   * Widening the fast-path gate to `contains_struct` correctly fixes the `List<Struct>` hole, but it also routes `Map`, `Dictionary<_, Struct>` and every array-of-records column through `PushdownChecker`, whose `Schema::index_of` is a linear name scan per column node β€” O(|exprs| Β· |schema|), paid once per file opened. Gating on the *projected* roots instead keeps the `List<Struct>` fix and is O(projected).
   * An O(GΒ²) `field_with_name` re-lookup for `get_field` roots (also picks the wrong field under duplicate root names).
   
   New benchmark `parquet_wide_schema_read_plan` β€” 1000-column schemas and 1000-subfield structs over 32 small files, so per-file planning dominates. Before β†’ after, back to back, with two untouched-path controls so the numbers mean something:
   
   | case | change |
   | --- | --- |
   | `flat_wide/column_projection` (control) | βˆ’0.1% (p = 0.93) |
   | `wide_struct/full_schema` (control) | +1.1% (p = 0.63) |
   | `wide_struct/unrelated_column` β€” wide struct present but not projected | **βˆ’13.6%** (p = 0.01) |
   | `wide_struct/narrowed_schema` β€” 1000 β†’ 500 subfield clip | **βˆ’7.2%** (p = 0.01) |
   
   The feature holds up well at that width: 90.8 ms narrowed vs 166.2 ms full, ~45% less wall time.
   
   ### Coverage
   
   `cargo-mutants` over the two files: 121 mutants, 82 viable, 16 survivors, of which 8 were real gaps β€” including that `start + o` (rebasing clip offsets onto absolute leaf indices) survives `start - o`, because every clip test casts to the struct's *first* field where the two are identical; and that the whole `get_field_accesses` branch of `build_read_plan_with_cast_clipping` is never executed by any test. All eight now have tests, each verified by re-applying the mutation.
   
   I also added two seeded randomised differential harnesses β€” one checking `clip_for_cast`'s predicted type and kept leaves against the real reader, one comparing a clipped scan against `cast_column` over an unclipped one end to end. ~63k and ~25k cases respectively, no further discrepancies. They're cheap by default (~3.5 s / ~1.2 s) and widen by env var.
   
   SLT coverage went from 110 to ~380 lines, moving most of the Rust-only assertions into SQL: `SELECT *`, mixed whole-column and field access, aggregation and filtering over a clipped column, a declared field order differing from the file's, struct-in-struct, the two-level `ARRAY<STRUCT<..., ARRAY<STRUCT<..>>>>` shape from the Comet issue, a `MAP` sibling, a scan mixing narrow and wide files, and the zero-overlap rejection.
   
   ### One thing that isn't yours
   
   While writing the SLT tests I hit a pre-existing soundness bug: with `pushdown_filters = true`, a `get_field` predicate is silently dropped whenever the file needs schema adaptation, so the query returns every row. It's not caused by this PR (`row_filter.rs` is untouched), but every table this feature targets carries exactly the cast that triggers it, so it's worth knowing about before this lands. Filed separately as #24109.
   
   ---
   _Generated by [Claude Code](https://claude.ai/code)_
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] feat: prune unread Parquet leaves when a nested column is cast to a narrower type [datafusion]

Posted by "mbutrovich (via GitHub)" <gi...@apache.org>.
mbutrovich commented on code in PR #24090:
URL: https://github.com/apache/datafusion/pull/24090#discussion_r3714736621


##########
datafusion/sqllogictest/test_files/parquet_nested_schema_pruning.slt:
##########
@@ -0,0 +1,83 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+##########
+# Nested projection pruning: a table whose declared nested type is narrower
+# than the Parquet file's physical type reads only the declared leaves.
+# The bytes-scanned assertions live in the Rust tests
+# (datafusion/core/tests/parquet/expr_adapter.rs); this file covers the
+# end-to-end SQL correctness path.

Review Comment:
   Added a full_schema table (no cast) and pinned bytes_scanned as a literal: 172 narrow vs 312 full. Rest stays masked, like limit_pruning.slt:103.   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org