You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "kosiew (via GitHub)" <gi...@apache.org> on 2026/08/13 11:49:29 UTC
[PR] Add focused benchmark for Parquet pruning setup cache [datafusion]
kosiew opened a new pull request, #24323:
URL: https://github.com/apache/datafusion/pull/24323
## Which issue does this PR close?
* Part of #21554
## Rationale for this change
`ParquetPruningSetupCache` in #21566 needs a focused benchmark that exercises it under deliberately cache-favourable conditions.
This benchmark provides a workload where many Parquet files share the same physical schema and use the same predicate, target partition count, and query, creating repeated opportunities to reuse pruning setup. It is intended to measure end-to-end scan cost for this specific workload shape, rather than represent general DataFusion performance or predict improvements in broader benchmarks such as ClickBench.
## What changes are included in this PR?
* Adds a Criterion benchmark that creates 128 Parquet files sharing one physical schema and repeatedly executes `SELECT id FROM t WHERE id >= 0`.
* Keeps file generation, runtime creation, table registration, and initial correctness validation outside the timed Criterion loop.
* Validates that the query returns all 16,384 expected rows before benchmarking.
* Enables Criterion's `async_tokio` feature so the asynchronous query execution can be benchmarked with the Tokio runtime.
* Registers the new `parquet_pruning_setup_cache` benchmark target.
* Documents that the benchmark is intentionally cache-favourable and that its results should not be interpreted as general DataFusion or ClickBench performance results.
## Are these changes tested?
The benchmark includes a correctness check that verifies the query returns `128 × 128 = 16,384` rows before the timed benchmark begins.
No additional tests are included in this patch.
## Are there any user-facing changes?
No. This PR adds a focused benchmark and benchmark configuration only; it does not change user-facing DataFusion behavior or public APIs.
## LLM-generated code disclosure
This PR includes LLM-generated code and comments. All LLM-generated content has been manually reviewed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add focused benchmark for Parquet pruning setup cache [datafusion]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #24323:
URL: https://github.com/apache/datafusion/pull/24323#issuecomment-5280291723
## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/24323?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 81.14%. Comparing base ([`c08832d`](https://app.codecov.io/gh/apache/datafusion/commit/c08832d481cea2dcea98e43393e3dd640d421064?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5b62f3e`](https://app.codecov.io/gh/apache/datafusion/commit/5b62f3ecbcbe0cc66de58556d134093f6872c1af?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 12 commits behind head on main.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #24323 +/- ##
==========================================
- Coverage 81.29% 81.14% -0.16%
==========================================
Files 1110 1112 +2
Lines 385205 386933 +1728
Branches 385205 386933 +1728
==========================================
+ Hits 313145 313963 +818
- Misses 53580 54475 +895
- Partials 18480 18495 +15
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/24323?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add focused benchmark for Parquet pruning setup cache [datafusion]
Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew commented on code in PR #24323:
URL: https://github.com/apache/datafusion/pull/24323#discussion_r3782420984
##########
benchmarks/benches/parquet_pruning_setup_cache.rs:
##########
@@ -0,0 +1,110 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Benchmarks end-to-end Parquet scan cost for a cache-favourable workload.
+//!
+//! The 128 files share one physical schema and use the same predicate and target
Review Comment:
> what would happen if the schema is not the same?
Different physical file schemas produce distinct `ParquetPruningSetupCache` keys in https://github.com/apache/datafusion/pull/21566, so their setup is not reused across schema variants.
> or schema is already merged?
Merged table schema is also not the targeted scenario in #21554.
This benchmark's purpose is to evaluate whether addressing #21554 improves performance: that issue targets avoiding repeated per-file schema adaptation, predicate rewrites, and pruning-predicate construction when many files have the **same physical schema**.
Schema evolution/merged-schema behavior is a separate workload and would dilute this focused benchmark.
Existing [clickbench_partitioned benchmarks](https://github.com/apache/datafusion/pull/21566#issuecomment-5264261199) do not show significant performance, I think because:
- pruning setup work constitutes a small percentage of the work
- the scenarios do not reuse cache
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add focused benchmark for Parquet pruning setup cache [datafusion]
Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew commented on PR #24323:
URL: https://github.com/apache/datafusion/pull/24323#issuecomment-5311200648
@comphead
Thanks for the review and feedback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add focused benchmark for Parquet pruning setup cache [datafusion]
Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew merged PR #24323:
URL: https://github.com/apache/datafusion/pull/24323
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add focused benchmark for Parquet pruning setup cache [datafusion]
Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead commented on code in PR #24323:
URL: https://github.com/apache/datafusion/pull/24323#discussion_r3777269111
##########
benchmarks/benches/parquet_pruning_setup_cache.rs:
##########
@@ -0,0 +1,110 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Benchmarks end-to-end Parquet scan cost for a cache-favourable workload.
+//!
+//! The 128 files share one physical schema and use the same predicate and target
Review Comment:
what would happen if the schema is not the same? or schema is already merged?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add focused benchmark for Parquet pruning setup cache [datafusion]
Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew commented on code in PR #24323:
URL: https://github.com/apache/datafusion/pull/24323#discussion_r3782420984
##########
benchmarks/benches/parquet_pruning_setup_cache.rs:
##########
@@ -0,0 +1,110 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Benchmarks end-to-end Parquet scan cost for a cache-favourable workload.
+//!
+//! The 128 files share one physical schema and use the same predicate and target
Review Comment:
> what would happen if the schema is not the same?
Different physical file schemas produce distinct `ParquetPruningSetupCache` keys in https://github.com/apache/datafusion/pull/21566, so their setup is not reused across schema variants.
> or schema is already merged?
Merged table schema is also not the targeted scenario in #21554.
This benchmark's purpose is to evaluate whether addressing #21554 improves performance: that issue targets avoiding repeated per-file schema adaptation, predicate rewrites, and pruning-predicate construction when many files have the **same physical schema**.
Schema evolution/merged-schema behavior is a separate workload and would dilute this focused benchmark.
Existing [clickbench_partitioned benchmarks](https://github.com/apache/datafusion/pull/21566#issuecomment-5264261199) do not show significant improvement, I think because:
- pruning setup work constitutes a small percentage of the work
- the scenarios do not reuse cache
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] Add focused benchmark for Parquet pruning setup cache [datafusion]
Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew commented on code in PR #24323:
URL: https://github.com/apache/datafusion/pull/24323#discussion_r3782420984
##########
benchmarks/benches/parquet_pruning_setup_cache.rs:
##########
@@ -0,0 +1,110 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Benchmarks end-to-end Parquet scan cost for a cache-favourable workload.
+//!
+//! The 128 files share one physical schema and use the same predicate and target
Review Comment:
> what would happen if the schema is not the same?
Different physical file schemas produce distinct `ParquetPruningSetupCache` keys in https://github.com/apache/datafusion/pull/21566, so their setup is not reused across schema variants.
> or schema is already merged?
Merged table schema is also not the targeted scenario in #21554.
This benchmark's purpose is to evaluate whether addressing #21554 improves performance: that issue targets avoiding repeated per-file schema adaptation, predicate rewrites, and pruning-predicate construction when many files have the **same physical schema**.
Schema evolution/merged-schema behavior is a separate workload and would dilute this focused benchmark.
[clickbench_partitioned benchmarks](https://github.com/apache/datafusion/pull/21566#issuecomment-5264261199) do not show significant performance, I think because:
- pruning setup work constitutes a small percentage of the work
- the scenarios do not reuse cache
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org