You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/08/14 05:45:38 UTC
(datafusion) branch gh-readonly-queue/main/pr-24328-0a429a37dbdf735c4d91efbce68a0fdcb271ea46 created (now 6adb504363)
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a change to branch gh-readonly-queue/main/pr-24328-0a429a37dbdf735c4d91efbce68a0fdcb271ea46
in repository https://gitbox.apache.org/repos/asf/datafusion.git
at 6adb504363 bench: add parquet_row_filter_skip suite for the fully-matched RowFilter skip (#24328)
This branch includes the following new commits:
new 6adb504363 bench: add parquet_row_filter_skip suite for the fully-matched RowFilter skip (#24328)
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org
(datafusion) 01/01: bench: add parquet_row_filter_skip suite for the fully-matched RowFilter skip (#24328)
Posted by gi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-24328-0a429a37dbdf735c4d91efbce68a0fdcb271ea46
in repository https://gitbox.apache.org/repos/asf/datafusion.git
commit 6adb5043638433864b0ce02b93cc416c89521b80
Author: Qi Zhu <zh...@gmail.com>
AuthorDate: Fri Aug 14 05:45:16 2026 +0000
bench: add parquet_row_filter_skip suite for the fully-matched RowFilter skip (#24328)
## Which issue does this PR close?
- Part of #23696 — a benchmark that mechanistically exercises the per-RG
fully-matched `RowFilter` skip.
## Rationale for this change
#23696 adds a per-row-group fully-matched `RowFilter` skip, but **none
of the existing benchmarks exercise it**:
- `sort_tpch` / `tpch` don't enable `pushdown_filters` by default, so
there is no `RowFilter` to skip;
- ClickBench's `URL LIKE …` / equality predicates rarely make a row
group's min/max fall entirely inside the satisfying range, so
fully-matched RGs are rare.
As raised in review (we should verify the optimization improves
something mechanistically, otherwise add a benchmark first), this adds a
suite that **necessarily** triggers the skip.
## What changes are included in this PR?
A new `sql_benchmarks/parquet_row_filter_skip/` suite:
- The **load SQL** enables `pushdown_filters` and `COPY`s a clustered
Parquet file — a fixed-width, zero-padded, monotonically increasing
string key (`skey`) so each row group holds a disjoint, sorted range —
plus 14 payload columns.
- The **query** applies a low-selectivity range filter (`skey >=
'0000100000'`, `skey` not projected). The first row group straddles the
threshold; every later RG is fully matched by statistics, so the per-row
`RowFilter` is skipped on the fully-matched run (and `skey` isn't
decoded there).
- `bench.sh` integration: `./bench.sh run parquet_row_filter_skip`, data
generated inline by the load SQL. Knobs: `PRED_ROWS` (row count),
`RG_SIZE` (parquet row-group size).
## Are these changes tested?
Smoke-tested locally via `cargo bench --bench sql -- --test`.
Local A/B (main vs #23696, 10M rows / 10 RGs / `skey >= '0000100000'`):
- **9 of 10 row groups fully matched** →
`row_filter_skipped_fully_matched=9`;
- **~18% faster** with the optimization (main ~0.143s → branch ~0.117s);
an int64-key variant is ~12%.
Once this lands, `run benchmark parquet_row_filter_skip` will compare
any PR (e.g. #23696) against `main` in CI.
## Are there any user-facing changes?
No — benchmark only.
---------
Co-authored-by: Claude <no...@anthropic.com>
---
benchmarks/bench.sh | 32 ++++++++++++++
benchmarks/sql_benchmarks/README.md | 4 +-
.../benchmarks/control/q10.benchmark | 7 +++
.../benchmarks/skip/q01.benchmark | 7 +++
.../benchmarks/skip/q02.benchmark | 7 +++
.../benchmarks/skip/q03.benchmark | 7 +++
.../parquet_row_filter_skip/init/cleanup.sql | 1 +
.../parquet_row_filter_skip/init/settings.sql | 9 ++++
.../parquet_row_filter_skip/load/clustered.sql | 44 +++++++++++++++++++
.../parquet_row_filter_skip/load/scrambled.sql | 43 +++++++++++++++++++
.../parquet_row_filter_skip.benchmark.template | 50 ++++++++++++++++++++++
.../parquet_row_filter_skip.suite | 26 +++++++++++
.../queries/control/q10.sql | 8 ++++
.../parquet_row_filter_skip/queries/skip/q01.sql | 7 +++
.../parquet_row_filter_skip/queries/skip/q02.sql | 6 +++
.../parquet_row_filter_skip/queries/skip/q03.sql | 8 ++++
.../parquet_row_filter_skip/scratch/.gitignore | 1 +
17 files changed, 266 insertions(+), 1 deletion(-)
diff --git a/benchmarks/bench.sh b/benchmarks/bench.sh
index 52b78c844a..df9b7f6c94 100755
--- a/benchmarks/bench.sh
+++ b/benchmarks/bench.sh
@@ -108,6 +108,10 @@ predicate_eval: Conjunctive (AND) filter-evaluation micro-benchmarks; ea
adaptive predicate-ordering system behaves across them (see https://github.com/apache/datafusion/issues/11262)
(subgroups via BENCH_SUBGROUP: costsel, cost, selectivity, cardinality, width, scale, neutral, correlation, drift)
(toggle a system under test with its native DATAFUSION_* env var; size data with PRED_ROWS, string width with PRED_FILL)
+parquet_row_filter_skip: Per-RG fully-matched RowFilter skip on Parquet (apache/datafusion#23696); clustered string key + low-selectivity
+ range filter + pushdown, so most row groups are fully matched and the per-row RowFilter is skipped on them
+ (subgroups via BENCH_SUBGROUP: skip = clustered key so the skip fires, control = scrambled key so it never fires)
+ (data generated inline by the suite's load SQL; knobs: PRED_ROWS, RG_SIZE)
# ClickBench Benchmarks
clickbench_1: ClickBench queries against a single parquet file
@@ -255,6 +259,10 @@ main() {
# Data is generated inline by the suite's load SQL.
echo "predicate_eval: no external data to generate"
;;
+ parquet_row_filter_skip)
+ # Data is generated inline by the suite's load SQL (COPY).
+ echo "parquet_row_filter_skip: no external data to generate"
+ ;;
tpcds)
data_tpcds
;;
@@ -475,6 +483,9 @@ main() {
predicate_eval)
run_predicate_eval
;;
+ parquet_row_filter_skip)
+ run_parquet_row_filter_skip
+ ;;
tpcds)
run_tpcds
;;
@@ -844,6 +855,27 @@ run_predicate_eval() {
bash -c "$SQL_CARGO_COMMAND"
}
+# Runs the parquet_row_filter_skip suite: the load SQL COPYs a Parquet file
+# inline (fixed-width string key, ordered so each row group holds a disjoint
+# sorted range) and the query applies a low-selectivity range filter, so all
+# but the first row group is fully matched by statistics and the per-row
+# RowFilter is skipped on them (apache/datafusion#23696). The control subgroup
+# scrambles the key so no row group is ever fully matched, measuring the
+# overhead of the check when it cannot fire. Data is inline, so no data step.
+# Knobs (string-substituted into the load SQL, not engine config):
+# BENCH_SUBGROUP run one subgroup (skip, control)
+# PRED_ROWS synthetic row count (default 10_000_000)
+# RG_SIZE parquet row-group size (default 1_000_000)
+run_parquet_row_filter_skip() {
+ echo "Running parquet_row_filter_skip benchmark (subgroup=${BENCH_SUBGROUP:-all}, rows=${PRED_ROWS:-10000000}, rg_size=${RG_SIZE:-1000000})..."
+ debug_run env BENCH_NAME=parquet_row_filter_skip \
+ ${BENCH_SUBGROUP:+BENCH_SUBGROUP="${BENCH_SUBGROUP}"} \
+ PRED_ROWS="${PRED_ROWS:-10000000}" \
+ RG_SIZE="${RG_SIZE:-1000000}" \
+ ${QUERY:+BENCH_QUERY="${QUERY}"} \
+ bash -c "$SQL_CARGO_COMMAND"
+}
+
# Runs the tpch in memory (needs tpch parquet data)
run_tpch_mem() {
SCALE_FACTOR=$1
diff --git a/benchmarks/sql_benchmarks/README.md b/benchmarks/sql_benchmarks/README.md
index dfb09e0a3a..1ce1fa488c 100644
--- a/benchmarks/sql_benchmarks/README.md
+++ b/benchmarks/sql_benchmarks/README.md
@@ -44,6 +44,7 @@ in the community:
| `tpch` | TPC‑H queries |
| `wide_schema` | Small-projection queries on a wide (1024-col, 256-file) synthetic dataset; runs `wide` + `narrow` subgroups for comparison |
| `predicate_eval` | Conjunctive (AND) filter-evaluation micro-benchmarks; each subgroup is a different predicate pattern, to test how an adaptive predicate-ordering system behaves across them ([#11262](https://github.com/apache/datafusion/issues/11262)). Subgroups (`--subgroup`): `costsel`, `cost`, `selectivity`, `cardinality`, `width`, `scale`, `neutral`, `correlation`, `drift`. Configure the system under test through its DataFusion settings. |
+| `parquet_row_filter_skip` | Micro-benchmark for the per-row-group fully-matched RowFilter skip on Parquet scans ([#23696](https://github.com/apache/datafusion/issues/23696)). Subgroups (`--subgroup`): `skip` (clustered key, most row groups fully matched by statistics so the per-row filter is skipped), `control` (scrambled key, no row group is ever fully matched). Size the data with `PRED_ROWS` and the row-group size with `RG_SIZE`. |
# Running Benchmarks
@@ -168,8 +169,9 @@ Some benchmarks use custom environment variables as outlined below:
| BENCH_SORTED | Used in the sort_tpch benchmark to indicate whether the lineitem table should be sorted. | false |
| SORTED_BY | Used in the clickbench_sorted benchmark to indicate the column to sort by. | `EventTime` |
| SORTED_ORDER | Used in the clickbench_sorted benchmark to indicate the sort order of the column. | `ASC` |
-| PRED_ROWS | Used in the predicate_eval benchmark to size the synthetic table (the `scale` subgroup overrides this per query). | `1000000` |
+| PRED_ROWS | Used in the predicate_eval benchmark to size the synthetic table (the `scale` subgroup overrides this per query), and in the parquet_row_filter_skip benchmark to size the generated Parquet datasets (default `10000000` there). | `1000000` |
| PRED_FILL | Used in the predicate_eval benchmark as the string-column width knob (filler chars per marker). | `30` |
+| RG_SIZE | Used in the parquet_row_filter_skip benchmark as the Parquet `max_row_group_size` for the generated datasets. | `1000000` |
## How it works
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/benchmarks/control/q10.benchmark b/benchmarks/sql_benchmarks/parquet_row_filter_skip/benchmarks/control/q10.benchmark
new file mode 100644
index 0000000000..4c64baa833
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/benchmarks/control/q10.benchmark
@@ -0,0 +1,7 @@
+subgroup control
+
+template sql_benchmarks/parquet_row_filter_skip/parquet_row_filter_skip.benchmark.template
+SUBGROUP=control
+QPAD=10
+DATASET=scrambled
+NAME=control_q10_scrambled_never_matched
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/benchmarks/skip/q01.benchmark b/benchmarks/sql_benchmarks/parquet_row_filter_skip/benchmarks/skip/q01.benchmark
new file mode 100644
index 0000000000..825605c6bb
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/benchmarks/skip/q01.benchmark
@@ -0,0 +1,7 @@
+subgroup skip
+
+template sql_benchmarks/parquet_row_filter_skip/parquet_row_filter_skip.benchmark.template
+SUBGROUP=skip
+QPAD=01
+DATASET=clustered
+NAME=skip_q01_string_range_low_selectivity
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/benchmarks/skip/q02.benchmark b/benchmarks/sql_benchmarks/parquet_row_filter_skip/benchmarks/skip/q02.benchmark
new file mode 100644
index 0000000000..30f1ff8b48
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/benchmarks/skip/q02.benchmark
@@ -0,0 +1,7 @@
+subgroup skip
+
+template sql_benchmarks/parquet_row_filter_skip/parquet_row_filter_skip.benchmark.template
+SUBGROUP=skip
+QPAD=02
+DATASET=clustered
+NAME=skip_q02_string_range_all_matched
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/benchmarks/skip/q03.benchmark b/benchmarks/sql_benchmarks/parquet_row_filter_skip/benchmarks/skip/q03.benchmark
new file mode 100644
index 0000000000..12015067f8
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/benchmarks/skip/q03.benchmark
@@ -0,0 +1,7 @@
+subgroup skip
+
+template sql_benchmarks/parquet_row_filter_skip/parquet_row_filter_skip.benchmark.template
+SUBGROUP=skip
+QPAD=03
+DATASET=clustered
+NAME=skip_q03_filter_column_projected
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/init/cleanup.sql b/benchmarks/sql_benchmarks/parquet_row_filter_skip/init/cleanup.sql
new file mode 100644
index 0000000000..48f076a9fa
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/init/cleanup.sql
@@ -0,0 +1 @@
+DROP TABLE IF EXISTS t;
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/init/settings.sql b/benchmarks/sql_benchmarks/parquet_row_filter_skip/init/settings.sql
new file mode 100644
index 0000000000..0f084fae93
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/init/settings.sql
@@ -0,0 +1,9 @@
+-- Session settings for the parquet_row_filter_skip suite. `init` runs after
+-- `load` and before the asserts and the benchmarked query, so these apply to
+-- the measured scan (the COPY in the load script does not need them).
+--
+-- information_schema is enabled so the template can assert that
+-- pushdown_filters actually took effect: without pushdown there is no
+-- RowFilter to skip and the suite would silently measure nothing.
+set datafusion.catalog.information_schema = true;
+set datafusion.execution.parquet.pushdown_filters = true;
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/load/clustered.sql b/benchmarks/sql_benchmarks/parquet_row_filter_skip/load/clustered.sql
new file mode 100644
index 0000000000..dd6bdfc42e
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/load/clustered.sql
@@ -0,0 +1,44 @@
+-- Clustered Parquet dataset for the fully-matched RowFilter-skip benchmark.
+--
+-- `skey` is a fixed-width, zero-padded, monotonically increasing string, so
+-- each row group holds a disjoint, sorted range of keys. With
+-- `pushdown_filters=true` (set in init/settings.sql), a low-selectivity range
+-- predicate (see the queries) leaves the first row group straddling and every
+-- later row group fully matched by min/max statistics, which is exactly what
+-- the per-RG RowFilter skip targets.
+--
+-- The ORDER BY is load-bearing: the whole benchmark rests on the file being
+-- written in key order so that row-group min/max ranges are disjoint. Without
+-- it the write order is at the mercy of the physical planner (a round-robin
+-- repartition + coalesce would silently interleave batches and destroy the
+-- clustering, leaving nothing to measure).
+--
+-- Knobs: PRED_ROWS (row count, must exceed the 100_000 predicate cutoff),
+-- RG_SIZE (parquet row-group size).
+COPY (
+ SELECT
+ lpad(CAST(value AS VARCHAR), 10, '0') AS skey,
+ (value * 7) % 1000000 AS p0,
+ (value * 13) % 1000000 AS p1,
+ (value * 17) % 1000000 AS p2,
+ (value * 19) % 1000000 AS p3,
+ (value * 23) % 1000000 AS p4,
+ (value * 29) % 1000000 AS p5,
+ (value * 31) % 1000000 AS p6,
+ (value * 37) % 1000000 AS p7,
+ (value * 41) % 1000000 AS p8,
+ (value * 43) % 1000000 AS p9,
+ (value * 47) % 1000000 AS p10,
+ (value * 53) % 1000000 AS p11,
+ (value * 59) % 1000000 AS p12,
+ (value * 61) % 1000000 AS p13
+ FROM generate_series(1, ${PRED_ROWS:-10000000})
+ ORDER BY value
+)
+TO 'sql_benchmarks/parquet_row_filter_skip/scratch/clustered.parquet'
+STORED AS PARQUET
+OPTIONS ('format.max_row_group_size' '${RG_SIZE:-1000000}');
+
+CREATE EXTERNAL TABLE t
+STORED AS PARQUET
+LOCATION 'sql_benchmarks/parquet_row_filter_skip/scratch/clustered.parquet';
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/load/scrambled.sql b/benchmarks/sql_benchmarks/parquet_row_filter_skip/load/scrambled.sql
new file mode 100644
index 0000000000..c8c0e0bf97
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/load/scrambled.sql
@@ -0,0 +1,43 @@
+-- Scrambled control dataset for the fully-matched RowFilter-skip benchmark.
+--
+-- Same schema, key domain, and payload as clustered.sql, but `skey` is a
+-- pseudo-random permutation of 1..PRED_ROWS (999983 is prime, so the map
+-- `v -> (v * 999983) % PRED_ROWS + 1` is a bijection whenever PRED_ROWS is
+-- not a multiple of 999983 -- true for the default and any power-of-ten
+-- size). Every row group therefore spans nearly the whole key range and no
+-- row group is ever fully matched by min/max statistics: the per-row
+-- RowFilter must run everywhere. Queries over this dataset measure the
+-- overhead of the fully-matched check when it can never fire.
+--
+-- Keeping the domain exactly 1..PRED_ROWS lets the template's row-count
+-- asserts hold for both datasets.
+--
+-- Knobs: PRED_ROWS (row count, must exceed the 100_000 predicate cutoff),
+-- RG_SIZE (parquet row-group size).
+COPY (
+ SELECT
+ lpad(CAST((value * 999983) % ${PRED_ROWS:-10000000} + 1 AS VARCHAR), 10, '0') AS skey,
+ (value * 7) % 1000000 AS p0,
+ (value * 13) % 1000000 AS p1,
+ (value * 17) % 1000000 AS p2,
+ (value * 19) % 1000000 AS p3,
+ (value * 23) % 1000000 AS p4,
+ (value * 29) % 1000000 AS p5,
+ (value * 31) % 1000000 AS p6,
+ (value * 37) % 1000000 AS p7,
+ (value * 41) % 1000000 AS p8,
+ (value * 43) % 1000000 AS p9,
+ (value * 47) % 1000000 AS p10,
+ (value * 53) % 1000000 AS p11,
+ (value * 59) % 1000000 AS p12,
+ (value * 61) % 1000000 AS p13
+ FROM generate_series(1, ${PRED_ROWS:-10000000})
+ ORDER BY value
+)
+TO 'sql_benchmarks/parquet_row_filter_skip/scratch/scrambled.parquet'
+STORED AS PARQUET
+OPTIONS ('format.max_row_group_size' '${RG_SIZE:-1000000}');
+
+CREATE EXTERNAL TABLE t
+STORED AS PARQUET
+LOCATION 'sql_benchmarks/parquet_row_filter_skip/scratch/scrambled.parquet';
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/parquet_row_filter_skip.benchmark.template b/benchmarks/sql_benchmarks/parquet_row_filter_skip/parquet_row_filter_skip.benchmark.template
new file mode 100644
index 0000000000..44c776f5f4
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/parquet_row_filter_skip.benchmark.template
@@ -0,0 +1,50 @@
+# Shared template for the parquet_row_filter_skip suite. Each qNN.benchmark sets
+# its `subgroup` directive and includes this template with parameters:
+# SUBGROUP subgroup name, also the query sub-directory (skip, control)
+# QPAD zero-padded query id / query file stem (e.g. 01)
+# DATASET load script stem under load/ (clustered, scrambled)
+# NAME criterion display name
+# Optional (consumed by the load script via ${...:-default}):
+# PRED_ROWS synthetic row count (default 10_000_000)
+# RG_SIZE parquet max row-group size (default 1_000_000)
+#
+# The load script COPYs the Parquet dataset and init/settings.sql enables
+# `pushdown_filters` so the fully-matched RowFilter-skip path is exercised.
+# The table is always named `t`, so the asserts and cleanup are uniform.
+#
+# The queries share a fixed cutoff key of '0000100000' (row 100_000), so
+# PRED_ROWS must stay above 100_000; the second assert fails loudly otherwise.
+# Both datasets use the key domain 1..PRED_ROWS, so the asserts hold for both.
+
+load sql_benchmarks/parquet_row_filter_skip/load/${DATASET}.sql
+
+init sql_benchmarks/parquet_row_filter_skip/init/settings.sql
+
+name ${NAME}
+group parquet_row_filter_skip
+
+# The dataset has exactly PRED_ROWS rows.
+assert I
+SELECT count(*) = ${PRED_ROWS:-10000000} FROM t;
+----
+true
+
+# The cutoff predicate matches exactly PRED_ROWS - 99_999 rows. This doubles
+# as a correctness canary for the optimization under test: a RowFilter skip
+# that fires on a not-fully-matched row group returns extra rows and trips
+# this assert.
+assert I
+SELECT count(*) = ${PRED_ROWS:-10000000} - 99999 FROM t WHERE skey >= '0000100000';
+----
+true
+
+# Guard against silent config no-ops: without pushdown_filters there is no
+# RowFilter at all and the suite measures nothing.
+assert I
+SELECT value = 'true' FROM information_schema.df_settings WHERE name = 'datafusion.execution.parquet.pushdown_filters';
+----
+true
+
+run sql_benchmarks/parquet_row_filter_skip/queries/${SUBGROUP}/q${QPAD}.sql
+
+cleanup sql_benchmarks/parquet_row_filter_skip/init/cleanup.sql
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/parquet_row_filter_skip.suite b/benchmarks/sql_benchmarks/parquet_row_filter_skip/parquet_row_filter_skip.suite
new file mode 100644
index 0000000000..1cf46dd2ef
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/parquet_row_filter_skip.suite
@@ -0,0 +1,26 @@
+description = "Micro-benchmark for the per-row-group fully-matched RowFilter skip on Parquet scans (apache/datafusion#23696). Builds a clustered Parquet file whose fixed-width string key is monotonically increasing, so each row group holds a disjoint sorted range. With filter pushdown enabled, a low-selectivity range predicate leaves the first row group straddling and the rest fully matched by statistics, so the decoder can skip the per-row RowFilter (and decoding the filter column) on t [...]
+
+query_pattern = "q{QUERY_ID_PADDED}.benchmark"
+
+[[options]]
+name = "rows"
+short = "r"
+env = "PRED_ROWS"
+default = "10000000"
+values = ["10000000", "..."]
+help = "Sets the number of rows in the generated Parquet datasets (must be greater than 100000, the fixed predicate cutoff)."
+
+[[options]]
+name = "rg-size"
+env = "RG_SIZE"
+default = "1000000"
+values = ["1000000", "..."]
+help = "Sets the Parquet max_row_group_size used when writing the generated datasets."
+
+[[examples]]
+command = "cargo run --release --bin benchmark_runner -- parquet_row_filter_skip"
+description = "Run every subgroup with the default 10M rows and 1M-row row groups."
+
+[[examples]]
+command = "cargo run --release --bin benchmark_runner -- parquet_row_filter_skip --subgroup skip --rg-size 100000"
+description = "Run only the skip subgroup with 100 row groups instead of 10."
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/queries/control/q10.sql b/benchmarks/sql_benchmarks/parquet_row_filter_skip/queries/control/q10.sql
new file mode 100644
index 0000000000..a5950e925b
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/queries/control/q10.sql
@@ -0,0 +1,8 @@
+-- Control: the same cutoff predicate as skip/q01 but over the scrambled
+-- dataset, where every row group spans nearly the whole key range, so no row
+-- group is ever fully matched and the per-row RowFilter runs everywhere.
+-- Measures the overhead of the fully-matched check when it cannot fire; the
+-- skip optimization should be performance-neutral here.
+SELECT sum(p0)+sum(p1)+sum(p2)+sum(p3)+sum(p4)+sum(p5)+sum(p6)+sum(p7)+sum(p8)+sum(p9)+sum(p10)+sum(p11)+sum(p12)+sum(p13) AS s
+FROM t
+WHERE skey >= '0000100000';
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/queries/skip/q01.sql b/benchmarks/sql_benchmarks/parquet_row_filter_skip/queries/skip/q01.sql
new file mode 100644
index 0000000000..33468c1922
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/queries/skip/q01.sql
@@ -0,0 +1,7 @@
+-- Low-selectivity range filter on the clustered string key: every row group
+-- except the first is fully matched by statistics, so the per-row RowFilter is
+-- skipped on those RGs. `skey` is not projected, so the skip also avoids
+-- decoding the filter column on the fully-matched run.
+SELECT sum(p0)+sum(p1)+sum(p2)+sum(p3)+sum(p4)+sum(p5)+sum(p6)+sum(p7)+sum(p8)+sum(p9)+sum(p10)+sum(p11)+sum(p12)+sum(p13) AS s
+FROM t
+WHERE skey >= '0000100000';
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/queries/skip/q02.sql b/benchmarks/sql_benchmarks/parquet_row_filter_skip/queries/skip/q02.sql
new file mode 100644
index 0000000000..a8eb2d6ca0
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/queries/skip/q02.sql
@@ -0,0 +1,6 @@
+-- Matches every row: the minimum key is '0000000001', so every row group --
+-- including the first -- is fully matched by statistics and the per-row
+-- RowFilter is skipped everywhere. The upper bound for the optimization.
+SELECT sum(p0)+sum(p1)+sum(p2)+sum(p3)+sum(p4)+sum(p5)+sum(p6)+sum(p7)+sum(p8)+sum(p9)+sum(p10)+sum(p11)+sum(p12)+sum(p13) AS s
+FROM t
+WHERE skey >= '0000000000';
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/queries/skip/q03.sql b/benchmarks/sql_benchmarks/parquet_row_filter_skip/queries/skip/q03.sql
new file mode 100644
index 0000000000..2879d222b2
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/queries/skip/q03.sql
@@ -0,0 +1,8 @@
+-- Same cutoff predicate as q01, but the filter column IS projected: the
+-- fully-matched skip still avoids evaluating the per-row filter, but can no
+-- longer avoid decoding `skey`. Measures the common case where the win is
+-- smaller than q01's best case.
+SELECT min(skey) AS first_key,
+ sum(p0)+sum(p1)+sum(p2)+sum(p3)+sum(p4)+sum(p5)+sum(p6)+sum(p7)+sum(p8)+sum(p9)+sum(p10)+sum(p11)+sum(p12)+sum(p13) AS s
+FROM t
+WHERE skey >= '0000100000';
diff --git a/benchmarks/sql_benchmarks/parquet_row_filter_skip/scratch/.gitignore b/benchmarks/sql_benchmarks/parquet_row_filter_skip/scratch/.gitignore
new file mode 100644
index 0000000000..4bed5da93f
--- /dev/null
+++ b/benchmarks/sql_benchmarks/parquet_row_filter_skip/scratch/.gitignore
@@ -0,0 +1 @@
+*.parquet
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org