You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "Xuanwo (via GitHub)" <gi...@apache.org> on 2026/07/23 06:27:21 UTC
[PR] bench: add ASOF join benchmarks [datafusion]
Xuanwo opened a new pull request, #23833:
URL: https://github.com/apache/datafusion/pull/23833
## Which issue does this PR close?
- Part of #318.
- Umbrella PR: #23738.
- Depends on #23830 (and transitively #23829 and #23828).
## Rationale for this change
This is the benchmark layer of the ASOF JOIN stack. It keeps performance
workloads separate from the execution, logical, and SQL implementation reviews.
GitHub shows the cumulative stack because fork branches cannot be selected as
upstream PR bases. The isolated diff for this layer is:
https://github.com/Xuanwo/datafusion/compare/xuanwo/asof-sql...xuanwo/asof-benchmarks
## What changes are included in this PR?
- Add a `dfbench asof` command with four end-to-end workloads:
ordered predecessor, grouped predecessor with optimizer-inserted
sort/repartition, wide payloads, and descending successor matching.
- Stream output batches during timing instead of retaining result buffers.
- Assert that each workload plans to `AsOfJoinExec`.
- Add a physical Criterion benchmark for Int64, wide UTF-8, and dictionary
payloads over pre-sorted inputs.
## Are these changes tested?
Yes:
- `cargo fmt --all`
- `cargo clippy --all-targets --all-features -- -D warnings`
- `cargo run -p datafusion-benchmarks --bin dfbench -- asof --iterations 1`
- `cargo bench -p datafusion-physical-plan --bench asof_join --features test_utils -- --quick`
- Extended workspace tests from the contributor guide
The end-to-end smoke run returned the expected 1,000,000, 1,000,000, 250,000,
and 500,000 rows. The Criterion smoke run completed all three payload cases.
These debug/quick timings validate the harness and are not presented as
performance claims.
## Are there any user-facing changes?
This adds ASOF workloads to the repository benchmark tooling. It does not change
query semantics or runtime behavior.
This PR should be reviewed and merged after #23830. After its dependencies
squash-merge, this branch will be refreshed from `main` without rewriting its
published history.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] bench: add ASOF join benchmarks [datafusion]
Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 commented on code in PR #23833:
URL: https://github.com/apache/datafusion/pull/23833#discussion_r3999009018
##########
benchmarks/src/asof.rs:
##########
@@ -0,0 +1,226 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use crate::util::{BenchmarkRun, CommonOpt, QueryResult};
+use clap::Args;
+use datafusion::physical_plan::execute_stream;
+use datafusion::{error::Result, prelude::SessionContext};
+use datafusion_common::instant::Instant;
+use datafusion_common::{DataFusionError, exec_datafusion_err, exec_err};
+use futures::StreamExt;
+
+/// Run end-to-end ASOF join benchmarks.
+///
+/// The cases cover broadcast-side size asymmetry, equality-key cardinality and
+/// skew, left-side parallelism, optimizer-inserted ordering, wide payload
+/// materialization, and descending successor matching.
+#[derive(Debug, Args, Clone)]
+#[command(verbatim_doc_comment)]
+pub struct RunOpt {
+ /// Query number (between 1 and 6). If not specified, runs all queries
+ #[arg(short, long)]
+ query: Option<usize>,
+
+ /// Common options
+ #[command(flatten)]
+ common: CommonOpt,
+
+ /// If present, write results json here
+ #[arg(short = 'o', long = "output")]
+ output_path: Option<std::path::PathBuf>,
+}
+
+const ASOF_QUERIES: &[&str] = &[
+ // Q1: small broadcast input and a large, equality-free probe input
+ r#"
+ WITH left_input AS (
+ SELECT value AS ts, value AS payload FROM range(1000000)
+ ),
+ right_input AS (
+ SELECT value AS ts, value AS payload FROM range(10000)
+ )
+ SELECT l.ts, l.payload, r.payload AS right_payload
+ FROM left_input l
+ ASOF JOIN right_input r MATCH_CONDITION (l.ts >= r.ts)
+ "#,
+ // Q2: grouped predecessor, optimizer partitions left and coalesces right
Review Comment:
Here we might want to only describe the workload, e.g., the left side is small and the right side is large, and leave out internal execution details like the left side getting repartitioned.
##########
benchmarks/src/asof.rs:
##########
@@ -0,0 +1,226 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use crate::util::{BenchmarkRun, CommonOpt, QueryResult};
+use clap::Args;
+use datafusion::physical_plan::execute_stream;
+use datafusion::{error::Result, prelude::SessionContext};
+use datafusion_common::instant::Instant;
+use datafusion_common::{DataFusionError, exec_datafusion_err, exec_err};
+use futures::StreamExt;
+
+/// Run end-to-end ASOF join benchmarks.
+///
+/// The cases cover broadcast-side size asymmetry, equality-key cardinality and
+/// skew, left-side parallelism, optimizer-inserted ordering, wide payload
+/// materialization, and descending successor matching.
+#[derive(Debug, Args, Clone)]
+#[command(verbatim_doc_comment)]
+pub struct RunOpt {
+ /// Query number (between 1 and 6). If not specified, runs all queries
+ #[arg(short, long)]
+ query: Option<usize>,
+
+ /// Common options
+ #[command(flatten)]
+ common: CommonOpt,
+
+ /// If present, write results json here
+ #[arg(short = 'o', long = "output")]
+ output_path: Option<std::path::PathBuf>,
+}
+
+const ASOF_QUERIES: &[&str] = &[
+ // Q1: small broadcast input and a large, equality-free probe input
+ r#"
+ WITH left_input AS (
+ SELECT value AS ts, value AS payload FROM range(1000000)
+ ),
+ right_input AS (
+ SELECT value AS ts, value AS payload FROM range(10000)
+ )
+ SELECT l.ts, l.payload, r.payload AS right_payload
+ FROM left_input l
+ ASOF JOIN right_input r MATCH_CONDITION (l.ts >= r.ts)
+ "#,
+ // Q2: grouped predecessor, optimizer partitions left and coalesces right
+ r#"
+ WITH left_input AS (
+ SELECT value % 10000 AS key,
+ value / 10000 + 1 AS ts,
+ value AS payload
+ FROM range(1000000)
+ ),
+ right_input AS (
+ SELECT value % 10000 AS key,
+ value / 10000 AS ts,
+ value AS payload
+ FROM range(1000000)
+ )
+ SELECT l.key, l.ts, l.payload, r.payload AS right_payload
+ FROM left_input l
+ ASOF JOIN right_input r MATCH_CONDITION (l.ts >= r.ts)
+ ON l.key = r.key
+ "#,
+ // Q3: grouped predecessor with a wide payload
+ r#"
+ WITH left_input AS (
+ SELECT value % 10000 AS key,
+ value / 10000 + 1 AS ts,
+ repeat('x', 256) AS payload
+ FROM range(250000)
+ ),
+ right_input AS (
+ SELECT value % 10000 AS key,
+ value / 10000 AS ts,
+ repeat('y', 256) AS payload
+ FROM range(250000)
+ )
+ SELECT l.key, l.ts, l.payload, r.payload AS right_payload
+ FROM left_input l
+ ASOF JOIN right_input r MATCH_CONDITION (l.ts >= r.ts)
+ ON l.key = r.key
+ "#,
+ // Q4: successor matching requires descending input order
Review Comment:
I don't get what's successor/predecessor after a quick read, maybe we can add more comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] bench: add ASOF join benchmarks [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on code in PR #23833:
URL: https://github.com/apache/datafusion/pull/23833#discussion_r3998954048
##########
datafusion/physical-plan/benches/asof_join.rs:
##########
@@ -0,0 +1,219 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Criterion benchmarks for the pre-sorted broadcast ASOF join kernel.
+
+use std::sync::Arc;
+
+use arrow::array::{
+ ArrayRef, Int64Array, RecordBatch, StringArray, StringDictionaryBuilder,
+};
+use arrow::datatypes::{DataType, Field, Int32Type, Schema, SchemaRef};
+use criterion::{BenchmarkId, Criterion, criterion_group, criterion_main};
+use datafusion_execution::{TaskContext, config::SessionConfig};
+use datafusion_expr::Operator;
+use datafusion_physical_expr::expressions::col;
+use datafusion_physical_plan::joins::{AsOfJoinExec, AsOfMatchExpr, utils::JoinOn};
+use datafusion_physical_plan::test::TestMemoryExec;
+use datafusion_physical_plan::{ExecutionPlan, collect};
+use tokio::runtime::Runtime;
+
+#[derive(Clone, Copy)]
+enum Payload {
+ Int64,
+ WideUtf8,
+ Dictionary,
+}
+
+impl Payload {
+ fn name(self) -> &'static str {
+ match self {
+ Self::Int64 => "int64",
+ Self::WideUtf8 => "wide_utf8",
+ Self::Dictionary => "dictionary",
+ }
+ }
+
+ fn data_type(self) -> DataType {
+ match self {
+ Self::Int64 => DataType::Int64,
+ Self::WideUtf8 => DataType::Utf8,
+ Self::Dictionary => {
+ DataType::Dictionary(Box::new(DataType::Int32), Box::new(DataType::Utf8))
+ }
+ }
+ }
+
+ fn array(self, rows: &[(i64, i64, usize)]) -> ArrayRef {
+ match self {
+ Self::Int64 => Arc::new(Int64Array::from_iter_values(
+ rows.iter().map(|(_, _, row)| *row as i64),
+ )),
+ Self::WideUtf8 => Arc::new(StringArray::from_iter_values(
+ rows.iter()
+ .map(|(_, _, row)| format!("row_{row:08}_{}", "x".repeat(244))),
+ )),
+ Self::Dictionary => {
+ let mut builder = StringDictionaryBuilder::<Int32Type>::new();
+ for (_, _, row) in rows {
+ builder.append_value(format!("category_{}", row % 64));
+ }
+ Arc::new(builder.finish())
+ }
+ }
+ }
+}
+
+fn schema(payload: Payload) -> SchemaRef {
+ Arc::new(Schema::new(vec![
+ Field::new("key", DataType::Int64, false),
+ Field::new("ts", DataType::Int64, false),
+ Field::new("payload", payload.data_type(), false),
+ ]))
+}
+
+fn build_sorted_batches(
+ num_rows: usize,
+ num_groups: usize,
+ time_offset: i64,
+ payload: Payload,
+ schema: &SchemaRef,
+) -> Vec<RecordBatch> {
+ let mut rows = (0..num_rows)
+ .map(|row| {
+ (
+ (row % num_groups) as i64,
+ (row / num_groups) as i64 + time_offset,
+ row,
+ )
+ })
+ .collect::<Vec<_>>();
+ rows.sort_unstable_by_key(|(key, ts, _)| (*key, *ts));
+
+ let batch = RecordBatch::try_new(
+ Arc::clone(schema),
+ vec![
+ Arc::new(Int64Array::from_iter_values(
+ rows.iter().map(|(key, _, _)| *key),
+ )),
+ Arc::new(Int64Array::from_iter_values(
+ rows.iter().map(|(_, ts, _)| *ts),
+ )),
+ payload.array(&rows),
+ ],
+ )
+ .unwrap();
+
+ let mut batches = Vec::new();
+ let mut offset = 0;
+ while offset < batch.num_rows() {
+ let len = (batch.num_rows() - offset).min(8192);
+ batches.push(batch.slice(offset, len));
+ offset += len;
+ }
+ batches
+}
+
+fn partition_batches(
+ batches: &[RecordBatch],
+ partition_count: usize,
+) -> Vec<Vec<RecordBatch>> {
+ let mut partitions = vec![Vec::new(); partition_count];
+ for (index, batch) in batches.iter().enumerate() {
+ partitions[index % partition_count].push(batch.clone());
+ }
+ partitions
+}
+
+fn make_exec(
+ partitions: &[Vec<RecordBatch>],
+ schema: &SchemaRef,
+) -> Arc<dyn ExecutionPlan> {
+ TestMemoryExec::try_new_exec(partitions, Arc::clone(schema), None).unwrap()
+}
+
+fn do_join(
+ left: Arc<dyn ExecutionPlan>,
+ right: Arc<dyn ExecutionPlan>,
+ rt: &Runtime,
+) -> usize {
+ let on: JoinOn = vec![(
+ col("key", &left.schema()).unwrap(),
+ col("key", &right.schema()).unwrap(),
+ )];
+ let left_match = col("ts", &left.schema()).unwrap();
+ let right_match = col("ts", &right.schema()).unwrap();
+ let join = AsOfJoinExec::try_new(
+ left,
+ right,
+ on,
+ AsOfMatchExpr::new(left_match, Operator::GtEq, right_match),
Review Comment:
- What's wrong: do_join always uses an equality key with Operator::GtEq, and both sides always have 100k rows.
- Why it matters: these kernel paths get no benchmark that excludes sort cost:
- the no-equality-key path (group_sort_options.is_empty())
- the descending </<= path
- the "large right, small left" case, where each left partition may scan the whole broadcast input
- Fix: add a no-key case and a LtEq case with reversed pre-sorted data. Note that build_sorted_batches hardcodes ascending order.
##########
benchmarks/src/asof.rs:
##########
@@ -0,0 +1,226 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use crate::util::{BenchmarkRun, CommonOpt, QueryResult};
+use clap::Args;
+use datafusion::physical_plan::execute_stream;
+use datafusion::{error::Result, prelude::SessionContext};
+use datafusion_common::instant::Instant;
+use datafusion_common::{DataFusionError, exec_datafusion_err, exec_err};
+use futures::StreamExt;
+
+/// Run end-to-end ASOF join benchmarks.
+///
+/// The cases cover broadcast-side size asymmetry, equality-key cardinality and
+/// skew, left-side parallelism, optimizer-inserted ordering, wide payload
+/// materialization, and descending successor matching.
+#[derive(Debug, Args, Clone)]
+#[command(verbatim_doc_comment)]
+pub struct RunOpt {
+ /// Query number (between 1 and 6). If not specified, runs all queries
+ #[arg(short, long)]
+ query: Option<usize>,
+
+ /// Common options
+ #[command(flatten)]
+ common: CommonOpt,
+
+ /// If present, write results json here
+ #[arg(short = 'o', long = "output")]
+ output_path: Option<std::path::PathBuf>,
+}
+
+const ASOF_QUERIES: &[&str] = &[
+ // Q1: small broadcast input and a large, equality-free probe input
+ r#"
+ WITH left_input AS (
+ SELECT value AS ts, value AS payload FROM range(1000000)
+ ),
+ right_input AS (
+ SELECT value AS ts, value AS payload FROM range(10000)
+ )
+ SELECT l.ts, l.payload, r.payload AS right_payload
+ FROM left_input l
+ ASOF JOIN right_input r MATCH_CONDITION (l.ts >= r.ts)
+ "#,
+ // Q2: grouped predecessor, optimizer partitions left and coalesces right
Review Comment:
- What's wrong: EXPLAIN at target_partitions = 4 shows the extra work around the join:
- Left side: round-robin repartition → projection → SortExec(key, ts) in each partition.
- Right side: repartition → SortExec → SortPreservingMergeExec.
- Result: Q2 and Q6 sort 2M rows in total on every iteration. Q4 also sorts both sides, but it says so. Only Q1 and Q5 skip the sort, because a single-input round-robin repartition keeps order within each partition.
- Why it matters: a regression in the join kernel is diluted by sort time in the three keyed workloads. The Criterion benchmark does isolate the kernel, but it only covers one shape (N3).
- Fix: at minimum, say in the Q2/Q3/Q6 comments that sort cost is included. Optionally add one keyed workload whose input already arrives in (key, ts) order, for example from a sorted MemTable or parquet file with declared sort order, so the end-to-end suite has a keyed case without the sort.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] bench: add ASOF join benchmarks [datafusion]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #23833:
URL: https://github.com/apache/datafusion/pull/23833#issuecomment-5652878525
Thanks @jayzhan211 and @2010YOUY01 for the thoughtful reviews! This is much cleaner now 🙌
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] bench: add ASOF join benchmarks [datafusion]
Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 merged PR #23833:
URL: https://github.com/apache/datafusion/pull/23833
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] bench: add ASOF join benchmarks [datafusion]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #23833:
URL: https://github.com/apache/datafusion/pull/23833#issuecomment-5651273365
There are many optimizations to do, but I think it good to have a benchmark in first. This PR will be the last one of this stack, all other changes will be non-blocking, optional follow ups to do.
Cc @2010YOUY01 and @jayzhan211, let's go! Time to close the issue created at 2012 🤓
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] bench: add ASOF join benchmarks [datafusion]
Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 commented on code in PR #23833:
URL: https://github.com/apache/datafusion/pull/23833#discussion_r4001905056
##########
benchmarks/sql_benchmarks/asof_join/scratch/.gitignore:
##########
@@ -0,0 +1 @@
+*.parquet
Review Comment:
Since it's leaving benchmark data in the directory, we could put it in the `benchmarks/data/` folder to keep large benchmark files together to make it easier to maintain.
To do that, data should be generated with
```
# Generate test data in `benchmarks/data/`
./bench.sh data asof_join
# later benchmark queries should only register the generated parquet file, and won't generate them again
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] bench: add ASOF join benchmarks [datafusion]
Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 commented on code in PR #23833:
URL: https://github.com/apache/datafusion/pull/23833#discussion_r3999040802
##########
datafusion/physical-plan/benches/asof_join.rs:
##########
@@ -0,0 +1,219 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Criterion benchmarks for the pre-sorted broadcast ASOF join kernel.
Review Comment:
I think micro bench on physical plan is not necessary, how about only keep SQL bench for simplicity? (Such existing order workload should also doable with SQL)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] bench: add ASOF join benchmarks [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #23833:
URL: https://github.com/apache/datafusion/pull/23833#issuecomment-5055461110
<!-- semver-check-comment -->
Thank you for opening this pull request!
Reviewer note: [cargo-semver-checks](https://github.com/obi1kenobi/cargo-semver-checks) reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).
<details>
<summary>Details</summary>
```
Cloning apache/main
Building datafusion v54.1.0 (current)
Built [ 89.965s] (current)
Parsing datafusion v54.1.0 (current)
Parsed [ 0.029s] (current)
Building datafusion v54.1.0 (baseline)
Built [ 87.027s] (baseline)
Parsing datafusion v54.1.0 (baseline)
Parsed [ 0.029s] (baseline)
Checking datafusion v54.1.0 -> v54.1.0 (no change; assume patch)
Checked [ 0.776s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 180.060s] datafusion
Building datafusion-expr v54.1.0 (current)
Built [ 21.432s] (current)
Parsing datafusion-expr v54.1.0 (current)
Parsed [ 0.063s] (current)
Building datafusion-expr v54.1.0 (baseline)
Built [ 21.454s] (baseline)
Parsing datafusion-expr v54.1.0 (baseline)
Parsed [ 0.064s] (baseline)
Checking datafusion-expr v54.1.0 -> v54.1.0 (no change; assume patch)
Checked [ 1.551s] 223 checks: 221 pass, 1 fail, 1 warn, 30 skip
--- failure enum_variant_added: enum variant added on exhaustive enum ---
Description:
A publicly-visible enum without #[non_exhaustive] has a new variant.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#enum-variant-new
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/enum_variant_added.ron
Failed in:
variant LogicalPlan:AsOfJoin in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:245
variant LogicalPlan:AsOfJoin in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:245
--- warning partial_ord_enum_variants_reordered: enum variants reordered in #[derive(PartialOrd)] enum ---
Description:
A public enum that derives PartialOrd had its variants reordered. #[derive(PartialOrd)] uses the enum variant order to set the enum's ordering behavior, so this change may break downstream code that relies on the previous order.
ref: https://doc.rust-lang.org/std/cmp/trait.PartialOrd.html#derivable
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/partial_ord_enum_variants_reordered.ron
Failed in:
LogicalPlan::Repartition moved from position 7 to 8, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:249
LogicalPlan::Union moved from position 8 to 9, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:253
LogicalPlan::TableScan moved from position 9 to 10, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:256
LogicalPlan::EmptyRelation moved from position 10 to 11, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:260
LogicalPlan::Subquery moved from position 11 to 12, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:263
LogicalPlan::SubqueryAlias moved from position 12 to 13, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:265
LogicalPlan::Limit moved from position 13 to 14, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:267
LogicalPlan::Statement moved from position 14 to 15, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:269
LogicalPlan::Values moved from position 15 to 16, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:274
LogicalPlan::Explain moved from position 16 to 17, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:277
LogicalPlan::Analyze moved from position 17 to 18, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:281
LogicalPlan::Extension moved from position 18 to 19, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:284
LogicalPlan::Distinct moved from position 19 to 20, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:287
LogicalPlan::Dml moved from position 20 to 21, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:289
LogicalPlan::Ddl moved from position 21 to 22, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:291
LogicalPlan::Copy moved from position 22 to 23, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:293
LogicalPlan::DescribeTable moved from position 23 to 24, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:296
LogicalPlan::Unnest moved from position 24 to 25, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:299
LogicalPlan::RecursiveQuery moved from position 25 to 26, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:301
LogicalPlan::Repartition moved from position 7 to 8, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:249
LogicalPlan::Union moved from position 8 to 9, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:253
LogicalPlan::TableScan moved from position 9 to 10, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:256
LogicalPlan::EmptyRelation moved from position 10 to 11, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:260
LogicalPlan::Subquery moved from position 11 to 12, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:263
LogicalPlan::SubqueryAlias moved from position 12 to 13, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:265
LogicalPlan::Limit moved from position 13 to 14, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:267
LogicalPlan::Statement moved from position 14 to 15, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:269
LogicalPlan::Values moved from position 15 to 16, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:274
LogicalPlan::Explain moved from position 16 to 17, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:277
LogicalPlan::Analyze moved from position 17 to 18, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:281
LogicalPlan::Extension moved from position 18 to 19, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:284
LogicalPlan::Distinct moved from position 19 to 20, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:287
LogicalPlan::Dml moved from position 20 to 21, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:289
LogicalPlan::Ddl moved from position 21 to 22, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:291
LogicalPlan::Copy moved from position 22 to 23, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:293
LogicalPlan::DescribeTable moved from position 23 to 24, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:296
LogicalPlan::Unnest moved from position 24 to 25, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:299
LogicalPlan::RecursiveQuery moved from position 25 to 26, in /home/runner/work/datafusion/datafusion/datafusion/expr/src/logical_plan/plan.rs:301
Summary semver requires new major version: 1 major and 0 minor checks failed
Warning produced 1 major and 0 minor level warnings
Finished [ 46.775s] datafusion-expr
Building datafusion-optimizer v54.1.0 (current)
Built [ 22.459s] (current)
Parsing datafusion-optimizer v54.1.0 (current)
Parsed [ 0.026s] (current)
Building datafusion-optimizer v54.1.0 (baseline)
Built [ 22.754s] (baseline)
Parsing datafusion-optimizer v54.1.0 (baseline)
Parsed [ 0.027s] (baseline)
Checking datafusion-optimizer v54.1.0 -> v54.1.0 (no change; assume patch)
Checked [ 0.206s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 46.422s] datafusion-optimizer
Building datafusion-physical-plan v54.1.0 (current)
Built [ 30.092s] (current)
Parsing datafusion-physical-plan v54.1.0 (current)
Parsed [ 0.120s] (current)
Building datafusion-physical-plan v54.1.0 (baseline)
Built [ 30.338s] (baseline)
Parsing datafusion-physical-plan v54.1.0 (baseline)
Parsed [ 0.123s] (baseline)
Checking datafusion-physical-plan v54.1.0 -> v54.1.0 (no change; assume patch)
Checked [ 0.827s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 63.105s] datafusion-physical-plan
Building datafusion-proto v54.1.0 (current)
Built [ 49.782s] (current)
Parsing datafusion-proto v54.1.0 (current)
Parsed [ 0.016s] (current)
Building datafusion-proto v54.1.0 (baseline)
Built [ 49.291s] (baseline)
Parsing datafusion-proto v54.1.0 (baseline)
Parsed [ 0.017s] (baseline)
Checking datafusion-proto v54.1.0 -> v54.1.0 (no change; assume patch)
Checked [ 0.269s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 100.751s] datafusion-proto
Building datafusion-sql v54.1.0 (current)
Built [ 33.915s] (current)
Parsing datafusion-sql v54.1.0 (current)
Parsed [ 0.027s] (current)
Building datafusion-sql v54.1.0 (baseline)
Built [ 33.596s] (baseline)
Parsing datafusion-sql v54.1.0 (baseline)
Parsed [ 0.027s] (baseline)
Checking datafusion-sql v54.1.0 -> v54.1.0 (no change; assume patch)
Checked [ 0.289s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 69.423s] datafusion-sql
Building datafusion-sqllogictest v54.1.0 (current)
Built [ 153.758s] (current)
Parsing datafusion-sqllogictest v54.1.0 (current)
Parsed [ 0.019s] (current)
Building datafusion-sqllogictest v54.1.0 (baseline)
Built [ 150.849s] (baseline)
Parsing datafusion-sqllogictest v54.1.0 (baseline)
Parsed [ 0.110s] (baseline)
Checking datafusion-sqllogictest v54.1.0 -> v54.1.0 (no change; assume patch)
Checked [ 0.104s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 307.780s] datafusion-sqllogictest
Building datafusion-substrait v54.1.0 (current)
Built [ 282.070s] (current)
Parsing datafusion-substrait v54.1.0 (current)
Parsed [ 0.015s] (current)
Building datafusion-substrait v54.1.0 (baseline)
Built [ 284.352s] (baseline)
Parsing datafusion-substrait v54.1.0 (baseline)
Parsed [ 0.016s] (baseline)
Checking datafusion-substrait v54.1.0 -> v54.1.0 (no change; assume patch)
Checked [ 0.268s] 223 checks: 223 pass, 30 skip
Summary no semver update required
Finished [ 568.744s] datafusion-substrait
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] bench: add ASOF join benchmarks [datafusion]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #23833:
URL: https://github.com/apache/datafusion/pull/23833#issuecomment-5673678898
Cc @2010YOUY01 for another queue 🤩
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] bench: add ASOF join benchmarks [datafusion]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #23833:
URL: https://github.com/apache/datafusion/pull/23833#issuecomment-5055403647
## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/23833?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `76.51045%` with `416 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 80.73%. Comparing base ([`47bd094`](https://app.codecov.io/gh/apache/datafusion/commit/47bd094c7fd2fc22cb2b94ee58458b87e365a56f?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`aff4ce3`](https://app.codecov.io/gh/apache/datafusion/commit/aff4ce3d4e1ca6e4a5331f39e7c84068046e00f2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/datafusion/pull/23833?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [datafusion/physical-plan/src/joins/asof\_join.rs](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&filepath=datafusion%2Fphysical-plan%2Fsrc%2Fjoins%2Fasof_join.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9waHlzaWNhbC1wbGFuL3NyYy9qb2lucy9hc29mX2pvaW4ucnM=) | 87.28% | [73 Missing and 55 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/expr/src/logical\_plan/plan.rs](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&filepath=datafusion%2Fexpr%2Fsrc%2Flogical_plan%2Fplan.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9leHByL3NyYy9sb2dpY2FsX3BsYW4vcGxhbi5ycw==) | 55.08% | [63 Missing and 12 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/sql/src/unparser/plan.rs](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&filepath=datafusion%2Fsql%2Fsrc%2Funparser%2Fplan.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9zcWwvc3JjL3VucGFyc2VyL3BsYW4ucnM=) | 66.12% | [49 Missing and 14 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [benchmarks/src/asof.rs](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&filepath=benchmarks%2Fsrc%2Fasof.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YmVuY2htYXJrcy9zcmMvYXNvZi5ycw==) | 0.00% | [62 Missing :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/optimizer/src/push\_down\_filter.rs](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&filepath=datafusion%2Foptimizer%2Fsrc%2Fpush_down_filter.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9vcHRpbWl6ZXIvc3JjL3B1c2hfZG93bl9maWx0ZXIucnM=) | 0.00% | [24 Missing :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/expr/src/logical\_plan/builder.rs](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&filepath=datafusion%2Fexpr%2Fsrc%2Flogical_plan%2Fbuilder.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9leHByL3NyYy9sb2dpY2FsX3BsYW4vYnVpbGRlci5ycw==) | 81.70% | [3 Missing and 12 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/sql/src/relation/join.rs](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&filepath=datafusion%2Fsql%2Fsrc%2Frelation%2Fjoin.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9zcWwvc3JjL3JlbGF0aW9uL2pvaW4ucnM=) | 83.33% | [7 Missing and 8 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/core/src/physical\_planner.rs](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&filepath=datafusion%2Fcore%2Fsrc%2Fphysical_planner.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9jb3JlL3NyYy9waHlzaWNhbF9wbGFubmVyLnJz) | 81.81% | [6 Missing and 4 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/expr/src/logical\_plan/display.rs](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&filepath=datafusion%2Fexpr%2Fsrc%2Flogical_plan%2Fdisplay.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9leHByL3NyYy9sb2dpY2FsX3BsYW4vZGlzcGxheS5ycw==) | 0.00% | [10 Missing :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [datafusion/expr/src/logical\_plan/tree\_node.rs](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&filepath=datafusion%2Fexpr%2Fsrc%2Flogical_plan%2Ftree_node.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-ZGF0YWZ1c2lvbi9leHByL3NyYy9sb2dpY2FsX3BsYW4vdHJlZV9ub2RlLnJz) | 93.02% | [0 Missing and 3 partials :warning: ](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| ... and [5 more](https://app.codecov.io/gh/apache/datafusion/pull/23833?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #23833 +/- ##
==========================================
- Coverage 80.75% 80.73% -0.03%
==========================================
Files 1089 1091 +2
Lines 368948 370706 +1758
Branches 368948 370706 +1758
==========================================
+ Hits 297952 299295 +1343
- Misses 53237 53539 +302
- Partials 17759 17872 +113
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/23833?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] bench: add ASOF join benchmarks [datafusion]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #23833:
URL: https://github.com/apache/datafusion/pull/23833#discussion_r3999440157
##########
benchmarks/src/asof.rs:
##########
@@ -0,0 +1,226 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use crate::util::{BenchmarkRun, CommonOpt, QueryResult};
+use clap::Args;
+use datafusion::physical_plan::execute_stream;
+use datafusion::{error::Result, prelude::SessionContext};
+use datafusion_common::instant::Instant;
+use datafusion_common::{DataFusionError, exec_datafusion_err, exec_err};
+use futures::StreamExt;
+
+/// Run end-to-end ASOF join benchmarks.
+///
+/// The cases cover broadcast-side size asymmetry, equality-key cardinality and
+/// skew, left-side parallelism, optimizer-inserted ordering, wide payload
+/// materialization, and descending successor matching.
+#[derive(Debug, Args, Clone)]
+#[command(verbatim_doc_comment)]
+pub struct RunOpt {
+ /// Query number (between 1 and 6). If not specified, runs all queries
+ #[arg(short, long)]
+ query: Option<usize>,
+
+ /// Common options
+ #[command(flatten)]
+ common: CommonOpt,
+
+ /// If present, write results json here
+ #[arg(short = 'o', long = "output")]
+ output_path: Option<std::path::PathBuf>,
+}
+
+const ASOF_QUERIES: &[&str] = &[
+ // Q1: small broadcast input and a large, equality-free probe input
+ r#"
+ WITH left_input AS (
+ SELECT value AS ts, value AS payload FROM range(1000000)
+ ),
+ right_input AS (
+ SELECT value AS ts, value AS payload FROM range(10000)
+ )
+ SELECT l.ts, l.payload, r.payload AS right_payload
+ FROM left_input l
+ ASOF JOIN right_input r MATCH_CONDITION (l.ts >= r.ts)
+ "#,
+ // Q2: grouped predecessor, optimizer partitions left and coalesces right
Review Comment:
Good catch. Q07 now covers the pre-sorted keyed case. Thanks!
##########
datafusion/physical-plan/benches/asof_join.rs:
##########
@@ -0,0 +1,219 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Criterion benchmarks for the pre-sorted broadcast ASOF join kernel.
+
+use std::sync::Arc;
+
+use arrow::array::{
+ ArrayRef, Int64Array, RecordBatch, StringArray, StringDictionaryBuilder,
+};
+use arrow::datatypes::{DataType, Field, Int32Type, Schema, SchemaRef};
+use criterion::{BenchmarkId, Criterion, criterion_group, criterion_main};
+use datafusion_execution::{TaskContext, config::SessionConfig};
+use datafusion_expr::Operator;
+use datafusion_physical_expr::expressions::col;
+use datafusion_physical_plan::joins::{AsOfJoinExec, AsOfMatchExpr, utils::JoinOn};
+use datafusion_physical_plan::test::TestMemoryExec;
+use datafusion_physical_plan::{ExecutionPlan, collect};
+use tokio::runtime::Runtime;
+
+#[derive(Clone, Copy)]
+enum Payload {
+ Int64,
+ WideUtf8,
+ Dictionary,
+}
+
+impl Payload {
+ fn name(self) -> &'static str {
+ match self {
+ Self::Int64 => "int64",
+ Self::WideUtf8 => "wide_utf8",
+ Self::Dictionary => "dictionary",
+ }
+ }
+
+ fn data_type(self) -> DataType {
+ match self {
+ Self::Int64 => DataType::Int64,
+ Self::WideUtf8 => DataType::Utf8,
+ Self::Dictionary => {
+ DataType::Dictionary(Box::new(DataType::Int32), Box::new(DataType::Utf8))
+ }
+ }
+ }
+
+ fn array(self, rows: &[(i64, i64, usize)]) -> ArrayRef {
+ match self {
+ Self::Int64 => Arc::new(Int64Array::from_iter_values(
+ rows.iter().map(|(_, _, row)| *row as i64),
+ )),
+ Self::WideUtf8 => Arc::new(StringArray::from_iter_values(
+ rows.iter()
+ .map(|(_, _, row)| format!("row_{row:08}_{}", "x".repeat(244))),
+ )),
+ Self::Dictionary => {
+ let mut builder = StringDictionaryBuilder::<Int32Type>::new();
+ for (_, _, row) in rows {
+ builder.append_value(format!("category_{}", row % 64));
+ }
+ Arc::new(builder.finish())
+ }
+ }
+ }
+}
+
+fn schema(payload: Payload) -> SchemaRef {
+ Arc::new(Schema::new(vec![
+ Field::new("key", DataType::Int64, false),
+ Field::new("ts", DataType::Int64, false),
+ Field::new("payload", payload.data_type(), false),
+ ]))
+}
+
+fn build_sorted_batches(
+ num_rows: usize,
+ num_groups: usize,
+ time_offset: i64,
+ payload: Payload,
+ schema: &SchemaRef,
+) -> Vec<RecordBatch> {
+ let mut rows = (0..num_rows)
+ .map(|row| {
+ (
+ (row % num_groups) as i64,
+ (row / num_groups) as i64 + time_offset,
+ row,
+ )
+ })
+ .collect::<Vec<_>>();
+ rows.sort_unstable_by_key(|(key, ts, _)| (*key, *ts));
+
+ let batch = RecordBatch::try_new(
+ Arc::clone(schema),
+ vec![
+ Arc::new(Int64Array::from_iter_values(
+ rows.iter().map(|(key, _, _)| *key),
+ )),
+ Arc::new(Int64Array::from_iter_values(
+ rows.iter().map(|(_, ts, _)| *ts),
+ )),
+ payload.array(&rows),
+ ],
+ )
+ .unwrap();
+
+ let mut batches = Vec::new();
+ let mut offset = 0;
+ while offset < batch.num_rows() {
+ let len = (batch.num_rows() - offset).min(8192);
+ batches.push(batch.slice(offset, len));
+ offset += len;
+ }
+ batches
+}
+
+fn partition_batches(
+ batches: &[RecordBatch],
+ partition_count: usize,
+) -> Vec<Vec<RecordBatch>> {
+ let mut partitions = vec![Vec::new(); partition_count];
+ for (index, batch) in batches.iter().enumerate() {
+ partitions[index % partition_count].push(batch.clone());
+ }
+ partitions
+}
+
+fn make_exec(
+ partitions: &[Vec<RecordBatch>],
+ schema: &SchemaRef,
+) -> Arc<dyn ExecutionPlan> {
+ TestMemoryExec::try_new_exec(partitions, Arc::clone(schema), None).unwrap()
+}
+
+fn do_join(
+ left: Arc<dyn ExecutionPlan>,
+ right: Arc<dyn ExecutionPlan>,
+ rt: &Runtime,
+) -> usize {
+ let on: JoinOn = vec![(
+ col("key", &left.schema()).unwrap(),
+ col("key", &right.schema()).unwrap(),
+ )];
+ let left_match = col("ts", &left.schema()).unwrap();
+ let right_match = col("ts", &right.schema()).unwrap();
+ let join = AsOfJoinExec::try_new(
+ left,
+ right,
+ on,
+ AsOfMatchExpr::new(left_match, Operator::GtEq, right_match),
Review Comment:
Covered in the SQL suite now. Thanks!
##########
benchmarks/src/asof.rs:
##########
@@ -0,0 +1,226 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use crate::util::{BenchmarkRun, CommonOpt, QueryResult};
+use clap::Args;
+use datafusion::physical_plan::execute_stream;
+use datafusion::{error::Result, prelude::SessionContext};
+use datafusion_common::instant::Instant;
+use datafusion_common::{DataFusionError, exec_datafusion_err, exec_err};
+use futures::StreamExt;
+
+/// Run end-to-end ASOF join benchmarks.
+///
+/// The cases cover broadcast-side size asymmetry, equality-key cardinality and
+/// skew, left-side parallelism, optimizer-inserted ordering, wide payload
+/// materialization, and descending successor matching.
+#[derive(Debug, Args, Clone)]
+#[command(verbatim_doc_comment)]
+pub struct RunOpt {
+ /// Query number (between 1 and 6). If not specified, runs all queries
+ #[arg(short, long)]
+ query: Option<usize>,
+
+ /// Common options
+ #[command(flatten)]
+ common: CommonOpt,
+
+ /// If present, write results json here
+ #[arg(short = 'o', long = "output")]
+ output_path: Option<std::path::PathBuf>,
+}
+
+const ASOF_QUERIES: &[&str] = &[
+ // Q1: small broadcast input and a large, equality-free probe input
+ r#"
+ WITH left_input AS (
+ SELECT value AS ts, value AS payload FROM range(1000000)
+ ),
+ right_input AS (
+ SELECT value AS ts, value AS payload FROM range(10000)
+ )
+ SELECT l.ts, l.payload, r.payload AS right_payload
+ FROM left_input l
+ ASOF JOIN right_input r MATCH_CONDITION (l.ts >= r.ts)
+ "#,
+ // Q2: grouped predecessor, optimizer partitions left and coalesces right
Review Comment:
Agreed, updated!
##########
benchmarks/src/asof.rs:
##########
@@ -0,0 +1,226 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use crate::util::{BenchmarkRun, CommonOpt, QueryResult};
+use clap::Args;
+use datafusion::physical_plan::execute_stream;
+use datafusion::{error::Result, prelude::SessionContext};
+use datafusion_common::instant::Instant;
+use datafusion_common::{DataFusionError, exec_datafusion_err, exec_err};
+use futures::StreamExt;
+
+/// Run end-to-end ASOF join benchmarks.
+///
+/// The cases cover broadcast-side size asymmetry, equality-key cardinality and
+/// skew, left-side parallelism, optimizer-inserted ordering, wide payload
+/// materialization, and descending successor matching.
+#[derive(Debug, Args, Clone)]
+#[command(verbatim_doc_comment)]
+pub struct RunOpt {
+ /// Query number (between 1 and 6). If not specified, runs all queries
+ #[arg(short, long)]
+ query: Option<usize>,
+
+ /// Common options
+ #[command(flatten)]
+ common: CommonOpt,
+
+ /// If present, write results json here
+ #[arg(short = 'o', long = "output")]
+ output_path: Option<std::path::PathBuf>,
+}
+
+const ASOF_QUERIES: &[&str] = &[
+ // Q1: small broadcast input and a large, equality-free probe input
+ r#"
+ WITH left_input AS (
+ SELECT value AS ts, value AS payload FROM range(1000000)
+ ),
+ right_input AS (
+ SELECT value AS ts, value AS payload FROM range(10000)
+ )
+ SELECT l.ts, l.payload, r.payload AS right_payload
+ FROM left_input l
+ ASOF JOIN right_input r MATCH_CONDITION (l.ts >= r.ts)
+ "#,
+ // Q2: grouped predecessor, optimizer partitions left and coalesces right
+ r#"
+ WITH left_input AS (
+ SELECT value % 10000 AS key,
+ value / 10000 + 1 AS ts,
+ value AS payload
+ FROM range(1000000)
+ ),
+ right_input AS (
+ SELECT value % 10000 AS key,
+ value / 10000 AS ts,
+ value AS payload
+ FROM range(1000000)
+ )
+ SELECT l.key, l.ts, l.payload, r.payload AS right_payload
+ FROM left_input l
+ ASOF JOIN right_input r MATCH_CONDITION (l.ts >= r.ts)
+ ON l.key = r.key
+ "#,
+ // Q3: grouped predecessor with a wide payload
+ r#"
+ WITH left_input AS (
+ SELECT value % 10000 AS key,
+ value / 10000 + 1 AS ts,
+ repeat('x', 256) AS payload
+ FROM range(250000)
+ ),
+ right_input AS (
+ SELECT value % 10000 AS key,
+ value / 10000 AS ts,
+ repeat('y', 256) AS payload
+ FROM range(250000)
+ )
+ SELECT l.key, l.ts, l.payload, r.payload AS right_payload
+ FROM left_input l
+ ASOF JOIN right_input r MATCH_CONDITION (l.ts >= r.ts)
+ ON l.key = r.key
+ "#,
+ // Q4: successor matching requires descending input order
Review Comment:
Yep, agreed. The semantics are explicit now.
##########
datafusion/physical-plan/benches/asof_join.rs:
##########
@@ -0,0 +1,219 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Criterion benchmarks for the pre-sorted broadcast ASOF join kernel.
Review Comment:
Sounds good. Keeping only the SQL benchmark now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] bench: add ASOF join benchmarks [datafusion]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on PR #23833:
URL: https://github.com/apache/datafusion/pull/23833#issuecomment-5658398929
> I've tested it locally and it works great.
>
> Suggested a small cleanup item — we could do it as a follow-up.
Thank you! And it looks like a simple change, let me do it now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org