You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "notashes (via GitHub)" <gi...@apache.org> on 2026/02/06 08:55:23 UTC

[PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

notashes opened a new pull request, #20182:
URL: https://github.com/apache/datafusion/pull/20182

   ## Which issue does this PR close?
   
   <!--
   We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
   -->
   
   - Closes #20181 
   
   ## Rationale for this change
   
   <!--
    Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.  
   -->
   Issue #20152 identifies potential areas for optimization for `RunArray` and `StructArray` hashing. But the existing `with_hashes` benchmark tests don't include coverage for these types.
   
   
   ## What changes are included in this PR?
   
   <!--
   There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
   -->
   Added benchmarks to `with_hashes.rs`:
     - **StructArray**: 4-column struct (bool, int32, int64, string)
     - **RunArray**: Int32 run-encoded array
     - Both include single/multiple columns and with/without nulls
   
   ## Are these changes tested?
   
   <!--
   We typically require tests for all PRs in order to:
   1. Prevent the code from being accidentally broken by subsequent changes
   2. Serve as another way to document the expected behavior of the code
   
   If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
   -->
   
   No additional tests added, but the benchmarks both compile and run. 
   
   <details>
   <summary>a sample run:</summary>
   
   ```
   ❯ cargo bench --features=parquet --bench with_hashes -- array
      Compiling datafusion-common v52.1.0 (/Users/notashes/dev/datafusion/datafusion/common)
       Finished `bench` profile [optimized] target(s) in 34.49s
        Running benches/with_hashes.rs (target/release/deps/with_hashes-2f180744d22084f3)
   Gnuplot not found, using plotters backend
   struct_array: single, no nulls
                           time:   [38.389 µs 38.437 µs 38.485 µs]
   Found 5 outliers among 100 measurements (5.00%)
     1 (1.00%) low severe
     2 (2.00%) low mild
     2 (2.00%) high mild
   
   struct_array: single, nulls
                           time:   [46.108 µs 46.197 µs 46.291 µs]
   Found 4 outliers among 100 measurements (4.00%)
     3 (3.00%) high mild
     1 (1.00%) high severe
   
   struct_array: multiple, no nulls
                           time:   [114.64 µs 114.79 µs 114.93 µs]
   Found 4 outliers among 100 measurements (4.00%)
     1 (1.00%) low severe
     2 (2.00%) low mild
     1 (1.00%) high mild
   
   struct_array: multiple, nulls
                           time:   [138.29 µs 138.62 µs 139.07 µs]
   Found 8 outliers among 100 measurements (8.00%)
     1 (1.00%) low severe
     4 (4.00%) low mild
     1 (1.00%) high mild
     2 (2.00%) high severe
   
   run_array_int32: single, no nulls
                           time:   [1.8777 µs 1.9098 µs 1.9457 µs]
   Found 3 outliers among 100 measurements (3.00%)
     3 (3.00%) high mild
   
   run_array_int32: single, nulls
                           time:   [2.0110 µs 2.0417 µs 2.0751 µs]
   Found 7 outliers among 100 measurements (7.00%)
     6 (6.00%) high mild
     1 (1.00%) high severe
   
   run_array_int32: multiple, no nulls
                           time:   [5.0511 µs 5.0603 µs 5.0693 µs]
   Found 6 outliers among 100 measurements (6.00%)
     1 (1.00%) low mild
     5 (5.00%) high mild
   
   run_array_int32: multiple, nulls
                           time:   [5.6052 µs 5.6201 µs 5.6353 µs]
   Found 4 outliers among 100 measurements (4.00%)
     3 (3.00%) high mild
     1 (1.00%) high severe
     ```
   
   
   </details>
   
   ## Are there any user-facing changes?
   
   <!--
   If there are user-facing changes then we may require documentation to be updated before approving the PR.
   -->
   
   <!--
   If there are any breaking changes to public APIs, please add the `api change` label.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20182:
URL: https://github.com/apache/datafusion/pull/20182#issuecomment-3886218449

   🤖 `./gh_compare_branch_bench.sh` [compare_branch_bench.sh](https://github.com/alamb/datafusion-benchmarking/blob/main/scripts/compare_branch_bench.sh) Running
   Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
   Comparing with_hashes (b485c65778071cf9a84e9b741d0513d0478bceb4) to f9d37cf769454fd1ce3f5af74ffd9323664a4c6d [diff](https://github.com/apache/datafusion/compare/f9d37cf769454fd1ce3f5af74ffd9323664a4c6d..b485c65778071cf9a84e9b741d0513d0478bceb4)
   BENCH_NAME=with_hashes
   BENCH_COMMAND=cargo bench --features=parquet --bench with_hashes
   BENCH_FILTER=
   BENCH_BRANCH_NAME=with_hashes
   Results will be posted here when complete
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "notashes (via GitHub)" <gi...@apache.org>.
notashes commented on code in PR #20182:
URL: https://github.com/apache/datafusion/pull/20182#discussion_r2779128558


##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -205,5 +269,79 @@ where
     Arc::new(array)
 }
 
+fn boolean_array(array_len: usize) -> ArrayRef {
+    let mut rng = make_rng();
+    Arc::new(
+        (0..array_len)
+            .map(|_| Some(rng.random::<bool>()))
+            .collect::<arrow::array::BooleanArray>(),
+    )
+}
+
+fn string_array(array_len: usize) -> ArrayRef {

Review Comment:
   done! don't think a different one offers any benefit! both seem to give me close to 10% speed up locally (with the struct_array optimization)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "notashes (via GitHub)" <gi...@apache.org>.
notashes commented on code in PR #20182:
URL: https://github.com/apache/datafusion/pull/20182#discussion_r2778645410


##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -47,50 +51,75 @@ fn criterion_benchmark(c: &mut Criterion) {
         BenchData {
             name: "int64",
             array: primitive_array::<Int64Type>(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "utf8",
             array: pool.string_array::<i32>(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "large_utf8",
             array: pool.string_array::<i64>(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "utf8_view",
             array: pool.string_view_array(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "utf8_view (small)",
             array: small_pool.string_view_array(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "dictionary_utf8_int32",
             array: pool.dictionary_array::<Int32Type>(BATCH_SIZE),
+            supports_nulls: true,
+        },
+        BenchData {
+            name: "struct_array",
+            array: create_struct_array(BATCH_SIZE),
+            supports_nulls: true,
+        },
+        BenchData {
+            name: "run_array_int32",
+            array: create_run_array::<Int32Type>(BATCH_SIZE),
+            supports_nulls: true,
         },
     ];
 
-    for BenchData { name, array } in cases {
-        // with_hash has different code paths for single vs multiple arrays and nulls vs no nulls
-        let nullable_array = add_nulls(&array);
+    for BenchData {
+        name,
+        array,
+        supports_nulls,
+    } in cases
+    {
         c.bench_function(&format!("{name}: single, no nulls"), |b| {
             do_hash_test(b, std::slice::from_ref(&array));
         });
-        c.bench_function(&format!("{name}: single, nulls"), |b| {
-            do_hash_test(b, std::slice::from_ref(&nullable_array));
-        });
         c.bench_function(&format!("{name}: multiple, no nulls"), |b| {
             let arrays = vec![array.clone(), array.clone(), array.clone()];
             do_hash_test(b, &arrays);
         });
-        c.bench_function(&format!("{name}: multiple, nulls"), |b| {
-            let arrays = vec![
-                nullable_array.clone(),
-                nullable_array.clone(),
-                nullable_array.clone(),
-            ];
-            do_hash_test(b, &arrays);
-        });
+
+        // Union arrays can't have null bitmasks

Review Comment:
   I've copied that from the other PR verbatim 😅 (to not have merge conflicts in the future?). but I think thats the wrong approach now! 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on code in PR #20182:
URL: https://github.com/apache/datafusion/pull/20182#discussion_r2778425679


##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -47,50 +51,75 @@ fn criterion_benchmark(c: &mut Criterion) {
         BenchData {
             name: "int64",
             array: primitive_array::<Int64Type>(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "utf8",
             array: pool.string_array::<i32>(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "large_utf8",
             array: pool.string_array::<i64>(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "utf8_view",
             array: pool.string_view_array(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "utf8_view (small)",
             array: small_pool.string_view_array(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "dictionary_utf8_int32",
             array: pool.dictionary_array::<Int32Type>(BATCH_SIZE),
+            supports_nulls: true,
+        },
+        BenchData {
+            name: "struct_array",
+            array: create_struct_array(BATCH_SIZE),
+            supports_nulls: true,
+        },
+        BenchData {
+            name: "run_array_int32",
+            array: create_run_array::<Int32Type>(BATCH_SIZE),
+            supports_nulls: true,
         },
     ];
 
-    for BenchData { name, array } in cases {
-        // with_hash has different code paths for single vs multiple arrays and nulls vs no nulls
-        let nullable_array = add_nulls(&array);
+    for BenchData {
+        name,
+        array,
+        supports_nulls,
+    } in cases
+    {
         c.bench_function(&format!("{name}: single, no nulls"), |b| {
             do_hash_test(b, std::slice::from_ref(&array));
         });
-        c.bench_function(&format!("{name}: single, nulls"), |b| {
-            do_hash_test(b, std::slice::from_ref(&nullable_array));
-        });
         c.bench_function(&format!("{name}: multiple, no nulls"), |b| {
             let arrays = vec![array.clone(), array.clone(), array.clone()];
             do_hash_test(b, &arrays);
         });
-        c.bench_function(&format!("{name}: multiple, nulls"), |b| {
-            let arrays = vec![
-                nullable_array.clone(),
-                nullable_array.clone(),
-                nullable_array.clone(),
-            ];
-            do_hash_test(b, &arrays);
-        });
+
+        // Union arrays can't have null bitmasks

Review Comment:
   Mentioning union array when we don't implement that here?



##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -122,16 +151,51 @@ where
     builder.finish().expect("should be nulls in buffer")
 }
 
-// Returns an new array that is the same as array, but with nulls
+// Returns a new array that is the same as array, but with nulls
+// Handles the special case of RunArray where nulls must be in the values array
 fn add_nulls(array: &ArrayRef) -> ArrayRef {
-    let array_data = array
-        .clone()
-        .into_data()
-        .into_builder()
-        .nulls(Some(create_null_mask(array.len())))
-        .build()
-        .unwrap();
-    make_array(array_data)
+    use arrow::datatypes::DataType;
+
+    match array.data_type() {
+        DataType::RunEndEncoded(_, _) => {
+            // RunArray can't have top-level nulls, so apply nulls to the values array
+            let run_array = array
+                .as_any()
+                .downcast_ref::<RunArray<Int32Type>>()
+                .expect("Expected RunArray");
+
+            let run_ends_buffer = run_array.run_ends().inner().clone();
+            let run_ends_array = PrimitiveArray::<Int32Type>::new(run_ends_buffer, None);
+            let values = run_array.values().clone();
+
+            // Add nulls to the values array
+            let values_with_nulls = {
+                let array_data = values
+                    .clone()
+                    .into_data()
+                    .into_builder()
+                    .nulls(Some(create_null_mask(values.len())))

Review Comment:
   Something to think about is how null density acts differently here for run arrays, since we'd apply null on entire runs 🤔 



##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -205,5 +269,79 @@ where
     Arc::new(array)
 }
 
+fn boolean_array(array_len: usize) -> ArrayRef {
+    let mut rng = make_rng();
+    Arc::new(
+        (0..array_len)
+            .map(|_| Some(rng.random::<bool>()))
+            .collect::<arrow::array::BooleanArray>(),
+    )
+}
+
+fn string_array(array_len: usize) -> ArrayRef {

Review Comment:
   Do we need this if we already have `StringPool` above?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on PR #20182:
URL: https://github.com/apache/datafusion/pull/20182#issuecomment-3884681965

   run benchmark with_hashes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "notashes (via GitHub)" <gi...@apache.org>.
notashes commented on code in PR #20182:
URL: https://github.com/apache/datafusion/pull/20182#discussion_r2778165967


##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -205,5 +222,123 @@ where
     Arc::new(array)
 }
 
+/// Create a StructArray with multiple columns
+fn create_struct_array(array_len: usize) -> ArrayRef {
+    let mut rng = make_rng();
+
+    // Create 4 columns of different types for our struct array
+    let bool_array: ArrayRef = Arc::new(
+        (0..array_len)
+            .map(|_| Some(rng.random::<bool>()))
+            .collect::<arrow::array::BooleanArray>(),
+    );
+
+    let int32_array: ArrayRef = Arc::new(
+        (0..array_len)
+            .map(|_| Some(rng.random::<i32>()))
+            .collect::<PrimitiveArray<Int32Type>>(),
+    );
+
+    let int64_array: ArrayRef = Arc::new(
+        (0..array_len)
+            .map(|_| Some(rng.random::<i64>()))
+            .collect::<PrimitiveArray<Int64Type>>(),
+    );
+
+    let string_array: ArrayRef = {
+        let strings: Vec<String> = (0..array_len)
+            .map(|_| {
+                let len = rng.random_range(5..20);
+                let value: Vec<u8> =
+                    rng.clone().sample_iter(&Alphanumeric).take(len).collect();
+                String::from_utf8(value).unwrap()
+            })
+            .collect();
+        Arc::new(StringArray::from(strings))
+    };
+
+    let fields = Fields::from(vec![
+        Field::new("bool_col", DataType::Boolean, false),
+        Field::new("int32_col", DataType::Int32, false),
+        Field::new("int64_col", DataType::Int64, false),
+        Field::new("string_col", DataType::Utf8, false),
+    ]);
+
+    Arc::new(StructArray::new(
+        fields,
+        vec![bool_array, int32_array, int64_array, string_array],
+        None,
+    ))
+}
+
+/// Create a RunArray to test run array hashing.
+fn create_run_array<T>(array_len: usize) -> ArrayRef
+where
+    T: ArrowPrimitiveType,
+    StandardUniform: Distribution<T::Native>,
+{
+    let mut rng = make_rng();
+
+    // Create runs of varying lengths
+    let mut run_ends = Vec::new();
+    let mut values = Vec::new();
+    let mut current_end = 0;
+
+    while current_end < array_len {
+        // Random run length between 1 and 50
+        let run_length = rng.random_range(1..=50).min(array_len - current_end);
+        current_end += run_length;
+        run_ends.push(current_end as i32);
+        values.push(Some(rng.random::<T::Native>()));
+    }
+
+    let run_ends_array = Arc::new(PrimitiveArray::<Int32Type>::from(run_ends));
+    let values_array: Arc<dyn Array> =
+        Arc::new(values.into_iter().collect::<PrimitiveArray<T>>());
+
+    Arc::new(
+        RunArray::try_new(&run_ends_array, values_array.as_ref())
+            .expect("Failed to create RunArray"),
+    )
+}
+
+/// Create a RunArray with null values
+fn create_run_array_with_null_values<T>(array_len: usize) -> ArrayRef

Review Comment:
   gotcha! i tried pattern matching within add_nulls() and extract out the values array to insert nulls there. it seems better!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "notashes (via GitHub)" <gi...@apache.org>.
notashes commented on code in PR #20182:
URL: https://github.com/apache/datafusion/pull/20182#discussion_r2778165967


##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -205,5 +222,123 @@ where
     Arc::new(array)
 }
 
+/// Create a StructArray with multiple columns
+fn create_struct_array(array_len: usize) -> ArrayRef {
+    let mut rng = make_rng();
+
+    // Create 4 columns of different types for our struct array
+    let bool_array: ArrayRef = Arc::new(
+        (0..array_len)
+            .map(|_| Some(rng.random::<bool>()))
+            .collect::<arrow::array::BooleanArray>(),
+    );
+
+    let int32_array: ArrayRef = Arc::new(
+        (0..array_len)
+            .map(|_| Some(rng.random::<i32>()))
+            .collect::<PrimitiveArray<Int32Type>>(),
+    );
+
+    let int64_array: ArrayRef = Arc::new(
+        (0..array_len)
+            .map(|_| Some(rng.random::<i64>()))
+            .collect::<PrimitiveArray<Int64Type>>(),
+    );
+
+    let string_array: ArrayRef = {
+        let strings: Vec<String> = (0..array_len)
+            .map(|_| {
+                let len = rng.random_range(5..20);
+                let value: Vec<u8> =
+                    rng.clone().sample_iter(&Alphanumeric).take(len).collect();
+                String::from_utf8(value).unwrap()
+            })
+            .collect();
+        Arc::new(StringArray::from(strings))
+    };
+
+    let fields = Fields::from(vec![
+        Field::new("bool_col", DataType::Boolean, false),
+        Field::new("int32_col", DataType::Int32, false),
+        Field::new("int64_col", DataType::Int64, false),
+        Field::new("string_col", DataType::Utf8, false),
+    ]);
+
+    Arc::new(StructArray::new(
+        fields,
+        vec![bool_array, int32_array, int64_array, string_array],
+        None,
+    ))
+}
+
+/// Create a RunArray to test run array hashing.
+fn create_run_array<T>(array_len: usize) -> ArrayRef
+where
+    T: ArrowPrimitiveType,
+    StandardUniform: Distribution<T::Native>,
+{
+    let mut rng = make_rng();
+
+    // Create runs of varying lengths
+    let mut run_ends = Vec::new();
+    let mut values = Vec::new();
+    let mut current_end = 0;
+
+    while current_end < array_len {
+        // Random run length between 1 and 50
+        let run_length = rng.random_range(1..=50).min(array_len - current_end);
+        current_end += run_length;
+        run_ends.push(current_end as i32);
+        values.push(Some(rng.random::<T::Native>()));
+    }
+
+    let run_ends_array = Arc::new(PrimitiveArray::<Int32Type>::from(run_ends));
+    let values_array: Arc<dyn Array> =
+        Arc::new(values.into_iter().collect::<PrimitiveArray<T>>());
+
+    Arc::new(
+        RunArray::try_new(&run_ends_array, values_array.as_ref())
+            .expect("Failed to create RunArray"),
+    )
+}
+
+/// Create a RunArray with null values
+fn create_run_array_with_null_values<T>(array_len: usize) -> ArrayRef

Review Comment:
   gotcha! i tried pattern matching within `add_nulls()` and extract out the `values` array to insert nulls there. it seems better!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "notashes (via GitHub)" <gi...@apache.org>.
notashes commented on code in PR #20182:
URL: https://github.com/apache/datafusion/pull/20182#discussion_r2778645410


##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -47,50 +51,75 @@ fn criterion_benchmark(c: &mut Criterion) {
         BenchData {
             name: "int64",
             array: primitive_array::<Int64Type>(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "utf8",
             array: pool.string_array::<i32>(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "large_utf8",
             array: pool.string_array::<i64>(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "utf8_view",
             array: pool.string_view_array(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "utf8_view (small)",
             array: small_pool.string_view_array(BATCH_SIZE),
+            supports_nulls: true,
         },
         BenchData {
             name: "dictionary_utf8_int32",
             array: pool.dictionary_array::<Int32Type>(BATCH_SIZE),
+            supports_nulls: true,
+        },
+        BenchData {
+            name: "struct_array",
+            array: create_struct_array(BATCH_SIZE),
+            supports_nulls: true,
+        },
+        BenchData {
+            name: "run_array_int32",
+            array: create_run_array::<Int32Type>(BATCH_SIZE),
+            supports_nulls: true,
         },
     ];
 
-    for BenchData { name, array } in cases {
-        // with_hash has different code paths for single vs multiple arrays and nulls vs no nulls
-        let nullable_array = add_nulls(&array);
+    for BenchData {
+        name,
+        array,
+        supports_nulls,
+    } in cases
+    {
         c.bench_function(&format!("{name}: single, no nulls"), |b| {
             do_hash_test(b, std::slice::from_ref(&array));
         });
-        c.bench_function(&format!("{name}: single, nulls"), |b| {
-            do_hash_test(b, std::slice::from_ref(&nullable_array));
-        });
         c.bench_function(&format!("{name}: multiple, no nulls"), |b| {
             let arrays = vec![array.clone(), array.clone(), array.clone()];
             do_hash_test(b, &arrays);
         });
-        c.bench_function(&format!("{name}: multiple, nulls"), |b| {
-            let arrays = vec![
-                nullable_array.clone(),
-                nullable_array.clone(),
-                nullable_array.clone(),
-            ];
-            do_hash_test(b, &arrays);
-        });
+
+        // Union arrays can't have null bitmasks

Review Comment:
   I've copied that from the other PR verbatim 😅 (to not have merge conflicts in the future?). but I'm getting a sense that it's the wrong approach here! 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "notashes (via GitHub)" <gi...@apache.org>.
notashes commented on code in PR #20182:
URL: https://github.com/apache/datafusion/pull/20182#discussion_r2778163891


##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -68,11 +71,25 @@ fn criterion_benchmark(c: &mut Criterion) {
             name: "dictionary_utf8_int32",
             array: pool.dictionary_array::<Int32Type>(BATCH_SIZE),
         },
+        BenchData {
+            name: "struct_array",
+            array: create_struct_array(BATCH_SIZE),
+        },
+        BenchData {
+            name: "run_array_int32",
+            array: create_run_array::<Int32Type>(BATCH_SIZE),
+        },
     ];
 
     for BenchData { name, array } in cases {
         // with_hash has different code paths for single vs multiple arrays and nulls vs no nulls
-        let nullable_array = add_nulls(&array);
+        // RunArray encodes nulls in the values array, not at the array level
+        let nullable_array = if name.starts_with("run_array") {

Review Comment:
   Hey @Jefffrey thanks for the reviews! I've followed the pattern from the other PR! 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "notashes (via GitHub)" <gi...@apache.org>.
notashes commented on code in PR #20182:
URL: https://github.com/apache/datafusion/pull/20182#discussion_r2778164988


##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -205,5 +222,123 @@ where
     Arc::new(array)
 }
 
+/// Create a StructArray with multiple columns
+fn create_struct_array(array_len: usize) -> ArrayRef {
+    let mut rng = make_rng();
+
+    // Create 4 columns of different types for our struct array
+    let bool_array: ArrayRef = Arc::new(
+        (0..array_len)
+            .map(|_| Some(rng.random::<bool>()))
+            .collect::<arrow::array::BooleanArray>(),
+    );
+
+    let int32_array: ArrayRef = Arc::new(

Review Comment:
   done! and yes the benchmark numbers are pretty much the same. 
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb merged PR #20182:
URL: https://github.com/apache/datafusion/pull/20182


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "alamb-ghbot (via GitHub)" <gi...@apache.org>.
alamb-ghbot commented on PR #20182:
URL: https://github.com/apache/datafusion/pull/20182#issuecomment-3886271127

   🤖: Benchmark completed
   
   <details><summary>Details</summary>
   <p>
   
   
   ```
   group                                        main                                   with_hashes
   -----                                        ----                                   -----------
   dictionary_utf8_int32: multiple, no nulls    1.00     74.1±0.79µs        ? ?/sec    1.03     76.2±2.05µs        ? ?/sec
   dictionary_utf8_int32: multiple, nulls       1.00     90.9±0.46µs        ? ?/sec    1.00     91.0±0.15µs        ? ?/sec
   dictionary_utf8_int32: single, no nulls      1.06     27.8±0.22µs        ? ?/sec    1.00     26.2±0.13µs        ? ?/sec
   dictionary_utf8_int32: single, nulls         1.00     38.5±1.10µs        ? ?/sec    1.00     38.3±0.17µs        ? ?/sec
   int64: multiple, no nulls                    1.00     38.7±0.20µs        ? ?/sec    1.00     38.8±0.15µs        ? ?/sec
   int64: multiple, nulls                       1.00     75.0±2.67µs        ? ?/sec    1.00     74.8±0.42µs        ? ?/sec
   int64: single, no nulls                      1.00     11.3±0.08µs        ? ?/sec    1.00     11.3±0.05µs        ? ?/sec
   int64: single, nulls                         1.00     23.9±0.12µs        ? ?/sec    1.00     23.9±0.33µs        ? ?/sec
   large_utf8: multiple, no nulls               1.00    229.6±5.45µs        ? ?/sec    1.00    229.1±1.86µs        ? ?/sec
   large_utf8: multiple, nulls                  1.00    280.4±1.33µs        ? ?/sec    1.00    280.9±3.47µs        ? ?/sec
   large_utf8: single, no nulls                 1.01     72.4±2.77µs        ? ?/sec    1.00     71.9±1.51µs        ? ?/sec
   large_utf8: single, nulls                    1.00     81.6±0.34µs        ? ?/sec    1.00     81.4±0.77µs        ? ?/sec
   run_array_int32: multiple, no nulls                                                 1.00     12.0±0.09µs        ? ?/sec
   run_array_int32: multiple, nulls                                                    1.00     13.4±0.07µs        ? ?/sec
   run_array_int32: single, no nulls                                                   1.00      3.9±0.03µs        ? ?/sec
   run_array_int32: single, nulls                                                      1.00      4.4±0.08µs        ? ?/sec
   struct_array: multiple, no nulls                                                    1.00   407.7±12.90µs        ? ?/sec
   struct_array: multiple, nulls                                                       1.00   442.9±15.81µs        ? ?/sec
   struct_array: single, no nulls                                                      1.00    135.5±0.53µs        ? ?/sec
   struct_array: single, nulls                                                         1.00    147.5±3.66µs        ? ?/sec
   utf8: multiple, no nulls                     1.00    244.4±3.36µs        ? ?/sec    1.00    243.8±1.25µs        ? ?/sec
   utf8: multiple, nulls                        1.00    264.4±3.37µs        ? ?/sec    1.00    263.8±2.87µs        ? ?/sec
   utf8: single, no nulls                       1.00     73.7±0.60µs        ? ?/sec    1.00     73.6±1.01µs        ? ?/sec
   utf8: single, nulls                          1.00     83.2±1.92µs        ? ?/sec    1.00     83.2±2.03µs        ? ?/sec
   utf8_view (small): multiple, no nulls        1.00     47.7±1.50µs        ? ?/sec    1.00     47.6±0.15µs        ? ?/sec
   utf8_view (small): multiple, nulls           1.00     64.1±0.30µs        ? ?/sec    1.00     64.2±0.22µs        ? ?/sec
   utf8_view (small): single, no nulls          1.00     13.9±0.27µs        ? ?/sec    1.00     13.9±0.04µs        ? ?/sec
   utf8_view (small): single, nulls             1.00     21.6±0.05µs        ? ?/sec    1.00     21.6±0.08µs        ? ?/sec
   utf8_view: multiple, no nulls                1.00    235.4±2.64µs        ? ?/sec    1.00    235.9±1.64µs        ? ?/sec
   utf8_view: multiple, nulls                   1.00    227.0±1.25µs        ? ?/sec    1.00    227.5±1.16µs        ? ?/sec
   utf8_view: single, no nulls                  1.00     72.2±0.54µs        ? ?/sec    1.01     72.6±1.64µs        ? ?/sec
   utf8_view: single, nulls                     1.00     71.9±0.23µs        ? ?/sec    1.00     72.0±0.55µs        ? ?/sec
   ```
   
   
   </p>
   </details>
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on code in PR #20182:
URL: https://github.com/apache/datafusion/pull/20182#discussion_r2777729730


##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -205,5 +222,123 @@ where
     Arc::new(array)
 }
 
+/// Create a StructArray with multiple columns
+fn create_struct_array(array_len: usize) -> ArrayRef {
+    let mut rng = make_rng();
+
+    // Create 4 columns of different types for our struct array
+    let bool_array: ArrayRef = Arc::new(
+        (0..array_len)
+            .map(|_| Some(rng.random::<bool>()))
+            .collect::<arrow::array::BooleanArray>(),
+    );
+
+    let int32_array: ArrayRef = Arc::new(

Review Comment:
   Maybe we could reuse the existing functions above to create these random arrays? e.g. `primitive_array()`
   
   The only difference I see is that it uses its own `rng`; is this a big concern, considering each of these functions currently use their own `rng` anyway?



##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -68,11 +71,25 @@ fn criterion_benchmark(c: &mut Criterion) {
             name: "dictionary_utf8_int32",
             array: pool.dictionary_array::<Int32Type>(BATCH_SIZE),
         },
+        BenchData {
+            name: "struct_array",
+            array: create_struct_array(BATCH_SIZE),
+        },
+        BenchData {
+            name: "run_array_int32",
+            array: create_run_array::<Int32Type>(BATCH_SIZE),
+        },
     ];
 
     for BenchData { name, array } in cases {
         // with_hash has different code paths for single vs multiple arrays and nulls vs no nulls
-        let nullable_array = add_nulls(&array);
+        // RunArray encodes nulls in the values array, not at the array level
+        let nullable_array = if name.starts_with("run_array") {

Review Comment:
   Maybe we should take the approach by #20179 to have this as an explicit property instead of checking by name



##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -205,5 +222,123 @@ where
     Arc::new(array)
 }
 
+/// Create a StructArray with multiple columns
+fn create_struct_array(array_len: usize) -> ArrayRef {
+    let mut rng = make_rng();
+
+    // Create 4 columns of different types for our struct array
+    let bool_array: ArrayRef = Arc::new(
+        (0..array_len)
+            .map(|_| Some(rng.random::<bool>()))
+            .collect::<arrow::array::BooleanArray>(),
+    );
+
+    let int32_array: ArrayRef = Arc::new(
+        (0..array_len)
+            .map(|_| Some(rng.random::<i32>()))
+            .collect::<PrimitiveArray<Int32Type>>(),
+    );
+
+    let int64_array: ArrayRef = Arc::new(
+        (0..array_len)
+            .map(|_| Some(rng.random::<i64>()))
+            .collect::<PrimitiveArray<Int64Type>>(),
+    );
+
+    let string_array: ArrayRef = {
+        let strings: Vec<String> = (0..array_len)
+            .map(|_| {
+                let len = rng.random_range(5..20);
+                let value: Vec<u8> =
+                    rng.clone().sample_iter(&Alphanumeric).take(len).collect();
+                String::from_utf8(value).unwrap()
+            })
+            .collect();
+        Arc::new(StringArray::from(strings))
+    };
+
+    let fields = Fields::from(vec![
+        Field::new("bool_col", DataType::Boolean, false),
+        Field::new("int32_col", DataType::Int32, false),
+        Field::new("int64_col", DataType::Int64, false),
+        Field::new("string_col", DataType::Utf8, false),
+    ]);
+
+    Arc::new(StructArray::new(
+        fields,
+        vec![bool_array, int32_array, int64_array, string_array],
+        None,
+    ))
+}
+
+/// Create a RunArray to test run array hashing.
+fn create_run_array<T>(array_len: usize) -> ArrayRef
+where
+    T: ArrowPrimitiveType,
+    StandardUniform: Distribution<T::Native>,
+{
+    let mut rng = make_rng();
+
+    // Create runs of varying lengths
+    let mut run_ends = Vec::new();
+    let mut values = Vec::new();
+    let mut current_end = 0;
+
+    while current_end < array_len {
+        // Random run length between 1 and 50
+        let run_length = rng.random_range(1..=50).min(array_len - current_end);
+        current_end += run_length;
+        run_ends.push(current_end as i32);
+        values.push(Some(rng.random::<T::Native>()));
+    }
+
+    let run_ends_array = Arc::new(PrimitiveArray::<Int32Type>::from(run_ends));
+    let values_array: Arc<dyn Array> =
+        Arc::new(values.into_iter().collect::<PrimitiveArray<T>>());
+
+    Arc::new(
+        RunArray::try_new(&run_ends_array, values_array.as_ref())
+            .expect("Failed to create RunArray"),
+    )
+}
+
+/// Create a RunArray with null values
+fn create_run_array_with_null_values<T>(array_len: usize) -> ArrayRef

Review Comment:
   I feel we can reduce the duplication with above function here if the only difference is nulls 🤔 
   
   It would just be a matter of calling `add_nulls()` on the values array



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] Add `StructArray` and `RunArray` benchmark tests to `with_hashes` [datafusion]

Posted by "notashes (via GitHub)" <gi...@apache.org>.
notashes commented on code in PR #20182:
URL: https://github.com/apache/datafusion/pull/20182#discussion_r2779152576


##########
datafusion/common/benches/with_hashes.rs:
##########
@@ -122,16 +151,51 @@ where
     builder.finish().expect("should be nulls in buffer")
 }
 
-// Returns an new array that is the same as array, but with nulls
+// Returns a new array that is the same as array, but with nulls
+// Handles the special case of RunArray where nulls must be in the values array
 fn add_nulls(array: &ArrayRef) -> ArrayRef {
-    let array_data = array
-        .clone()
-        .into_data()
-        .into_builder()
-        .nulls(Some(create_null_mask(array.len())))
-        .build()
-        .unwrap();
-    make_array(array_data)
+    use arrow::datatypes::DataType;
+
+    match array.data_type() {
+        DataType::RunEndEncoded(_, _) => {
+            // RunArray can't have top-level nulls, so apply nulls to the values array
+            let run_array = array
+                .as_any()
+                .downcast_ref::<RunArray<Int32Type>>()
+                .expect("Expected RunArray");
+
+            let run_ends_buffer = run_array.run_ends().inner().clone();
+            let run_ends_array = PrimitiveArray::<Int32Type>::new(run_ends_buffer, None);
+            let values = run_array.values().clone();
+
+            // Add nulls to the values array
+            let values_with_nulls = {
+                let array_data = values
+                    .clone()
+                    .into_data()
+                    .into_builder()
+                    .nulls(Some(create_null_mask(values.len())))

Review Comment:
   I was thinking about it for a while. It probably should come up to be around the same 3% zone even though the variance could be a bit high.
   
   I've set the run_length to be within 1..50. 
   Let's say we have  ~300 runs on average, with each each one carrying ~25 elements. 3% of which will roughly translate to `10 * 25 = 250`. But yes that is probably our ideal scenario.  
   
   let me know what you think? i'll try to do some testing regarding this. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org