You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "rluvaton (via GitHub)" <gi...@apache.org> on 2026/10/01 17:41:21 UTC

[PR] chore: reuse allocations in partition [arrow-rs]

rluvaton opened a new pull request, #11324:
URL: https://github.com/apache/arrow-rs/pull/11324

   # Which issue does this PR close?
   
   None
   
   # Rationale for this change
   
   No need to add extra allocation of the boolean buffer, we can reuse the existing one
   
   # What changes are included in this PR?
   changed partition to use mutable bit ops 
   
   # Are these changes tested?
   Existing tests
   
   # Are there any user-facing changes?
   No
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-6003380850

   run benchmark partition_kernels


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-6020119159

   Thank you @rluvaton and @Jefffrey 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#discussion_r4178155759


##########
arrow-ord/src/partition.rs:
##########
@@ -166,6 +169,46 @@ fn find_boundaries(v: &dyn Array) -> Result<BooleanBuffer, ArrowError> {
     Ok((0..slice_len).map(|i| !cmp(i, i).is_eq()).collect())
 }
 
+fn convert_boolean_buffer_to_builder(buffer: BooleanBuffer) -> BooleanBufferBuilder {
+    let len = buffer.len();
+
+    // This should be 0, but in case it doesn't simplify the code by doing a copy to a new buffer

Review Comment:
   I don't understand the comment about why it doesn't simplify the code. Is it because the builder can't handle non zero offsets?



##########
arrow-ord/src/partition.rs:
##########
@@ -166,6 +169,46 @@ fn find_boundaries(v: &dyn Array) -> Result<BooleanBuffer, ArrowError> {
     Ok((0..slice_len).map(|i| !cmp(i, i).is_eq()).collect())
 }
 
+fn convert_boolean_buffer_to_builder(buffer: BooleanBuffer) -> BooleanBufferBuilder {
+    let len = buffer.len();
+
+    // This should be 0, but in case it doesn't simplify the code by doing a copy to a new buffer
+    let buffer = if buffer.offset() != 0 {
+        // Recreate
+        BooleanBuffer::from_iter(buffer.iter())
+    } else {
+        buffer
+    };
+    assert_eq!(buffer.offset(), 0);
+
+    let buffer = buffer.into_inner();
+
+    let mutable = buffer
+        .into_mutable()
+        // This should not happen but just in case copy to owned mutable buffer
+        .unwrap_or_else(|buffer| {
+            let mut mutable = MutableBuffer::new(buffer.len());
+            // SAFETY: We just allocated a buffer of the same length as the original buffer, so it is safe to set the length to the original buffer's length.
+            unsafe { mutable.set_len(buffer.len()) };
+            mutable.as_slice_mut().copy_from_slice(buffer.as_slice());
+
+            mutable
+        });
+
+    BooleanBufferBuilder::new_from_buffer(mutable, len)
+}
+
+fn or(lhs: &mut BooleanBufferBuilder, rhs: BooleanBuffer) {
+    apply_bitwise_binary_op(
+        lhs.as_slice_mut(),
+        0,

Review Comment:
   You know this is zero because the builder always has zero offset?



##########
arrow-ord/src/partition.rs:
##########
@@ -143,12 +144,14 @@ pub fn partition(columns: &[ArrayRef]) -> Result<Partitions, ArrowError> {
     }
 
     let acc = find_boundaries(&columns[0])?;
-    let acc = columns
-        .iter()
-        .skip(1)
-        .try_fold(acc, |acc, c| find_boundaries(c.as_ref()).map(|b| &acc | &b))?;
+    let mut builder = convert_boolean_buffer_to_builder(acc);
 
-    Ok(Partitions(Some(acc)))
+    for column in columns.iter().skip(1) {
+        let boundaries = find_boundaries(column.as_ref())?;

Review Comment:
   Here could weconvert back to boolean buffer, call `or` and then recreate the builder? I do agree that having a native `or` implementation on the builder would make the code simpler



##########
arrow-ord/src/partition.rs:
##########
@@ -166,6 +169,46 @@ fn find_boundaries(v: &dyn Array) -> Result<BooleanBuffer, ArrowError> {
     Ok((0..slice_len).map(|i| !cmp(i, i).is_eq()).collect())
 }
 
+fn convert_boolean_buffer_to_builder(buffer: BooleanBuffer) -> BooleanBufferBuilder {
+    let len = buffer.len();
+
+    // This should be 0, but in case it doesn't simplify the code by doing a copy to a new buffer
+    let buffer = if buffer.offset() != 0 {
+        // Recreate
+        BooleanBuffer::from_iter(buffer.iter())
+    } else {
+        buffer
+    };
+    assert_eq!(buffer.offset(), 0);
+
+    let buffer = buffer.into_inner();
+
+    let mutable = buffer
+        .into_mutable()
+        // This should not happen but just in case copy to owned mutable buffer
+        .unwrap_or_else(|buffer| {
+            let mut mutable = MutableBuffer::new(buffer.len());
+            // SAFETY: We just allocated a buffer of the same length as the original buffer, so it is safe to set the length to the original buffer's length.
+            unsafe { mutable.set_len(buffer.len()) };
+            mutable.as_slice_mut().copy_from_slice(buffer.as_slice());
+
+            mutable
+        });
+
+    BooleanBufferBuilder::new_from_buffer(mutable, len)
+}
+
+fn or(lhs: &mut BooleanBufferBuilder, rhs: BooleanBuffer) {
+    apply_bitwise_binary_op(
+        lhs.as_slice_mut(),
+        0,
+        rhs.values(),
+        rhs.offset(),
+        rhs.len(),
+        |a, b| a | b,
+    )
+}

Review Comment:
   Maybe it is time to add the API to `BooleanBufferBuilder` -- with a usecase it seems reasonable to me
   
   We could also note the differences potentially as a comment



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#discussion_r4178153310


##########
arrow-ord/src/partition.rs:
##########
@@ -166,6 +169,46 @@ fn find_boundaries(v: &dyn Array) -> Result<BooleanBuffer, ArrowError> {
     Ok((0..slice_len).map(|i| !cmp(i, i).is_eq()).collect())
 }
 
+fn convert_boolean_buffer_to_builder(buffer: BooleanBuffer) -> BooleanBufferBuilder {

Review Comment:
   I agree it would be easier to find and reuse as a `From` impl to `BooleanBufferBuilder`
   
   Then it would be straightforward to go back and forth between them



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-5981360283

   🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11324#issuecomment-5981359704)
   
   **Target:** partition&#95;kernels
   
   **Sharding factor: 1** (1 workers).
   
   Comparing reuse-allocation-in-partition (e9e85903403da613e1de85909149943a6f881c8e) to 2166741 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/2166741578fed12de6133cb29f0223456fd32123..e9e85903403da613e1de85909149943a6f881c8e)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark partition_kernels
   shards: 1
   
   ```
   
   </details>
   
   Results will be posted when all workers finish.
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:5981359704:start:41dd25c8822086cb51b276101141a0bb -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-5981388314

   > Thanks @rluvaton  and @Jefffrey  -- adding some bitwise methods to the builder seems like a good idea 
   > 
   > Reviewing the comments on https://github.com/apache/arrow-rs/pull/11324/changes#r4158553171 it seems I was overwhelmed with that PR trying to get it in, so scoped back the APIs to limit what I had to review.
   > 
   > I would be happy to review a PR to add bitwise kernels on the builder now (ideally as a separate PR but we could also add them to this one0
   
   I'll create a separate pr shortly


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-5981359704

   run benchmark partition_kernels


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-6003383282

   🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11324#issuecomment-6003380850)
   
   **Target:** partition&#95;kernels
   
   **Sharding factor: 1** (1 workers).
   
   Comparing reuse-allocation-in-partition (25a4877efd855cb935a592347984e8d32b133723) to 103ebe7 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/103ebe7548db35c37e8ff35fcb02438bcd182fa7..25a4877efd855cb935a592347984e8d32b133723)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark partition_kernels
   shards: 1
   
   ```
   
   </details>
   
   Results will be posted when all workers finish.
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6003380850:start:1bde2bcc33177198c46a221cc4d9f350 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#discussion_r4189045719


##########
arrow-ord/src/partition.rs:
##########
@@ -143,10 +143,14 @@ pub fn partition(columns: &[ArrayRef]) -> Result<Partitions, ArrowError> {
     }
 
     let acc = find_boundaries(&columns[0])?;
-    let acc = columns
-        .iter()
-        .skip(1)
-        .try_fold(acc, |acc, c| find_boundaries(c.as_ref()).map(|b| &acc | &b))?;
+    let acc = columns.iter().skip(1).try_fold(acc, |mut acc, c| {
+        find_boundaries(c.as_ref()).map(|b| {
+            // Do OrAssign to try to reuse the existing buffer if possible
+            acc |= &b;

Review Comment:
   🧑‍🍳 👌 
   
   Very nice



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-6004705015

   
   run benchmark partition_kernels
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-6007429417

   PR is much nicer now, though should we be concerned with the benchmark results here? was hoping at least itll be 1.00 across the board at the worst 🤔 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-5992860796

   I've updated the code to use existing `BooleanBuffer` `OrAssign` which already handle the case unable to get mutable, and if it can do in place update, so the code is much simpler now 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-5981703535

   🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11324#issuecomment-5981702926)
   
   **Target:** partition&#95;kernels
   
   **Sharding factor: 1** (1 workers).
   
   Comparing reuse-allocation-in-partition (e9e85903403da613e1de85909149943a6f881c8e) to 2166741 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/2166741578fed12de6133cb29f0223456fd32123..e9e85903403da613e1de85909149943a6f881c8e)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark partition_kernels
   shards: 1
   
   ```
   
   </details>
   
   Results will be posted when all workers finish.
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:5981702926:start:00b2eb4848873b3be093395c964eb944 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-6004706567

   🤖 Benchmark starting (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11324#issuecomment-6004705015)
   
   **Target:** partition&#95;kernels
   
   **Sharding factor: 1** (1 workers).
   
   Comparing reuse-allocation-in-partition (25a4877efd855cb935a592347984e8d32b133723) to 103ebe7 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/103ebe7548db35c37e8ff35fcb02438bcd182fa7..25a4877efd855cb935a592347984e8d32b133723)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark partition_kernels
   shards: 1
   
   ```
   
   </details>
   
   Results will be posted when all workers finish.
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6004705015:start:5acdb496e19021576f110aba6c806016 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on code in PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#discussion_r4178194214


##########
arrow-ord/src/partition.rs:
##########
@@ -166,6 +169,46 @@ fn find_boundaries(v: &dyn Array) -> Result<BooleanBuffer, ArrowError> {
     Ok((0..slice_len).map(|i| !cmp(i, i).is_eq()).collect())
 }
 
+fn convert_boolean_buffer_to_builder(buffer: BooleanBuffer) -> BooleanBufferBuilder {
+    let len = buffer.len();
+
+    // This should be 0, but in case it doesn't simplify the code by doing a copy to a new buffer
+    let buffer = if buffer.offset() != 0 {
+        // Recreate
+        BooleanBuffer::from_iter(buffer.iter())
+    } else {
+        buffer
+    };
+    assert_eq!(buffer.offset(), 0);
+
+    let buffer = buffer.into_inner();
+
+    let mutable = buffer
+        .into_mutable()
+        // This should not happen but just in case copy to owned mutable buffer
+        .unwrap_or_else(|buffer| {
+            let mut mutable = MutableBuffer::new(buffer.len());
+            // SAFETY: We just allocated a buffer of the same length as the original buffer, so it is safe to set the length to the original buffer's length.
+            unsafe { mutable.set_len(buffer.len()) };
+            mutable.as_slice_mut().copy_from_slice(buffer.as_slice());
+
+            mutable
+        });
+
+    BooleanBufferBuilder::new_from_buffer(mutable, len)
+}
+
+fn or(lhs: &mut BooleanBufferBuilder, rhs: BooleanBuffer) {
+    apply_bitwise_binary_op(
+        lhs.as_slice_mut(),
+        0,

Review Comment:
   This is zero since we made sure that the input start at zero



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on code in PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#discussion_r4178192820


##########
arrow-ord/src/partition.rs:
##########
@@ -166,6 +169,46 @@ fn find_boundaries(v: &dyn Array) -> Result<BooleanBuffer, ArrowError> {
     Ok((0..slice_len).map(|i| !cmp(i, i).is_eq()).collect())
 }
 
+fn convert_boolean_buffer_to_builder(buffer: BooleanBuffer) -> BooleanBufferBuilder {
+    let len = buffer.len();
+
+    // This should be 0, but in case it doesn't simplify the code by doing a copy to a new buffer

Review Comment:
   The builder have no notion of offset
   The offset is only for slicing immutable buffers and you need to slice bit wise and not byte wise



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-5981702926

   
   run benchmark partition_kernels
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-6019487762

   @Jefffrey , @alamb can you please merge?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-6004824304

   🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11324#issuecomment-6004705015)
   
   **Target:** partition&#95;kernels
   
   Comparing reuse-allocation-in-partition (25a4877efd855cb935a592347984e8d32b133723) to 103ebe7 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/103ebe7548db35c37e8ff35fcb02438bcd182fa7..25a4877efd855cb935a592347984e8d32b133723)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark partition_kernels
   shards: 1
   
   ```
   
   </details>
   
   <details><summary>Details</summary>
   
   ```
   group                              base                                   changed
   -----                              ----                                   -------
   partition(f64) 2^10                1.00      2.3±0.00µs        ? ?/sec    1.01      2.4±0.01µs        ? ?/sec
   partition(low cardinality) 1024    1.01    521.0±3.53ns        ? ?/sec    1.00    517.4±1.10ns        ? ?/sec
   partition(u8) 2^10                 1.00   1458.1±3.32ns        ? ?/sec    1.02   1484.7±4.04ns        ? ?/sec
   partition(u8) 2^10 with nulls      1.00   1844.8±8.29ns        ? ?/sec    1.04  1922.8±12.07ns        ? ?/sec
   partition(u8) 2^12                 1.01      2.9±0.01µs        ? ?/sec    1.00      2.8±0.00µs        ? ?/sec
   partition(u8) 2^12 with nulls      1.00      3.4±0.01µs        ? ?/sec    1.02      3.5±0.01µs        ? ?/sec
   
   ```
   
   </details>
   
   <details><summary>Per-runner information</summary>
   
   <details><summary>partition&#95;kernels — shard 1/1</summary>
   
   **Node:** gk3-benchmark-cluster-nap-wqu6m05g-249ad343-vfqk
   
   **Instance:** c4a-highcpu-48 (12 vCPU / 65 GiB)
   
   **uname:**
   
   ```text
   Linux bench-c6004705015-3070-6xjlh 6.12.94+ #1 SMP Fri Aug 21 08:00:16 UTC 2026 aarch64 GNU/Linux
   ```
   
   **BENCH_COMMAND:**
   
   ```sh
   cargo bench --features=arrow,async,test_common,experimental,object_store --bench partition_kernels
   ```
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```text
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  48
   On-line CPU(s) list:                     0-47
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     48
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               3 MiB (48 instances)
   L1i cache:                               3 MiB (48 instances)
   L2 cache:                                96 MiB (48 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-47
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 60.0s |
   | Peak memory | 14.0 MiB |
   | Avg memory | 12.0 MiB |
   | CPU user | 56.9s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 60.0s |
   | Peak memory | 14.0 MiB |
   | Avg memory | 11.8 MiB |
   | CPU user | 56.1s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   
   </details>
   
   </details>
   
   </details>
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6004705015:finish:8a394bf7810344e0c55a4bbc07cce1f6 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-6012472432

   I don't think the benchmark is important here as long as there is no major regression


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-6003510348

   🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11324#issuecomment-6003380850)
   
   **Target:** partition&#95;kernels
   
   Comparing reuse-allocation-in-partition (25a4877efd855cb935a592347984e8d32b133723) to 103ebe7 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/103ebe7548db35c37e8ff35fcb02438bcd182fa7..25a4877efd855cb935a592347984e8d32b133723)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark partition_kernels
   shards: 1
   
   ```
   
   </details>
   
   <details><summary>Details</summary>
   
   ```
   group                              base                                   changed
   -----                              ----                                   -------
   partition(f64) 2^10                1.00      2.4±0.01µs        ? ?/sec    1.01      2.4±0.01µs        ? ?/sec
   partition(low cardinality) 1024    1.00    518.5±4.53ns        ? ?/sec    1.01    521.2±1.80ns        ? ?/sec
   partition(u8) 2^10                 1.00   1469.8±9.10ns        ? ?/sec    1.01   1488.4±5.83ns        ? ?/sec
   partition(u8) 2^10 with nulls      1.00  1826.0±12.41ns        ? ?/sec    1.05   1922.1±4.91ns        ? ?/sec
   partition(u8) 2^12                 1.02      2.9±0.01µs        ? ?/sec    1.00      2.8±0.01µs        ? ?/sec
   partition(u8) 2^12 with nulls      1.00      3.4±0.01µs        ? ?/sec    1.02      3.5±0.01µs        ? ?/sec
   
   ```
   
   </details>
   
   <details><summary>Per-runner information</summary>
   
   <details><summary>partition&#95;kernels — shard 1/1</summary>
   
   **Node:** gk3-benchmark-cluster-nap-12napcoj-90fd63dc-7kd9
   
   **Instance:** c4a-highmem-16 (12 vCPU / 65 GiB)
   
   **uname:**
   
   ```text
   Linux bench-c6003380850-3069-fnxfr 6.12.94+ #1 SMP Fri Aug 21 08:00:16 UTC 2026 aarch64 GNU/Linux
   ```
   
   **BENCH_COMMAND:**
   
   ```sh
   cargo bench --features=arrow,async,test_common,experimental,object_store --bench partition_kernels
   ```
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```text
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 60.0s |
   | Peak memory | 14.0 MiB |
   | Avg memory | 11.3 MiB |
   | CPU user | 53.8s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 60.0s |
   | Peak memory | 14.0 MiB |
   | Avg memory | 11.8 MiB |
   | CPU user | 56.1s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   
   </details>
   
   </details>
   
   </details>
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:6003380850:finish:1d2773d2e9a877f1736caa05493857c8 -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on code in PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#discussion_r4177464439


##########
arrow-ord/src/partition.rs:
##########
@@ -166,6 +169,46 @@ fn find_boundaries(v: &dyn Array) -> Result<BooleanBuffer, ArrowError> {
     Ok((0..slice_len).map(|i| !cmp(i, i).is_eq()).collect())
 }
 
+fn convert_boolean_buffer_to_builder(buffer: BooleanBuffer) -> BooleanBufferBuilder {
+    let len = buffer.len();
+
+    // This should be 0, but in case it doesn't simplify the code by doing a copy to a new buffer
+    let buffer = if buffer.offset() != 0 {
+        // Recreate
+        BooleanBuffer::from_iter(buffer.iter())
+    } else {
+        buffer
+    };
+    assert_eq!(buffer.offset(), 0);
+
+    let buffer = buffer.into_inner();
+
+    let mutable = buffer
+        .into_mutable()
+        // This should not happen but just in case copy to owned mutable buffer
+        .unwrap_or_else(|buffer| {
+            let mut mutable = MutableBuffer::new(buffer.len());
+            // SAFETY: We just allocated a buffer of the same length as the original buffer, so it is safe to set the length to the original buffer's length.
+            unsafe { mutable.set_len(buffer.len()) };
+            mutable.as_slice_mut().copy_from_slice(buffer.as_slice());
+
+            mutable
+        });
+
+    BooleanBufferBuilder::new_from_buffer(mutable, len)
+}
+
+fn or(lhs: &mut BooleanBufferBuilder, rhs: BooleanBuffer) {

Review Comment:
   similarly this could be a `BitXor` implementaiton on `BooleanBufferBuilder`



##########
arrow-ord/src/partition.rs:
##########
@@ -166,6 +169,46 @@ fn find_boundaries(v: &dyn Array) -> Result<BooleanBuffer, ArrowError> {
     Ok((0..slice_len).map(|i| !cmp(i, i).is_eq()).collect())
 }
 
+fn convert_boolean_buffer_to_builder(buffer: BooleanBuffer) -> BooleanBufferBuilder {

Review Comment:
   maybe this could be a `From` implementation on `BooleanBufferBuilder`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-5981463551

   🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11324#issuecomment-5981359704)
   
   **Target:** partition&#95;kernels
   
   Comparing reuse-allocation-in-partition (e9e85903403da613e1de85909149943a6f881c8e) to 2166741 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/2166741578fed12de6133cb29f0223456fd32123..e9e85903403da613e1de85909149943a6f881c8e)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark partition_kernels
   shards: 1
   
   ```
   
   </details>
   
   <details><summary>Details</summary>
   
   ```
   group                              base                                   changed
   -----                              ----                                   -------
   partition(f64) 2^10                1.03      2.4±0.02µs        ? ?/sec    1.00      2.3±0.01µs        ? ?/sec
   partition(low cardinality) 1024    1.00    521.7±8.12ns        ? ?/sec    1.08    565.0±1.55ns        ? ?/sec
   partition(u8) 2^10                 1.02  1476.5±17.68ns        ? ?/sec    1.00   1451.8±4.67ns        ? ?/sec
   partition(u8) 2^10 with nulls      1.00  1837.2±13.38ns        ? ?/sec    1.01   1864.5±5.73ns        ? ?/sec
   partition(u8) 2^12                 1.01      2.9±0.02µs        ? ?/sec    1.00      2.9±0.00µs        ? ?/sec
   partition(u8) 2^12 with nulls      1.00      3.5±0.02µs        ? ?/sec    1.00      3.5±0.00µs        ? ?/sec
   
   ```
   
   </details>
   
   <details><summary>Per-runner information</summary>
   
   <details><summary>partition&#95;kernels — shard 1/1</summary>
   
   **Node:** gk3-benchmark-cluster-nap-whizcbau-a77e5bca-z5qt
   
   **Instance:** c4a-highmem-16 (12 vCPU / 65 GiB)
   
   **uname:**
   
   ```text
   Linux bench-c5981359704-3055-kckhx 6.12.94+ #1 SMP Wed Aug 19 07:47:20 UTC 2026 aarch64 GNU/Linux
   ```
   
   **BENCH_COMMAND:**
   
   ```sh
   cargo bench --features=arrow,async,test_common,experimental,object_store --bench partition_kernels
   ```
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```text
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 60.0s |
   | Peak memory | 13.9 MiB |
   | Avg memory | 11.3 MiB |
   | CPU user | 53.8s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 55.0s |
   | Peak memory | 13.9 MiB |
   | Avg memory | 12.1 MiB |
   | CPU user | 53.0s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   
   </details>
   
   </details>
   
   </details>
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:5981359704:finish:979d83fcec62b6582a2b3bcd5cbd544d -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "adriangbot (via GitHub)" <gi...@apache.org>.
adriangbot commented on PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#issuecomment-5981837168

   🤖 Benchmark completed (GKE) | [trigger](https://github.com/apache/arrow-rs/pull/11324#issuecomment-5981702926)
   
   **Target:** partition&#95;kernels
   
   Comparing reuse-allocation-in-partition (e9e85903403da613e1de85909149943a6f881c8e) to 2166741 (merge-base) [diff](https://github.com/apache/arrow-rs/compare/2166741578fed12de6133cb29f0223456fd32123..e9e85903403da613e1de85909149943a6f881c8e)
   
   <details><summary>Run configuration</summary>
   
   ```yaml
   run benchmark partition_kernels
   shards: 1
   
   ```
   
   </details>
   
   <details><summary>Details</summary>
   
   ```
   group                              base                                   changed
   -----                              ----                                   -------
   partition(f64) 2^10                1.03      2.3±0.01µs        ? ?/sec    1.00      2.3±0.00µs        ? ?/sec
   partition(low cardinality) 1024    1.00    525.1±3.38ns        ? ?/sec    1.08    564.8±1.30ns        ? ?/sec
   partition(u8) 2^10                 1.02  1470.4±11.55ns        ? ?/sec    1.00   1448.2±3.48ns        ? ?/sec
   partition(u8) 2^10 with nulls      1.00   1830.3±8.13ns        ? ?/sec    1.02   1870.7±8.86ns        ? ?/sec
   partition(u8) 2^12                 1.00      2.9±0.01µs        ? ?/sec    1.00      2.9±0.00µs        ? ?/sec
   partition(u8) 2^12 with nulls      1.00      3.5±0.01µs        ? ?/sec    1.00      3.5±0.00µs        ? ?/sec
   
   ```
   
   </details>
   
   <details><summary>Per-runner information</summary>
   
   <details><summary>partition&#95;kernels — shard 1/1</summary>
   
   **Node:** gk3-benchmark-cluster-nap-r1moxof8-9281de77-xl5r
   
   **Instance:** c4a-highmem-16 (12 vCPU / 65 GiB)
   
   **uname:**
   
   ```text
   Linux bench-c5981702926-3057-2d499 6.12.94+ #1 SMP Wed Aug 19 07:47:20 UTC 2026 aarch64 GNU/Linux
   ```
   
   **BENCH_COMMAND:**
   
   ```sh
   cargo bench --features=arrow,async,test_common,experimental,object_store --bench partition_kernels
   ```
   
   <details><summary>CPU Details (lscpu)</summary>
   
   ```text
   Architecture:                            aarch64
   CPU op-mode(s):                          64-bit
   Byte Order:                              Little Endian
   CPU(s):                                  16
   On-line CPU(s) list:                     0-15
   Vendor ID:                               ARM
   Model name:                              Neoverse-V2
   Model:                                   1
   Thread(s) per core:                      1
   Core(s) per cluster:                     16
   Socket(s):                               -
   Cluster(s):                              1
   Stepping:                                r0p1
   BogoMIPS:                                2000.00
   Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
   L1d cache:                               1 MiB (16 instances)
   L1i cache:                               1 MiB (16 instances)
   L2 cache:                                32 MiB (16 instances)
   L3 cache:                                80 MiB (1 instance)
   NUMA node(s):                            1
   NUMA node0 CPU(s):                       0-15
   Vulnerability Gather data sampling:      Not affected
   Vulnerability Indirect target selection: Not affected
   Vulnerability Itlb multihit:             Not affected
   Vulnerability L1tf:                      Not affected
   Vulnerability Mds:                       Not affected
   Vulnerability Meltdown:                  Not affected
   Vulnerability Mmio stale data:           Not affected
   Vulnerability Reg file data sampling:    Not affected
   Vulnerability Retbleed:                  Not affected
   Vulnerability Spec rstack overflow:      Not affected
   Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
   Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
   Vulnerability Spectre v2:                Mitigation; CSV2, BHB
   Vulnerability Srbds:                     Not affected
   Vulnerability Tsa:                       Not affected
   Vulnerability Tsx async abort:           Not affected
   Vulnerability Vmscape:                   Not affected
   ```
   
   </details>
   
   <details><summary>Resource Usage</summary>
   
   **base (merge-base)**
   | Metric | Value |
   |--------|-------|
   | Wall time | 60.0s |
   | Peak memory | 13.9 MiB |
   | Avg memory | 11.3 MiB |
   | CPU user | 53.8s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   **branch**
   | Metric | Value |
   |--------|-------|
   | Wall time | 55.0s |
   | Peak memory | 13.9 MiB |
   | Avg memory | 12.1 MiB |
   | CPU user | 53.1s |
   | CPU sys | 0.0s |
   | Peak spill | 0 B |
   
   
   </details>
   
   </details>
   
   </details>
   
   ---
   [File an issue](https://github.com/adriangb/datafusion-benchmarking/issues) against this benchmark runner
   
   <!-- benchmark-request:5981702926:finish:2682504f2276cd14d25a888600aba3fd -->


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] chore: reuse allocations in partition [arrow-rs]

Posted by "rluvaton (via GitHub)" <gi...@apache.org>.
rluvaton commented on code in PR #11324:
URL: https://github.com/apache/arrow-rs/pull/11324#discussion_r4158553171


##########
arrow-ord/src/partition.rs:
##########
@@ -166,6 +169,46 @@ fn find_boundaries(v: &dyn Array) -> Result<BooleanBuffer, ArrowError> {
     Ok((0..slice_len).map(|i| !cmp(i, i).is_eq()).collect())
 }
 
+fn convert_boolean_buffer_to_builder(buffer: BooleanBuffer) -> BooleanBufferBuilder {
+    let len = buffer.len();
+
+    // This should be 0, but in case it doesn't simplify the code by doing a copy to a new buffer
+    let buffer = if buffer.offset() != 0 {
+        // Recreate
+        BooleanBuffer::from_iter(buffer.iter())
+    } else {
+        buffer
+    };
+    assert_eq!(buffer.offset(), 0);
+
+    let buffer = buffer.into_inner();
+
+    let mutable = buffer
+        .into_mutable()
+        // This should not happen but just in case copy to owned mutable buffer
+        .unwrap_or_else(|buffer| {
+            let mut mutable = MutableBuffer::new(buffer.len());
+            // SAFETY: We just allocated a buffer of the same length as the original buffer, so it is safe to set the length to the original buffer's length.
+            unsafe { mutable.set_len(buffer.len()) };
+            mutable.as_slice_mut().copy_from_slice(buffer.as_slice());
+
+            mutable
+        });
+
+    BooleanBufferBuilder::new_from_buffer(mutable, len)
+}
+
+fn or(lhs: &mut BooleanBufferBuilder, rhs: BooleanBuffer) {
+    apply_bitwise_binary_op(
+        lhs.as_slice_mut(),
+        0,
+        rhs.values(),
+        rhs.offset(),
+        rhs.len(),
+        |a, b| a | b,
+    )
+}

Review Comment:
   @alamb This is what I meant in:
   - https://github.com/apache/arrow-rs/pull/8619#issuecomment-5937021914
   
   to keep the code simple I had to extract this



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org